From pje at telecommunity.com  Tue Sep  1 00:06:37 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 31 Aug 2009 18:06:37 -0400
Subject: [Python-Dev] how important is setting co_filename for a module
 being imported to what __file__ is set to?
In-Reply-To: <bbaeab100908311457j67713f3ey88a1995e2c421184@mail.gmail.co
 m>
References: <bbaeab100908301628l7fefbae1rbc7bda460ec15c64@mail.gmail.com>
	<bbaeab100908310927t673c81e1ibe20b50651c74923@mail.gmail.com>
	<ca471dc20908310933v318b3863me0d8e330e18cade1@mail.gmail.com>
	<bbaeab100908310957q4a768945m543f7ec403d68567@mail.gmail.com>
	<ca471dc20908311002v5c36a313sa2d12531b3e15fe0@mail.gmail.com>
	<bbaeab100908311200n66f779e2w29b86144e64eefa6@mail.gmail.com>
	<loom.20090831T212535-621@post.gmane.org>
	<bbaeab100908311259p21837359u436bd16a1a1f6984@mail.gmail.com>
	<bbaeab100908311339u6d39efccp71786ba524add406@mail.gmail.com>
	<20090831215249.A11393A4093@sparrow.telecommunity.com>
	<bbaeab100908311457j67713f3ey88a1995e2c421184@mail.gmail.com>
Message-ID: <20090831220718.44AB53A4093@sparrow.telecommunity.com>

At 02:57 PM 8/31/2009 -0700, Brett Cannon wrote:
>Ignoring that 'new' is not in Python 3.x (luckily 'types' is), I want
>a proper solution that doesn't require reconstructing every code
>object that I happen to import.

Practicality beats purity.  ;-)  (Especially if it allows importlib 
to run on older Pythons.)

Also, surely you're not worried about *performance* here?


From brett at python.org  Tue Sep  1 00:12:43 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 31 Aug 2009 15:12:43 -0700
Subject: [Python-Dev] how important is setting co_filename for a module
	being imported to what __file__ is set to?
In-Reply-To: <20090831220718.44AB53A4093@sparrow.telecommunity.com>
References: <bbaeab100908301628l7fefbae1rbc7bda460ec15c64@mail.gmail.com> 
	<bbaeab100908310957q4a768945m543f7ec403d68567@mail.gmail.com> 
	<ca471dc20908311002v5c36a313sa2d12531b3e15fe0@mail.gmail.com> 
	<bbaeab100908311200n66f779e2w29b86144e64eefa6@mail.gmail.com> 
	<loom.20090831T212535-621@post.gmane.org>
	<bbaeab100908311259p21837359u436bd16a1a1f6984@mail.gmail.com> 
	<bbaeab100908311339u6d39efccp71786ba524add406@mail.gmail.com> 
	<20090831215249.A11393A4093@sparrow.telecommunity.com>
	<bbaeab100908311457j67713f3ey88a1995e2c421184@mail.gmail.com> 
	<20090831220718.44AB53A4093@sparrow.telecommunity.com>
Message-ID: <bbaeab100908311512x3f11e428t9385d5008ac8f2df@mail.gmail.com>

On Mon, Aug 31, 2009 at 15:06, P.J. Eby<pje at telecommunity.com> wrote:
> At 02:57 PM 8/31/2009 -0700, Brett Cannon wrote:
>>
>> Ignoring that 'new' is not in Python 3.x (luckily 'types' is), I want
>> a proper solution that doesn't require reconstructing every code
>> object that I happen to import.
>
> Practicality beats purity. ?;-) ?(Especially if it allows importlib to run
> on older Pythons.)
>

I don't care about making importlib run on older versions of Python
before 3.1. And this is a minor enough thing that I am not worried
about missing in Python 3.1.

> Also, surely you're not worried about *performance* here?

I do care about performance to an extent, but that is not the primary
motivating factor to wanting to go with the marshal API change.

-Brett

From greg at krypto.org  Tue Sep  1 00:07:49 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 31 Aug 2009 15:07:49 -0700
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <loom.20090831T231829-502@post.gmane.org>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> 
	<loom.20090831T231829-502@post.gmane.org>
Message-ID: <52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com>

On Mon, Aug 31, 2009 at 2:20 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Gregory P. Smith <greg <at> krypto.org> writes:
> >
> > food for thought as noticed by a coworker who has been profiling some hot
> code
> to optimize a library...If a function does not have a return statement we
> return
> None.  Ironically this makes the foo2 function below faster than the bar2
> function at least as measured using bytecode size
>
> I would be surprised if this "bytecode size" difference made a significant
> difference in runtimes, given that function call cost should dwarf the
> cumulated
> cost of POP_TOP and LOAD_CONST (two of the simplest opcodes you could
> find).
>
>
the attached sample code repeatably shows that it makes a difference though
its really not much of one (2-3%).

I was just wondering if a bytecode for a superinstruction of the common
sequence:

             6 POP_TOP
              7 LOAD_CONST               0 (None)
             10 RETURN_VALUE

might be worth it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090831/48320427/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: none_speed.py
Type: text/x-python
Size: 619 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090831/48320427/attachment-0001.py>

From collinw at gmail.com  Tue Sep  1 00:28:29 2009
From: collinw at gmail.com (Collin Winter)
Date: Mon, 31 Aug 2009 15:28:29 -0700
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com>
Message-ID: <43aa6ff70908311528w54ac19cft64cfdc7b2094967a@mail.gmail.com>

On Mon, Aug 31, 2009 at 3:07 PM, Gregory P. Smith<greg at krypto.org> wrote:
>
>
> On Mon, Aug 31, 2009 at 2:20 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> Gregory P. Smith <greg <at> krypto.org> writes:
>> >
>> > food for thought as noticed by a coworker who has been profiling some
>> > hot code
>> to optimize a library...If a function does not have a return statement we
>> return
>> None.? Ironically this makes the foo2 function below faster than the bar2
>> function at least as measured using bytecode size
>>
>> I would be surprised if this "bytecode size" difference made a significant
>> difference in runtimes, given that function call cost should dwarf the
>> cumulated
>> cost of POP_TOP and LOAD_CONST (two of the simplest opcodes you could
>> find).
>>
>
> the attached sample code repeatably shows that it makes a difference though
> its really not much of one (2-3%).
>
> I was just wondering if a bytecode for a superinstruction of the common
> sequence:
>
> ???????????? 6 POP_TOP
> ????????????? 7 LOAD_CONST?????????????? 0 (None)
> ???????????? 10 RETURN_VALUE
>
> might be worth it.

I doubt it. You'd save a bit of stack manipulation, but since this
will only appear at the end of a function, I'd be skeptical that this
would make any macrobenchmarks (statistically) significantly faster.

Collin Winter

From benjamin at python.org  Tue Sep  1 00:28:39 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 31 Aug 2009 17:28:39 -0500
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com>
Message-ID: <1afaf6160908311528g14065933k9085bcef9b9ab94@mail.gmail.com>

2009/8/31 Gregory P. Smith <greg at krypto.org>:
> I was just wondering if a bytecode for a superinstruction of the common
> sequence:
>
> ???????????? 6 POP_TOP
> ????????????? 7 LOAD_CONST?????????????? 0 (None)
> ???????????? 10 RETURN_VALUE
>
> might be worth it.

We could just utilize RETURN_VALUE's op argument.



-- 
Regards,
Benjamin

From solipsis at pitrou.net  Tue Sep  1 00:29:48 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 31 Aug 2009 22:29:48 +0000 (UTC)
Subject: [Python-Dev] default of returning None hurts performance?
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com>
Message-ID: <loom.20090901T002507-427@post.gmane.org>

Gregory P. Smith <greg <at> krypto.org> writes:
> 
> I was just wondering if a bytecode for a superinstruction of the common 
sequence:
> 6 POP_TOP
> 7 LOAD_CONST?????????????? 0 (None)
> 10 RETURN_VALUE
> might be worth it.

I think superinstructions in general would be a good thing to experiment, as
wpython showed. Direct addressing (via a pseudo register file combining locals
and constants) would eliminate many bookkeeping-related opcodes in common 
bytecode.

Regards

Antoine.



From python at rcn.com  Tue Sep  1 00:39:47 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 31 Aug 2009 15:39:47 -0700
Subject: [Python-Dev] default of returning None hurts performance?
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com><loom.20090831T231829-502@post.gmane.org><52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com>
	<43aa6ff70908311528w54ac19cft64cfdc7b2094967a@mail.gmail.com>
Message-ID: <F29A3FAC909844BEA9A6C927B0A2F3C8@RaymondLaptop1>



> I was just wondering if a bytecode for a superinstruction of the common
> sequence:
>
> 6 POP_TOP
> 7 LOAD_CONST 0 (None)
> 10 RETURN_VALUE
>
> might be worth it.

[Collin Winter]
> I doubt it. You'd save a bit of stack manipulation, but since this
> will only appear at the end of a function, I'd be skeptical that this
> would make any macrobenchmarks (statistically) significantly faster.

I concur with Collin.  And since it appears only at the end of a function,
the optimization doesn't help inner-loops in a function (where most of
the time usually spent).


Raymond


From greg.ewing at canterbury.ac.nz  Tue Sep  1 02:01:27 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 01 Sep 2009 12:01:27 +1200
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <loom.20090831T231829-502@post.gmane.org>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
Message-ID: <4A9C6457.2060609@canterbury.ac.nz>

Antoine Pitrou wrote:

> Did your coworker run any timings instead of basing his assumptions on bytecode
> size?

In any case, what are you suggesting -- that the last value
returned by a function call in the body should be the
default return value?

I don't think the unpredictability that would introduce
would be a good idea.

-- 
Greg

From greg at krypto.org  Tue Sep  1 03:03:12 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 31 Aug 2009 18:03:12 -0700
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <4A9C6457.2060609@canterbury.ac.nz>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> 
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
Message-ID: <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com>

On Mon, Aug 31, 2009 at 5:01 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> Antoine Pitrou wrote:
>
>  Did your coworker run any timings instead of basing his assumptions on
>> bytecode
>> size?
>>
>
> In any case, what are you suggesting -- that the last value
> returned by a function call in the body should be the
> default return value?
>
> I don't think the unpredictability that would introduce
> would be a good idea.
>

gads no.  we're not shell or perl!  don't do that :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090831/76aa1845/attachment.htm>

From s7v7nislands at gmail.com  Tue Sep  1 05:47:44 2009
From: s7v7nislands at gmail.com (xiaobing jiang)
Date: Tue, 1 Sep 2009 11:47:44 +0800
Subject: [Python-Dev] why different between staticmethod and classmethod on
	non-callable object?
Message-ID: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>

hi all:
In the svn thunk tree, I find in file Object/funcobject.c, the two
functions: sm_init and cm_init have a little different. the cm_init
function will check the object is really callable, but the sm_init
don't.
the code like this:
    if (!PyCallable_Check(callable)) {
        PyErr_Format(PyExc_TypeError, "'%s' object is not callable",
             callable->ob_type->tp_name);
        return -1;
    }

so when use staticmethod and classmethod on a class variable, there
are two different behavior.

class Test(object):
    name = "s7v7nislands"

name = classmethod(Test.name)   # will raise error.
name = staticmethod(Test.name)  #  will not raise error.


My idea is: here, the two functions (or maybe classes) should have the
same behavior).
so is this a bug or something I missing ?

thanks!

s7v7nislands

From amentajo at msu.edu  Tue Sep  1 07:10:35 2009
From: amentajo at msu.edu (Joe Amenta)
Date: Tue, 1 Sep 2009 01:10:35 -0400
Subject: [Python-Dev] 3to2 0.1 alpha 1 released
In-Reply-To: <bbaeab100908311225n6a6993dt8c23349ceefeead3@mail.gmail.com>
References: <4dc473a50908261229i3d0f600dq38286edafc076e48@mail.gmail.com> 
	<4A959A89.4000000@v.loewis.de>
	<4dc473a50908261555o2bc4cdbexff1b2dbabb4bab0@mail.gmail.com> 
	<op.uzjjwsodbrrvlq@double>
	<bbaeab100908311225n6a6993dt8c23349ceefeead3@mail.gmail.com>
Message-ID: <4dc473a50908312210n249faa32xc01c8adc002fa9ac@mail.gmail.com>

Thanks Brett and Sridhar for the info.  3to2's version info has been
PEP386-ified, and the latest version can always be found at
http://pypi.python.org/pypi/3to2/ (the previous link will now generate an
error).

--Joe

On Mon, Aug 31, 2009 at 3:25 PM, Brett Cannon <brett at python.org> wrote:

> On Mon, Aug 31, 2009 at 12:18, Sridhar
> Ratnakumar<SridharR at activestate.com> wrote:
> > On Wed, 26 Aug 2009 15:55:54 -0700, Joe Amenta <amentajo at msu.edu> wrote:
> >
> >> -- 3to2 is now registered with PyPI.  Did I do it right?
> >
> >> http://pypi.python.org/pypi/3to2/0.1%20alpha%201
> >
> > Please fix the version number to not contain any whitespace characters.
> Also
> > set the `version` argument in setup(..) in your setup.py. And then you
> may
> > want to use the `upload` command to upload the new tarball to PyPI. See
> > http://wiki.python.org/moin/Distutils/Tutorial for more details.
>
> See PEP 386 (http://www.python.org/dev/peps/pep-0386/) for what the
> current thinking on version numbers is.
>
> -Brett
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090901/bdc5e704/attachment.htm>

From scott+python-dev at scottdial.com  Tue Sep  1 09:52:51 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 01 Sep 2009 03:52:51 -0400
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <F29A3FAC909844BEA9A6C927B0A2F3C8@RaymondLaptop1>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com><loom.20090831T231829-502@post.gmane.org><52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com>	<43aa6ff70908311528w54ac19cft64cfdc7b2094967a@mail.gmail.com>
	<F29A3FAC909844BEA9A6C927B0A2F3C8@RaymondLaptop1>
Message-ID: <4A9CD2D3.9090806@scottdial.com>

Raymond Hettinger wrote:
>> I was just wondering if a bytecode for a superinstruction of the common
>> sequence:
>>
>> 6 POP_TOP
>> 7 LOAD_CONST 0 (None)
>> 10 RETURN_VALUE
>>
>> might be worth it.
> 
> [Collin Winter]
>> I doubt it. You'd save a bit of stack manipulation, but since this
>> will only appear at the end of a function, I'd be skeptical that this
>> would make any macrobenchmarks (statistically) significantly faster.
> 
> I concur with Collin.  And since it appears only at the end of a function,
> the optimization doesn't help inner-loops in a function (where most of
> the time usually spent).
> 

I fail to understand this crude logic. How often is the inner-loop
really going to solely call C code? Any call to Python in an inner-loop
is going to suffer this penalty on the order of the number of loop
iterations)?

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From ncoghlan at gmail.com  Tue Sep  1 12:36:46 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 01 Sep 2009 20:36:46 +1000
Subject: [Python-Dev] how important is setting co_filename for a	module
 being imported to what __file__ is set to?
In-Reply-To: <4E5EF4D6-7544-4911-9E6E-607D3525ADD9@acm.org>
References: <bbaeab100908301628l7fefbae1rbc7bda460ec15c64@mail.gmail.com>	<1afaf6160908310755vc5d3872i2f7be0ce33c09040@mail.gmail.com>	<loom.20090831T170406-846@post.gmane.org>	<bbaeab100908310927t673c81e1ibe20b50651c74923@mail.gmail.com>	<ca471dc20908310933v318b3863me0d8e330e18cade1@mail.gmail.com>	<bbaeab100908310957q4a768945m543f7ec403d68567@mail.gmail.com>	<ca471dc20908311002v5c36a313sa2d12531b3e15fe0@mail.gmail.com>	<bbaeab100908311200n66f779e2w29b86144e64eefa6@mail.gmail.com>	<loom.20090831T212535-621@post.gmane.org>	<bbaeab100908311259p21837359u436bd16a1a1f6984@mail.gmail.com>	<bbaeab100908311339u6d39efccp71786ba524add406@mail.gmail.com>	<loom.20090831T224716-505@post.gmane.org>
	<4E5EF4D6-7544-4911-9E6E-607D3525ADD9@acm.org>
Message-ID: <4A9CF93E.7040107@gmail.com>

Fred Drake wrote:
> Every time I've been bitten by the wrong co_filename values (usually
> from tracebacks), changing the way marshal creates code objects to use a
> values passed in has been the thing that made the most sense to me.
> 
> The feature request that's involved here, getting correct co_filename
> values, can be implemented in different ways, sure.  This particular
> change produces the least impact in the because it *doesn't* change the
> mutability of code objects.
> 
> I for one appreciate that, mostly because I'm simply wary of making code
> objects mutable in this way having unexpected side effects in some library.

"linecache" was the one that immediately popped into mind for me (not
saying it is affected, just saying it would be the first place I would
like for possible side effects).

I think this is one of those situations where something has behaved a
certain way for so long that it would be really hard to be confident
that we had considered all the possible ramifications of changing it.

Modifying marshal to allow Python code to override the stored value
instead of making it a free-for-all preserves the post-compile
immutability characteristic while still letting Brett make importlib
mimic the behaviour of import.c.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Tue Sep  1 12:58:30 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 01 Sep 2009 20:58:30 +1000
Subject: [Python-Dev] runpy.py
In-Reply-To: <bbaeab100908311220l2162cabfwe9a6ca4f38d4b20c@mail.gmail.com>
References: <332972a20908230909q67517e53y1e65b9155a9fc0b1@mail.gmail.com>
	<4A92B9C3.5040701@simplistix.co.uk>
	<1afaf6160908240907y2bb8fabbud044171de72ae052@mail.gmail.com>
	<4A939F4F.9060306@simplistix.co.uk>
	<1afaf6160908250125h4cc16ff7h2c33f1162cd25cca@mail.gmail.com>
	<4A93A85D.3010108@gmail.com>
	<d38f5330908250850m58acb083i335c1310555509e0@mail.gmail.com>
	<4A951C43.8050707@gmail.com> <4A9BBB21.4040504@simplistix.co.uk>
	<4A9BD1EA.9020204@gmail.com>
	<bbaeab100908311220l2162cabfwe9a6ca4f38d4b20c@mail.gmail.com>
Message-ID: <4A9CFE56.6020108@gmail.com>

Brett Cannon wrote:
> On Mon, Aug 31, 2009 at 06:36, Nick Coghlan<ncoghlan at gmail.com> wrote:
>> That said, while actually ditching the C code might cause an argument,
>> expanding runpy with Python equivalents of the C level functionality
>> (i.e. run script by name, run directory/zipfile by name, '-c' switch,
>> and other odds and ends that I'm probably forgetting right now, with all
>> associated modifications to sys.argv and the __main__ module attributes)
>> should be far less controversial.
> 
> It also has the perk of letting alternative VMs not have to implement
> all of that stuff themselves, potentially helping to unify even the
> command-line interfaces for all the VMs.

I created a tracker item for the idea so I don't forget about it:

http://bugs.python.org/issue6816

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From scott+python-ideas at scottdial.com  Tue Sep  1 09:51:49 2009
From: scott+python-ideas at scottdial.com (Scott Dial)
Date: Tue, 01 Sep 2009 03:51:49 -0400
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <F29A3FAC909844BEA9A6C927B0A2F3C8@RaymondLaptop1>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com><loom.20090831T231829-502@post.gmane.org><52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com>	<43aa6ff70908311528w54ac19cft64cfdc7b2094967a@mail.gmail.com>
	<F29A3FAC909844BEA9A6C927B0A2F3C8@RaymondLaptop1>
Message-ID: <4A9CD295.2090605@scottdial.com>

Raymond Hettinger wrote:
>> I was just wondering if a bytecode for a superinstruction of the common
>> sequence:
>>
>> 6 POP_TOP
>> 7 LOAD_CONST 0 (None)
>> 10 RETURN_VALUE
>>
>> might be worth it.
> 
> [Collin Winter]
>> I doubt it. You'd save a bit of stack manipulation, but since this
>> will only appear at the end of a function, I'd be skeptical that this
>> would make any macrobenchmarks (statistically) significantly faster.
> 
> I concur with Collin.  And since it appears only at the end of a function,
> the optimization doesn't help inner-loops in a function (where most of
> the time usually spent).
> 

I fail to understand this crude logic. How often is the inner-loop
really going to solely call C code? Any call to Python in an inner-loop
is going to suffer this penalty on the order of the number of loop
iterations)?

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From steve at pearwood.info  Tue Sep  1 15:04:50 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 1 Sep 2009 23:04:50 +1000
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <4A9CD295.2090605@scottdial.com>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<F29A3FAC909844BEA9A6C927B0A2F3C8@RaymondLaptop1>
	<4A9CD295.2090605@scottdial.com>
Message-ID: <200909012304.51889.steve@pearwood.info>

On Tue, 1 Sep 2009 05:51:49 pm Scott Dial wrote:
> Raymond Hettinger wrote:
> >> I was just wondering if a bytecode for a superinstruction of the
> >> common sequence:
> >>
> >> 6 POP_TOP
> >> 7 LOAD_CONST 0 (None)
> >> 10 RETURN_VALUE
> >>
> >> might be worth it.
> >
> > [Collin Winter]
> >
> >> I doubt it. You'd save a bit of stack manipulation, but since this
> >> will only appear at the end of a function, I'd be skeptical that
> >> this would make any macrobenchmarks (statistically) significantly
> >> faster.
> >
> > I concur with Collin.  And since it appears only at the end of a
> > function, the optimization doesn't help inner-loops in a function
> > (where most of the time usually spent).
>
> I fail to understand this crude logic. How often is the inner-loop
> really going to solely call C code? Any call to Python in an
> inner-loop is going to suffer this penalty on the order of the number
> of loop iterations)?

Most functions don't suffer this penalty. Consider the following two 
functions:


def g(x):
    return x()

def h(x):
    x()

Now disassemble:

>>> dis.dis(g)
  2           0 LOAD_FAST                0 (x)
              3 CALL_FUNCTION            0
              6 RETURN_VALUE
>>> dis.dis(h)
  2           0 LOAD_FAST                0 (x)
              3 CALL_FUNCTION            0
              6 POP_TOP
              7 LOAD_CONST               0 (None)
             10 RETURN_VALUE

The first doesn't suffer any such default "return None" penalty, and so 
won't gain any performance benefit from optimizing it. It is only the 
subset of functions which don't explicitly return anything which will 
see any potential benefit. Let me call such functions "procedures" to 
avoid confusion with those functions which won't see any benefit.

While procedures may see some benefit, it's a trivial amount, probably 
not worth the extra complexity. According to Gregory's tests, the 
difference is approximately 2% on a trivial do-nothing function. 
According to my tests on my PC, I might hope to save somewhat less than 
0.1 microsecond per procedure call as an absolute saving. As a relative 
saving though, it will most likely be insignificant: for comparison's 
sake, urllib2.Request('http://example.com') takes around 150?s on my 
machine, and math.sin(1.1) around 90?s. For any procedure which does 
non-trivial amounts of work, saving 0.1?s is insignificant, no matter 
how many times it is called inside a loop.


-- 
Steven D'Aprano

From solipsis at pitrou.net  Tue Sep  1 15:25:06 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 01 Sep 2009 15:25:06 +0200
Subject: [Python-Dev] [OT] implicit return values
In-Reply-To: <7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
	<52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com>
	<7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net>
Message-ID: <1251811506.5631.44.camel@localhost>

Le mardi 01 septembre 2009 ? 15:09 +0200, Xavier Morel a ?crit :
> "We" are not Erlang, Smalltalk, OCaml or Haskell either, sadly.

Well, feel free to prefer an unreadable language if you want :)
Having implicit return values is certainly not something which follows
Python's design principles. Even C abandoned the idea.

In any case, this discussion is off-topic for this thread. If you want
to discuss the topic further, you can post to python-list or
python-ideas (it will most certainly be shot down anyway).



From catch-all at masklinn.net  Tue Sep  1 15:09:23 2009
From: catch-all at masklinn.net (Xavier Morel)
Date: Tue, 1 Sep 2009 15:09:23 +0200
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
	<52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com>
Message-ID: <7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net>


On 1 Sep 2009, at 03:03 , Gregory P. Smith wrote:

> On Mon, Aug 31, 2009 at 5:01 PM, Greg Ewing <greg.ewing at canterbury.ac.nz 
> >wrote:
>
>> Antoine Pitrou wrote:
>>
>> Did your coworker run any timings instead of basing his assumptions  
>> on
>>> bytecode
>>> size?
>>>
>>
>> In any case, what are you suggesting -- that the last value
>> returned by a function call in the body should be the
>> default return value?
>>
>> I don't think the unpredictability that would introduce
>> would be a good idea.
>>
>
> gads no.  we're not shell or perl!  don't do that :)
"We" are not Erlang, Smalltalk, OCaml or Haskell either, sadly.

From catch-all at masklinn.net  Tue Sep  1 15:12:34 2009
From: catch-all at masklinn.net (Xavier Morel)
Date: Tue, 1 Sep 2009 15:12:34 +0200
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <4A9C6457.2060609@canterbury.ac.nz>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
Message-ID: <D06CB4B3-5717-4E81-8E5F-6DEAB824C833@masklinn.net>

On 1 Sep 2009, at 02:01 , Greg Ewing wrote:
> I don't think the unpredictability that would introduce
> would be a good idea.
I fail to grasp the unpredictability of "the last expression evaluated  
in the body of a function is its return value".

It couldn't work in Python because statements aren't expressions,  
therefore I think

     def foo():
         if cond:
             3
         else:
             4

would break (given if:else: doesn't return a value, the function  
couldn't have a return value), but in languages where everything is an  
expression (where if:else: does return a value) there's nothing  
unpredictable about it.

From benjamin at python.org  Tue Sep  1 16:21:05 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 1 Sep 2009 09:21:05 -0500
Subject: [Python-Dev] why different between staticmethod and classmethod
	on non-callable object?
In-Reply-To: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>
Message-ID: <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>

2009/8/31 xiaobing jiang <s7v7nislands at gmail.com>:
> My idea is: here, the two functions (or maybe classes) should have the
> same behavior).
> so is this a bug or something I missing ?

I think they should both not check their arguments in __init__ to
allow for duck typing.



-- 
Regards,
Benjamin

From python at rcn.com  Tue Sep  1 17:49:07 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 1 Sep 2009 08:49:07 -0700
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <4A9CD295.2090605@scottdial.com>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com><loom.20090831T231829-502@post.gmane.org><52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com>	<43aa6ff70908311528w54ac19cft64cfdc7b2094967a@mail.gmail.com>
	<F29A3FAC909844BEA9A6C927B0A2F3C8@RaymondLaptop1>
	<4A9CD295.2090605@scottdial.com>
Message-ID: <CB3BBC37-B57F-489D-A56F-A67121090A66@rcn.com>


>
> I fail to understand this crude logic.

If a function contains any looping (C or otherwise) or does much in  
the way of meaningful work, then it's unlikely that the single POP_TOP  
associated with an implied "return None" is taking much of the total  
runtime.


Raymond

From xavier.morel at masklinn.net  Tue Sep  1 16:36:28 2009
From: xavier.morel at masklinn.net (Xavier Morel)
Date: Tue, 1 Sep 2009 16:36:28 +0200
Subject: [Python-Dev] [OT] implicit return values
In-Reply-To: <1251811506.5631.44.camel@localhost>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
	<52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com>
	<7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net>
	<1251811506.5631.44.camel@localhost>
Message-ID: <B928ABA7-8AA6-43E1-8AA5-F21F3F9C75E6@masklinn.net>

On 1 Sep 2009, at 15:25 , Antoine Pitrou wrote:
Le mardi 01 septembre 2009 ? 15:09 +0200, Xavier Morel a ?crit :
>> "We" are not Erlang, Smalltalk, OCaml or Haskell either, sadly.
>
> Well, feel free to prefer an unreadable language if you want :)
Smalltalk or Haskell are hardly inherently unreadable. And it's not  
like the Python community never lifts features from such languages, so  
obviously they do (some at least) things right.

> it will most certainly be shot down anyway
Yep, so there's not much point in bringing it up there.



From mcguire at google.com  Tue Sep  1 18:40:17 2009
From: mcguire at google.com (Jake McGuire)
Date: Tue, 1 Sep 2009 09:40:17 -0700
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <4A9C6457.2060609@canterbury.ac.nz>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
Message-ID: <77c780b40909010940v4c9bc50bs59cf82742b6409f@mail.gmail.com>

On Mon, Aug 31, 2009 at 5:01 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> Antoine Pitrou wrote:
>
>  Did your coworker run any timings instead of basing his assumptions on
>> bytecode
>> size?
>>
>
> In any case, what are you suggesting -- that the last value
> returned by a function call in the body should be the
> default return value?
>
> I don't think the unpredictability that would introduce
> would be a good idea.
>

Never mind the fact that there is a lot of Python code out there that
assumes that functions without an explicit return value, return None.

This is pretty fundamental to the Python API.

-jake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090901/114fa872/attachment.htm>

From rnk at mit.edu  Tue Sep  1 20:01:35 2009
From: rnk at mit.edu (Reid Kleckner)
Date: Tue, 1 Sep 2009 14:01:35 -0400
Subject: [Python-Dev] Whether to call Py_Finalize when exiting from the
	child process of a fork from a spawned thread
Message-ID: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com>

Hi all,

I'm working on http://bugs.python.org/issue6642 for unladen swallow
(because it happens to bite us in a weird way), and Jeff Yasskin told
me to ask python-dev what the proper behavior should be at exit from
the child process of a fork from a spawned thread.

Right now, it seems that there is an assumption that when exiting the
process, control will always return through Py_Main, which in turn
calls Py_Finalize, to do things like GC and calling atexit handlers.
Normally, if you fork a process from the main thread, this assumption
will remain true, because main and Py_Main are still at the bottom of
the stack in the child process.  However, if you fork from a spawned
thread, then the root of the stack will be the thread bootstrap
routine for the platform's pythreads implementation.

On one hand, you may not want to call the user's atexit handlers
multiple times from different processes if they have externally
visible effects.  On the other hand, people seem to assume that
Py_Finalize will be called at process exit to do various cleanups.  On
the third hand, maybe Python could just clear out all the atexit
handlers in the child after a fork.  So what should the correct
behavior be?

Thanks,
Reid

From brett at python.org  Tue Sep  1 20:36:13 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 1 Sep 2009 11:36:13 -0700
Subject: [Python-Dev] why different between staticmethod and classmethod
	on non-callable object?
In-Reply-To: <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> 
	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>
Message-ID: <bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>

On Tue, Sep 1, 2009 at 07:21, Benjamin Peterson<benjamin at python.org> wrote:
> 2009/8/31 xiaobing jiang <s7v7nislands at gmail.com>:
>> My idea is: here, the two functions (or maybe classes) should have the
>> same behavior).
>> so is this a bug or something I missing ?
>
> I think they should both not check their arguments in __init__ to
> allow for duck typing.

But what is the point of wrapping something with classmethod or
staticmethod that can't be called? It isn't like it is checking
explicitly for a function or method, just that it can be called which
seems reasonable to me (unless PyCallable_Check() is as off as
callable() was).

-Brett

From martin at v.loewis.de  Tue Sep  1 20:58:20 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 01 Sep 2009 20:58:20 +0200
Subject: [Python-Dev] Whether to call Py_Finalize when exiting from the
 child process of a fork from a spawned thread
In-Reply-To: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com>
References: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com>
Message-ID: <4A9D6ECC.1030201@v.loewis.de>

> On one hand, you may not want to call the user's atexit handlers
> multiple times from different processes if they have externally
> visible effects.  On the other hand, people seem to assume that
> Py_Finalize will be called at process exit to do various cleanups.  On
> the third hand, maybe Python could just clear out all the atexit
> handlers in the child after a fork.  So what should the correct
> behavior be?

Standard POSIX fork semantics should be a guidance. IIUC, termination
of the last thread is equivalent to calling exit(0) (although return
from main() still means that exit is invoked right away, and the return
value of main is the exit code - right?). Calling exit means to call
all exit handlers.

So to match POSIX semantics, we should also call the exit handlers in
the child process (along with invoking a final garbage collection etc.)

Regards,
Martin

From rnk at mit.edu  Tue Sep  1 21:05:04 2009
From: rnk at mit.edu (Reid Kleckner)
Date: Tue, 1 Sep 2009 15:05:04 -0400
Subject: [Python-Dev] Whether to call Py_Finalize when exiting from the
	child process of a fork from a spawned thread
In-Reply-To: <4A9D6ECC.1030201@v.loewis.de>
References: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com> 
	<4A9D6ECC.1030201@v.loewis.de>
Message-ID: <9a9942200909011205o16734de0sbcea5b9218bacc04@mail.gmail.com>

On Tue, Sep 1, 2009 at 2:58 PM, "Martin v. L?wis"<martin at v.loewis.de> wrote:
>> On one hand, you may not want to call the user's atexit handlers
>> multiple times from different processes if they have externally
>> visible effects. ?On the other hand, people seem to assume that
>> Py_Finalize will be called at process exit to do various cleanups. ?On
>> the third hand, maybe Python could just clear out all the atexit
>> handlers in the child after a fork. ?So what should the correct
>> behavior be?
>
> Standard POSIX fork semantics should be a guidance. IIUC, termination
> of the last thread is equivalent to calling exit(0) (although return
> from main() still means that exit is invoked right away, and the return
> value of main is the exit code - right?). Calling exit means to call
> all exit handlers.

It depends, there is also _exit, which exists solely for the purpose
of working around exit handlers being called from a forked child
process at exit.  Which semantics should Python have?  In my opinion,
it is more obvious that the user's handlers would be called than not,
so I agree with you.

Reid

From martin at v.loewis.de  Tue Sep  1 21:48:41 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 01 Sep 2009 21:48:41 +0200
Subject: [Python-Dev] Whether to call Py_Finalize when exiting from the
 child process of a fork from a spawned thread
In-Reply-To: <9a9942200909011205o16734de0sbcea5b9218bacc04@mail.gmail.com>
References: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com>
	<4A9D6ECC.1030201@v.loewis.de>
	<9a9942200909011205o16734de0sbcea5b9218bacc04@mail.gmail.com>
Message-ID: <4A9D7A99.8040804@v.loewis.de>

>> Standard POSIX fork semantics should be a guidance. IIUC, termination
>> of the last thread is equivalent to calling exit(0) (although return
>> from main() still means that exit is invoked right away, and the return
>> value of main is the exit code - right?). Calling exit means to call
>> all exit handlers.
> 
> It depends, there is also _exit, which exists solely for the purpose
> of working around exit handlers being called from a forked child
> process at exit.

I don't think so. There are other cases where you don't want to run
atexit handlers, eg. in a segfault signal handler. _exit has been
there "forever", I believe (of course, so did fork()).

>  Which semantics should Python have?  In my opinion,
> it is more obvious that the user's handlers would be called than not,
> so I agree with you.

POSIX says that ending the last thread is equivalent to exit(0),
and this is what Python should also do when the last thread ends.

This is independent of somebody calling _exit(), which should have
the very effect that calling _exit has (i.e. immediate process
termination).

Regards,
Martin

From benjamin at python.org  Tue Sep  1 23:54:17 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 1 Sep 2009 16:54:17 -0500
Subject: [Python-Dev] why different between staticmethod and classmethod
	on non-callable object?
In-Reply-To: <bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>
	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>
	<bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>
Message-ID: <1afaf6160909011454t2e4fa960k74783be550a50f95@mail.gmail.com>

2009/9/1 Brett Cannon <brett at python.org>:
> On Tue, Sep 1, 2009 at 07:21, Benjamin Peterson<benjamin at python.org> wrote:
>> 2009/8/31 xiaobing jiang <s7v7nislands at gmail.com>:
>>> My idea is: here, the two functions (or maybe classes) should have the
>>> same behavior).
>>> so is this a bug or something I missing ?
>>
>> I think they should both not check their arguments in __init__ to
>> allow for duck typing.
>
> But what is the point of wrapping something with classmethod or
> staticmethod that can't be called? It isn't like it is checking
> explicitly for a function or method, just that it can be called which
> seems reasonable to me (unless PyCallable_Check() is as off as
> callable() was).

Well, if checking if tp_call is not NULL is as bad as callable, then yes.

I don't see any reason to use staticmethod or classmethod with a
non-callable, but to be consistent, I would, given the choice between
removing code and adding another type check, perfer to remove a type
check.



-- 
Regards,
Benjamin

From greg.ewing at canterbury.ac.nz  Wed Sep  2 00:04:46 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 02 Sep 2009 10:04:46 +1200
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <D06CB4B3-5717-4E81-8E5F-6DEAB824C833@masklinn.net>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
	<D06CB4B3-5717-4E81-8E5F-6DEAB824C833@masklinn.net>
Message-ID: <4A9D9A7E.5080209@canterbury.ac.nz>

Xavier Morel wrote:

> I fail to grasp the unpredictability of "the last expression evaluated  
> in the body of a function is its return value".

It's unpredictable in the sense that if you're writing
a function that's not intended to return a value, you're
not thinking about what the last call you make in the
function returns, so to a first approximation it's just
some random value.

I often write code that makes use of the fact that falling
off the end of a function returns None. This has been a
documented part of the Python language from the beginning,
and changing it would break a lot of code for no good
reason.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Wed Sep  2 00:10:25 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 02 Sep 2009 10:10:25 +1200
Subject: [Python-Dev] [OT] implicit return values
In-Reply-To: <B928ABA7-8AA6-43E1-8AA5-F21F3F9C75E6@masklinn.net>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
	<52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com>
	<7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net>
	<1251811506.5631.44.camel@localhost>
	<B928ABA7-8AA6-43E1-8AA5-F21F3F9C75E6@masklinn.net>
Message-ID: <4A9D9BD1.8000501@canterbury.ac.nz>

> Le mardi 01 septembre 2009 ? 15:09 +0200, Xavier Morel a ?crit :
> 
>>> "We" are not Erlang, Smalltalk, OCaml or Haskell either, sadly.

IIRC, the default return value of a Smalltalk method is
self, not the last thing evaluated.

(And no, that's not going to happen in Python either --
the BDFL has rejected similar suggestions on previous
occasions.)

-- 
Greg

From brett at python.org  Wed Sep  2 00:13:20 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 1 Sep 2009 15:13:20 -0700
Subject: [Python-Dev] why different between staticmethod and classmethod
	on non-callable object?
In-Reply-To: <1afaf6160909011454t2e4fa960k74783be550a50f95@mail.gmail.com>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> 
	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> 
	<bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com> 
	<1afaf6160909011454t2e4fa960k74783be550a50f95@mail.gmail.com>
Message-ID: <bbaeab100909011513i6d07f7f9vd1873a48d8b5bd83@mail.gmail.com>

On Tue, Sep 1, 2009 at 14:54, Benjamin Peterson<benjamin at python.org> wrote:
> 2009/9/1 Brett Cannon <brett at python.org>:
>> On Tue, Sep 1, 2009 at 07:21, Benjamin Peterson<benjamin at python.org> wrote:
>>> 2009/8/31 xiaobing jiang <s7v7nislands at gmail.com>:
>>>> My idea is: here, the two functions (or maybe classes) should have the
>>>> same behavior).
>>>> so is this a bug or something I missing ?
>>>
>>> I think they should both not check their arguments in __init__ to
>>> allow for duck typing.
>>
>> But what is the point of wrapping something with classmethod or
>> staticmethod that can't be called? It isn't like it is checking
>> explicitly for a function or method, just that it can be called which
>> seems reasonable to me (unless PyCallable_Check() is as off as
>> callable() was).
>
> Well, if checking if tp_call is not NULL is as bad as callable, then yes.
>
> I don't see any reason to use staticmethod or classmethod with a
> non-callable, but to be consistent, I would, given the choice between
> removing code and adding another type check, perfer to remove a type
> check.

Fine by me. I guess it will just fail later with a slightly more
cryptic exception.

-Brett

From greg.ewing at canterbury.ac.nz  Wed Sep  2 00:15:17 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 02 Sep 2009 10:15:17 +1200
Subject: [Python-Dev] Whether to call Py_Finalize when exiting from the
 child process of a fork from a spawned thread
In-Reply-To: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com>
References: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com>
Message-ID: <4A9D9CF5.802@canterbury.ac.nz>

Reid Kleckner wrote:

> On one hand, you may not want to call the user's atexit handlers
> multiple times from different processes if they have externally
> visible effects.  On the other hand, people seem to assume that
> Py_Finalize will be called at process exit to do various cleanups.  On
> the third hand, maybe Python could just clear out all the atexit
> handlers in the child after a fork.  So what should the correct
> behavior be?

Seems to me there's no single answer to that, because some
kinds of atexit handlers may need to be called in child
processes, while others may need *not* to be.

Maybe we need a flag when registering an atexit handler
to indicate whether it should be kept across forks?

-- 
Greg

From greg.ewing at canterbury.ac.nz  Wed Sep  2 00:28:06 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 02 Sep 2009 10:28:06 +1200
Subject: [Python-Dev] why different between staticmethod and classmethod
 on non-callable object?
In-Reply-To: <bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>
	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>
	<bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>
Message-ID: <4A9D9FF6.3090408@canterbury.ac.nz>

Brett Cannon wrote:
> It isn't like it is checking
> explicitly for a function or method, just that it can be called which
> seems reasonable to me (unless PyCallable_Check() is as off as
> callable() was).

I think it just checks that there's something in the
tp_call slot, which is reasonable -- if it's empty,
there's no way that calling the object could ever
succeed, so you might as well fail early.

-- 
Greg

From benjamin at python.org  Wed Sep  2 00:31:26 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 1 Sep 2009 17:31:26 -0500
Subject: [Python-Dev] why different between staticmethod and classmethod
	on non-callable object?
In-Reply-To: <4A9D9FF6.3090408@canterbury.ac.nz>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>
	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>
	<bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>
	<4A9D9FF6.3090408@canterbury.ac.nz>
Message-ID: <1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com>

2009/9/1 Greg Ewing <greg.ewing at canterbury.ac.nz>:
> Brett Cannon wrote:
>>
>> It isn't like it is checking
>> explicitly for a function or method, just that it can be called which
>> seems reasonable to me (unless PyCallable_Check() is as off as
>> callable() was).
>
> I think it just checks that there's something in the
> tp_call slot, which is reasonable -- if it's empty,
> there's no way that calling the object could ever
> succeed, so you might as well fail early.

It depends on whether you're keeping the "callable" object around or
not. Somebody could add a __call__ method later.



-- 
Regards,
Benjamin

From greg.ewing at canterbury.ac.nz  Wed Sep  2 00:36:18 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 02 Sep 2009 10:36:18 +1200
Subject: [Python-Dev] why different between staticmethod and classmethod
 on non-callable object?
In-Reply-To: <1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>
	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>
	<bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>
	<4A9D9FF6.3090408@canterbury.ac.nz>
	<1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com>
Message-ID: <4A9DA1E2.8000801@canterbury.ac.nz>

Benjamin Peterson wrote:

> It depends on whether you're keeping the "callable" object around or
> not. Somebody could add a __call__ method later.

Good point. Removing the check sounds like the
right thing to do, then.

-- 
Greg

From python at rcn.com  Wed Sep  2 01:33:39 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 1 Sep 2009 16:33:39 -0700
Subject: [Python-Dev] why different between staticmethod and classmethod
	on non-callable object?
In-Reply-To: <1afaf6160909011454t2e4fa960k74783be550a50f95@mail.gmail.com>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>
	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>
	<bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>
	<1afaf6160909011454t2e4fa960k74783be550a50f95@mail.gmail.com>
Message-ID: <FBDA078F-BDD6-488B-B26A-CE70FE0FF993@rcn.com>


On Sep 1, 2009, at 2:54 PM, Benjamin Peterson wrote:

> 2009/9/1 Brett Cannon <brett at python.org>:
>> On Tue, Sep 1, 2009 at 07:21, Benjamin  
>> Peterson<benjamin at python.org> wrote:
>>> 2009/8/31 xiaobing jiang <s7v7nislands at gmail.com>:
>>>> My idea is: here, the two functions (or maybe classes) should  
>>>> have the
>>>> same behavior).
>>>> so is this a bug or something I missing ?
>>>
>>> I think they should both not check their arguments in __init__ to
>>> allow for duck typing.
>>
>> But what is the point of wrapping something with classmethod or
>> staticmethod that can't be called? It isn't like it is checking
>> explicitly for a function or method, just that it can be called which
>> seems reasonable to me (unless PyCallable_Check() is as off as
>> callable() was).
>
> Well, if checking if tp_call is not NULL is as bad as callable, then  
> yes.
>
> I don't see any reason to use staticmethod or classmethod with a
> non-callable, but to be consistent, I would, given the choice between
> removing code and adding another type check, perfer to remove a type
> check.
>

Removing the type check is also my preference.



Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090901/3e8e15b4/attachment.htm>

From tjreedy at udel.edu  Wed Sep  2 02:50:37 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 01 Sep 2009 20:50:37 -0400
Subject: [Python-Dev] why different between staticmethod and classmethod
 on non-callable object?
In-Reply-To: <4A9DA1E2.8000801@canterbury.ac.nz>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>	<bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>	<4A9D9FF6.3090408@canterbury.ac.nz>	<1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com>
	<4A9DA1E2.8000801@canterbury.ac.nz>
Message-ID: <h7kfgt$2os$1@ger.gmane.org>

Greg Ewing wrote:
> Benjamin Peterson wrote:
> 
>> It depends on whether you're keeping the "callable" object around or
>> not. Somebody could add a __call__ method later.
> 
> Good point. Removing the check sounds like the
> right thing to do, then.

Both classmethod & staticmethod are documented as having a *function* 
(callable, as I interprete that) as their single argument. Seems 
reasonable to me. Turning the argument into a function after the fact 
seems like a really esoteric use case.


From pje at telecommunity.com  Wed Sep  2 03:27:08 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 01 Sep 2009 21:27:08 -0400
Subject: [Python-Dev] why different between staticmethod and classmethod
 on non-callable object?
In-Reply-To: <h7kfgt$2os$1@ger.gmane.org>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>
	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>
	<bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>
	<4A9D9FF6.3090408@canterbury.ac.nz>
	<1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com>
	<4A9DA1E2.8000801@canterbury.ac.nz> <h7kfgt$2os$1@ger.gmane.org>
Message-ID: <20090902012752.3AFBB3A4093@sparrow.telecommunity.com>

At 08:50 PM 9/1/2009 -0400, Terry Reedy wrote:
>Greg Ewing wrote:
>>Benjamin Peterson wrote:
>>
>>>It depends on whether you're keeping the "callable" object around or
>>>not. Somebody could add a __call__ method later.
>>Good point. Removing the check sounds like the
>>right thing to do, then.
>
>Both classmethod & staticmethod are documented as having a 
>*function* (callable, as I interprete that) as their single 
>argument. Seems reasonable to me. Turning the argument into a 
>function after the fact seems like a really esoteric use case.

The main use case for staticmethod is to prevent __get__ from being 
called on an object retrieved from a class or an instance.  It just 
happens that the most common types of objects you'd want to do that 
on are functions.

However, if for some reason you intend to make a *descriptor* 
available as an attribute (via a class default), then wrapping it 
with staticmethod is the only easy way to do it.

For example, if you're writing a class whose instances have an 
attribute that holds a "property" instance, and you want to provide a 
class-level default, the simplest way to do it is to wrap the default 
property instance with staticmethod, so that it's not treated as a 
property of the class/instance.

(Property instances are of course not callable.)


From tjreedy at udel.edu  Wed Sep  2 07:53:05 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 02 Sep 2009 01:53:05 -0400
Subject: [Python-Dev] why different between staticmethod and classmethod
 on non-callable object?
In-Reply-To: <20090902012752.3AFBB3A4093@sparrow.telecommunity.com>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>	<bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>	<4A9D9FF6.3090408@canterbury.ac.nz>	<1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com>	<4A9DA1E2.8000801@canterbury.ac.nz>
	<h7kfgt$2os$1@ger.gmane.org>
	<20090902012752.3AFBB3A4093@sparrow.telecommunity.com>
Message-ID: <h7l180$8na$1@ger.gmane.org>

P.J. Eby wrote:
> At 08:50 PM 9/1/2009 -0400, Terry Reedy wrote:
>> Greg Ewing wrote:
>>> Benjamin Peterson wrote:
>>>
>>>> It depends on whether you're keeping the "callable" object around or
>>>> not. Somebody could add a __call__ method later.
>>> Good point. Removing the check sounds like the
>>> right thing to do, then.
>>
>> Both classmethod & staticmethod are documented as having a *function* 
>> (callable, as I interprete that) as their single argument. Seems 
>> reasonable to me. Turning the argument into a function after the fact 
>> seems like a really esoteric use case.
> 
> The main use case for staticmethod is to prevent __get__ from being 
> called on an object retrieved from a class or an instance.  It just 
> happens that the most common types of objects you'd want to do that on 
> are functions.

If so, then it is mis-named and mis-documented, and it seems to me that 
there is *a* rationale (not necessarily determinative) for the current 
difference in implementation.

> However, if for some reason you intend to make a *descriptor* available 
> as an attribute (via a class default), then wrapping it with 
> staticmethod is the only easy way to do it.
> 
> For example, if you're writing a class whose instances have an attribute 
> that holds a "property" instance, and you want to provide a class-level 
> default, the simplest way to do it is to wrap the default property 
> instance with staticmethod, so that it's not treated as a property of 
> the class/instance.
> 
> (Property instances are of course not callable.)

So that it is a 'static object' in some sense, but not a static *method*.

Thanks for the clarification and example.

tjr


From s7v7nislands at gmail.com  Wed Sep  2 10:24:30 2009
From: s7v7nislands at gmail.com (xiaobing jiang)
Date: Wed, 2 Sep 2009 16:24:30 +0800
Subject: [Python-Dev] why different between staticmethod and classmethod
	on non-callable object?
In-Reply-To: <20090902012752.3AFBB3A4093@sparrow.telecommunity.com>
References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com>
	<1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com>
	<bbaeab100909011136u7fa1e730sbecbd48deb4769d4@mail.gmail.com>
	<4A9D9FF6.3090408@canterbury.ac.nz>
	<1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com>
	<4A9DA1E2.8000801@canterbury.ac.nz> <h7kfgt$2os$1@ger.gmane.org>
	<20090902012752.3AFBB3A4093@sparrow.telecommunity.com>
Message-ID: <8f43df260909020124u19071822yd35ef884f08da77@mail.gmail.com>

the three types: function, classmethod, staticmethod are descriptors.
but staticmethod's __get__  return the orignal value, others return
object of instancemethod. (from souce in Objects/funcobject.c)
so the staticmethod just like a wrap that make the wrapped object
'frozen'. like in your example.
is it right? the name and document are mis-understanded.

thanks


2009/9/2 P.J. Eby <pje at telecommunity.com>:
> At 08:50 PM 9/1/2009 -0400, Terry Reedy wrote:
>>
>> Greg Ewing wrote:
>>>
>>> Benjamin Peterson wrote:
>>>
>>>> It depends on whether you're keeping the "callable" object around or
>>>> not. Somebody could add a __call__ method later.
>>>
>>> Good point. Removing the check sounds like the
>>> right thing to do, then.
>>
>> Both classmethod & staticmethod are documented as having a *function*
>> (callable, as I interprete that) as their single argument. Seems reasonable
>> to me. Turning the argument into a function after the fact seems like a
>> really esoteric use case.
>
> The main use case for staticmethod is to prevent __get__ from being called
> on an object retrieved from a class or an instance. ?It just happens that
> the most common types of objects you'd want to do that on are functions.
>
> However, if for some reason you intend to make a *descriptor* available as
> an attribute (via a class default), then wrapping it with staticmethod is
> the only easy way to do it.
>
> For example, if you're writing a class whose instances have an attribute
> that holds a "property" instance, and you want to provide a class-level
> default, the simplest way to do it is to wrap the default property instance
> with staticmethod, so that it's not treated as a property of the
> class/instance.
>
> (Property instances are of course not callable.)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/s7v7nislands%40gmail.com
>

From catch-all at masklinn.net  Wed Sep  2 11:46:31 2009
From: catch-all at masklinn.net (Xavier Morel)
Date: Wed, 2 Sep 2009 11:46:31 +0200
Subject: [Python-Dev] [OT] implicit return values
In-Reply-To: <4A9D9BD1.8000501@canterbury.ac.nz>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
	<52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com>
	<7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net>
	<1251811506.5631.44.camel@localhost>
	<B928ABA7-8AA6-43E1-8AA5-F21F3F9C75E6@masklinn.net>
	<4A9D9BD1.8000501@canterbury.ac.nz>
Message-ID: <E8E6089B-B8E6-4164-95DC-2B6D7F8CC0B0@masklinn.net>

On 2 Sep 2009, at 00:10 , Greg Ewing wrote:
Le mardi 01 septembre 2009 ? 15:09 +0200, Xavier Morel a ?crit :
>>>> "We" are not Erlang, Smalltalk, OCaml or Haskell either, sadly.
>
> IIRC, the default return value of a Smalltalk method is
> self, not the last thing evaluated.
Methods yes (and that's one of the few Smalltalk design "features" I  
consider truly dumb, considering it has message cascading), but I was  
referring to blocks rather than methods and the return value of blocks  
is the last evaluated expression.

> (And no, that's not going to happen in Python either --
> the BDFL has rejected similar suggestions on previous
> occasions.)
I know.

From rob.cliffe at btinternet.com  Wed Sep  2 12:15:39 2009
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Wed, 2 Sep 2009 11:15:39 +0100
Subject: [Python-Dev] Decorator syntax
Message-ID: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>

Hi All,
This is my first post to python-dev so I will briefly introduce myself:  My name is Rob Cliffe and I am a commercial programmer living in London, UK.  I have some 30 years of programming experience but have only been using Python for a couple of years.
First I want to say what a fantastic language Python is.  It is THE best language for development in my opinion, and a joy to use.

My specific issue:
I eventually got my head round decorator syntax and realised that what came after the '@' was (basically) a function that took a function as argument and returned a function as result.
However it seems to me unPythonesque (i.e. an exception to Python's normal consistency) that the syntax of what follows the '@' should be restricted to either a single (function) identifier or a single (function) identifier with an argument list.
The example I tried, which seems not an unreasonable sort of thing to do, was along the lines of:

def deco1(func):
    <deco1-suite>
def deco2(func):
    <deco2-suite>

DecoList = [deco1, deco2]

@DecoList[0]    # NO - CAUSES SYNTAX ERROR
def foo():
    pass

I am sure other guys have their own examples.

I am of course not the first person to raise this issue, and I see that Guido has a "gut feeling" against allowing a general expression after the '@'.

BUT - a general expression can be "smuggled in" very easily as a function argument:

def Identity(x): return x

@Identity(DecoList[0])    # THIS WORKS
def foo():
    pass

So - the syntax restriction seems not only inconsistent, but pointless; it doesn't forbid anything, but merely means we have to do it in a slightly convoluted (unPythonesque) way.  So please, Guido, will you reconsider?

Best wishes
Rob Cliffe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090902/f95e8e03/attachment.htm>

From cmjohnson.mailinglist at gmail.com  Wed Sep  2 12:34:49 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Wed, 2 Sep 2009 00:34:49 -1000
Subject: [Python-Dev] Decorator syntax
In-Reply-To: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>
References: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>
Message-ID: <3bdda690909020334m4c81cecdm8f57dce54c2da398@mail.gmail.com>

Crossposting to Python-ideas,

I asked for the same change to the grammar a couple months back on
python-ideas.

See http://mail.python.org/pipermail/python-ideas/2009-February/thread.html#2787

I'm all for it, but you'll have to convince Guido that this won't
result in confusing to read code. My own examples, unfortunately did
not advance your cause, as Guido explained, "My brain hurts trying to
understand all this. I don't think this bodes well as a use case for a
proposed feature." :-D The trouble is that I was using lambdas upon
lambdas to do all kinds of Ruby block-esque tricks. OTOH, if you come
up with some simple, clear use cases though, and I think he might
still be persuadable to make a simple change to the grammar.


? Carl Johnson

Rob Cliffe <rob.cliffe at btinternet.com> wrote:

> Hi All,
> This is my first post to python-dev so I will briefly introduce myself:? My
> name is Rob Cliffe and I am a commercial programmer living in London, UK.? I
> have some 30 years of programming experience but have only been using Python
> for a couple of years.
> First I want to say what a fantastic language Python is.? It is THE best
> language for development in my opinion, and a joy to use.
>
> My specific issue:
> I eventually got my head round decorator syntax and realised that what came
> after the '@' was (basically) a function that took a function as argument
> and returned a function as result.
> However it seems to me?unPythonesque (i.e. an exception to Python's normal
> consistency) that the syntax of what follows the '@' should be restricted to
> either a single (function) identifier or a single (function) identifier with
> an argument list.
> The example I tried, which seems not an unreasonable sort of thing to do,
> was along the lines of:
>
> def deco1(func):
> ??? <deco1-suite>
> def deco2(func):
> ??? <deco2-suite>
>
> DecoList = [deco1, deco2]
>
> @DecoList[0]??? # NO - CAUSES SYNTAX ERROR
> def foo():
> ??? pass
>
> I am sure other guys have their own examples.
>
> I am of course not the first person to raise this issue, and I see that
> Guido has a "gut feeling" against allowing a general expression after the
> '@'.
>
> BUT - a general expression can be "smuggled in" very easily as a function
> argument:
>
> def Identity(x): return x
>
> @Identity(DecoList[0])??? # THIS WORKS
> def foo():
> ??? pass
>
> So - the syntax restriction seems not only?inconsistent, but?pointless; it
> doesn't forbid anything, but merely means we have to do it in a slightly
> convoluted (unPythonesque) way.? So please, Guido, will you reconsider?
>
> Best wishes
> Rob Cliffe

From fuzzyman at gmail.com  Wed Sep  2 12:49:31 2009
From: fuzzyman at gmail.com (Michael Foord)
Date: Wed, 2 Sep 2009 11:49:31 +0100
Subject: [Python-Dev] [Python-ideas]  Decorator syntax
In-Reply-To: <3bdda690909020334m4c81cecdm8f57dce54c2da398@mail.gmail.com>
References: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>
	<3bdda690909020334m4c81cecdm8f57dce54c2da398@mail.gmail.com>
Message-ID: <6f4025010909020349h31573e63if2c112ba471b8872@mail.gmail.com>

I actually encountered this for the first time yesterday and didn't realise
that the decorator syntax was limited in this way (I was mentally preparing
a blog entry when these emails arrived).

What I needed to do was turn a Python function  into a .NET event handler in
IronPython. The simple case is this:

from System import EventHandler

@EventHandler
def on_event(sender, event):
    # do stuff...

This works fine of course, but then I needed to use the 'typed' form which
is like this:

@EventHandler[HtmlEventArgs]
def on_event(sender, event):
    # do stuff...

I didn't realise this was invalid syntax - nor the neat trick with the
identity function to bypass the limitation.

Michael

2009/9/2 Carl Johnson <cmjohnson.mailinglist at gmail.com>

> Crossposting to Python-ideas,
>
> I asked for the same change to the grammar a couple months back on
> python-ideas.
>
> See
> http://mail.python.org/pipermail/python-ideas/2009-February/thread.html#2787
>
> I'm all for it, but you'll have to convince Guido that this won't
> result in confusing to read code. My own examples, unfortunately did
> not advance your cause, as Guido explained, "My brain hurts trying to
> understand all this. I don't think this bodes well as a use case for a
> proposed feature." :-D The trouble is that I was using lambdas upon
> lambdas to do all kinds of Ruby block-esque tricks. OTOH, if you come
> up with some simple, clear use cases though, and I think he might
> still be persuadable to make a simple change to the grammar.
>
>
> ? Carl Johnson
>
> Rob Cliffe <rob.cliffe at btinternet.com> wrote:
>
> > Hi All,
> > This is my first post to python-dev so I will briefly introduce myself:
> My
> > name is Rob Cliffe and I am a commercial programmer living in London,
> UK.  I
> > have some 30 years of programming experience but have only been using
> Python
> > for a couple of years.
> > First I want to say what a fantastic language Python is.  It is THE best
> > language for development in my opinion, and a joy to use.
> >
> > My specific issue:
> > I eventually got my head round decorator syntax and realised that what
> came
> > after the '@' was (basically) a function that took a function as argument
> > and returned a function as result.
> > However it seems to me unPythonesque (i.e. an exception to Python's
> normal
> > consistency) that the syntax of what follows the '@' should be restricted
> to
> > either a single (function) identifier or a single (function) identifier
> with
> > an argument list.
> > The example I tried, which seems not an unreasonable sort of thing to do,
> > was along the lines of:
> >
> > def deco1(func):
> >     <deco1-suite>
> > def deco2(func):
> >     <deco2-suite>
> >
> > DecoList = [deco1, deco2]
> >
> > @DecoList[0]    # NO - CAUSES SYNTAX ERROR
> > def foo():
> >     pass
> >
> > I am sure other guys have their own examples.
> >
> > I am of course not the first person to raise this issue, and I see that
> > Guido has a "gut feeling" against allowing a general expression after the
> > '@'.
> >
> > BUT - a general expression can be "smuggled in" very easily as a function
> > argument:
> >
> > def Identity(x): return x
> >
> > @Identity(DecoList[0])    # THIS WORKS
> > def foo():
> >     pass
> >
> > So - the syntax restriction seems not only inconsistent, but pointless;
> it
> > doesn't forbid anything, but merely means we have to do it in a slightly
> > convoluted (unPythonesque) way.  So please, Guido, will you reconsider?
> >
> > Best wishes
> > Rob Cliffe
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090902/3a3f3350/attachment-0001.htm>

From xavier.morel at masklinn.net  Wed Sep  2 12:50:33 2009
From: xavier.morel at masklinn.net (Xavier Morel)
Date: Wed, 2 Sep 2009 12:50:33 +0200
Subject: [Python-Dev] Decorator syntax
In-Reply-To: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>
References: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>
Message-ID: <126B4B7C-38AF-45DF-B5F2-DB146E7146C8@masklinn.net>

On 2 Sep 2009, at 12:15 , Rob Cliffe wrote:
>
> @Identity(DecoList[0])    # THIS WORKS
> def foo():
>    pass
For what it's worth, you don't need an id function, you can simply write

     @itemgetter(0)(decorators)
     def foo():
         'whatever'
or

     @decorators.__getitem__(0)
     def foo():
         'whatever'


From ncoghlan at gmail.com  Wed Sep  2 13:11:40 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 02 Sep 2009 21:11:40 +1000
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <4A9D9A7E.5080209@canterbury.ac.nz>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>	<loom.20090831T231829-502@post.gmane.org>	<4A9C6457.2060609@canterbury.ac.nz>	<D06CB4B3-5717-4E81-8E5F-6DEAB824C833@masklinn.net>
	<4A9D9A7E.5080209@canterbury.ac.nz>
Message-ID: <4A9E52EC.3030809@gmail.com>

Greg Ewing wrote:
> Xavier Morel wrote:
> 
>> I fail to grasp the unpredictability of "the last expression
>> evaluated  in the body of a function is its return value".
> 
> It's unpredictable in the sense that if you're writing
> a function that's not intended to return a value, you're
> not thinking about what the last call you make in the
> function returns, so to a first approximation it's just
> some random value.
> 
> I often write code that makes use of the fact that falling
> off the end of a function returns None. This has been a
> documented part of the Python language from the beginning,
> and changing it would break a lot of code for no good
> reason.

It also means adding a debugging message, assertion, or otherwise
side-effect free statement can change the return value of the function.
Not cool.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From kristjan at ccpgames.com  Wed Sep  2 14:07:19 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Wed, 2 Sep 2009 12:07:19 +0000
Subject: [Python-Dev] default of returning None hurts performance?
In-Reply-To: <4A9E52EC.3030809@gmail.com>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
	<D06CB4B3-5717-4E81-8E5F-6DEAB824C833@masklinn.net>
	<4A9D9A7E.5080209@canterbury.ac.nz> <4A9E52EC.3030809@gmail.com>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098F163EC4@exchis.ccp.ad.local>

Not advocating a change, merely pointing out that it's how Ruby works.
K

> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf
> Of Nick Coghlan
> Sent: 2. september 2009 11:12
> To: Greg Ewing
> Cc: python-dev at python.org
> Subject: Re: [Python-Dev] default of returning None hurts performance?


> It also means adding a debugging message, assertion, or otherwise
> side-effect free statement can change the return value of the function.
> Not cool.
> 


From p.f.moore at gmail.com  Wed Sep  2 16:16:40 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 2 Sep 2009 15:16:40 +0100
Subject: [Python-Dev] Setting up a buildbot
In-Reply-To: <79990c6b0908221308k7d5c87b9m673ea6183946586c@mail.gmail.com>
References: <79990c6b0908221228y46f2f592web53226cc33473c4@mail.gmail.com>
	<20090822194049.GE19190@nexus.in-nomine.org>
	<79990c6b0908221308k7d5c87b9m673ea6183946586c@mail.gmail.com>
Message-ID: <79990c6b0909020716m4148bd46ie7f19a8ac12ec9fb@mail.gmail.com>

2009/8/22 Paul Moore <p.f.moore at gmail.com>:
> 2009/8/22 Jeroen Ruigrok van der Werven <asmodai at in-nomine.org>:
>> -On [20090822 21:30], Paul Moore (p.f.moore at gmail.com) wrote:
>>>I've just had a look on python.org, but couldn't immediately see a
>>>pointer to instructions on what the process is to set up a buildbot.
>>
>> http://wiki.python.org/moin/BuildBot comes to mind.
>
> Ah, thanks. I'll take a look.

I saw on planet Python that the buildbots have currently been shut
down. I guess this makes my question fairly irrelevant for the moment,
then :-(

Paul

From foom at fuhm.net  Wed Sep  2 16:35:29 2009
From: foom at fuhm.net (James Y Knight)
Date: Wed, 2 Sep 2009 10:35:29 -0400
Subject: [Python-Dev] Decorator syntax
In-Reply-To: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>
References: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>
Message-ID: <E3E4B628-7F91-40C2-9036-16863D836BE7@fuhm.net>

On Sep 2, 2009, at 6:15 AM, Rob Cliffe wrote:

> So - the syntax restriction seems not only inconsistent, but  
> pointless; it doesn't forbid anything, but merely means we have to  
> do it in a slightly convoluted (unPythonesque) way.  So please,  
> Guido, will you reconsider?

Indeed, it's a silly inconsistent restriction. When it was first added  
I too suggested that any expression be allowed after the @, rather  
than having a uniquely special restricted syntax. I argued from  
consistency of grammar standpoint. But Guido was not persuaded. Good  
luck to you. :)

Here's some of the more relevant messages from the thread back when  
the @decorator feature was first introduced:
http://mail.python.org/pipermail/python-dev/2004-August/046654.html
http://mail.python.org/pipermail/python-dev/2004-August/046659.html
http://mail.python.org/pipermail/python-dev/2004-August/046675.html
http://mail.python.org/pipermail/python-dev/2004-August/046711.html
http://mail.python.org/pipermail/python-dev/2004-August/046741.html
http://mail.python.org/pipermail/python-dev/2004-August/046753.html
http://mail.python.org/pipermail/python-dev/2004-August/046818.html

James

From bxs187 at case.edu  Wed Sep  2 17:19:11 2009
From: bxs187 at case.edu (Boya Sun)
Date: Wed, 2 Sep 2009 11:19:11 -0400
Subject: [Python-Dev] Bugs discovered by researchers at Case Western Reserve
	University
Message-ID: <3efbea50909020819x5aba605fid2a78887894ddccf@mail.gmail.com>

Dear Developers,

I am a Ph.D student from Case Western Reserve University, specialized at
software engineering.  Our recent approach analyzes bugs that are being
fixed in the issue database, and tries to discover any latent bug instances
that are the same as the fixed bug but are left unfixed.  We have found some
spurious code in your project in this approach, and pointed out these code
by comments (sometimes also with patches) to the fixed bugs in the issue DB
from which it is discovered as follows:

Issue 6817: http://bugs.python.org/issue6817 (A new issue, created following
Amaury's comments)
Issue 2620: http://bugs.python.org/issue2620
Issue 3139: http://bugs.python.org/issue3139
Issue 5705: http://bugs.python.org/issue5705

We hope that we have discovered some real bugs for you.  Any comments or
suggestions are GREATLY appreciated, since your opinions are very, very
precious to us.

Boya
-- 
BOYA SUN
Computer Science Division
Electrical Engineering & Computer Science Department
513 Olin Building
Case Western Reserve University
10900 Euclid Avenue
Cleveland, OH 44106
boya.sun at case.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090902/5d32f8d7/attachment.htm>

From hyugaricdeau at gmail.com  Wed Sep  2 18:06:34 2009
From: hyugaricdeau at gmail.com (Erik Bray)
Date: Wed, 2 Sep 2009 12:06:34 -0400
Subject: [Python-Dev] Decorator syntax
In-Reply-To: <E3E4B628-7F91-40C2-9036-16863D836BE7@fuhm.net>
References: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>
	<E3E4B628-7F91-40C2-9036-16863D836BE7@fuhm.net>
Message-ID: <e71e26940909020906q7c2ed377qaebd2c4a752ff20a@mail.gmail.com>

On Wed, Sep 2, 2009 at 10:35 AM, James Y Knight<foom at fuhm.net> wrote:
> On Sep 2, 2009, at 6:15 AM, Rob Cliffe wrote:
>
>> So - the syntax restriction seems not only inconsistent, but pointless; it
>> doesn't forbid anything, but merely means we have to do it in a slightly
>> convoluted (unPythonesque) way. ?So please, Guido, will you reconsider?
>
> Indeed, it's a silly inconsistent restriction. When it was first added I too
> suggested that any expression be allowed after the @, rather than having a
> uniquely special restricted syntax. I argued from consistency of grammar
> standpoint. But Guido was not persuaded. Good luck to you. :)
>
> Here's some of the more relevant messages from the thread back when the
> @decorator feature was first introduced:
> http://mail.python.org/pipermail/python-dev/2004-August/046654.html
> http://mail.python.org/pipermail/python-dev/2004-August/046659.html
> http://mail.python.org/pipermail/python-dev/2004-August/046675.html
> http://mail.python.org/pipermail/python-dev/2004-August/046711.html
> http://mail.python.org/pipermail/python-dev/2004-August/046741.html
> http://mail.python.org/pipermail/python-dev/2004-August/046753.html
> http://mail.python.org/pipermail/python-dev/2004-August/046818.html

I think Guido may have a point about not allowing any arbitrary
expression.  But I do think that if it allows calls, it should also at
least support the itemgetter syntax, for which there seems to be a
demonstrable use case.  But that's just adding on another special
case, so it might be simpler to allow arbitrary expressions.

From eric at trueblade.com  Wed Sep  2 18:11:48 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 02 Sep 2009 12:11:48 -0400
Subject: [Python-Dev] Decorator syntax
In-Reply-To: <e71e26940909020906q7c2ed377qaebd2c4a752ff20a@mail.gmail.com>
References: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>	<E3E4B628-7F91-40C2-9036-16863D836BE7@fuhm.net>
	<e71e26940909020906q7c2ed377qaebd2c4a752ff20a@mail.gmail.com>
Message-ID: <4A9E9944.6020900@trueblade.com>

Erik Bray wrote:
> I think Guido may have a point about not allowing any arbitrary
> expression.  But I do think that if it allows calls, it should also at
> least support the itemgetter syntax, for which there seems to be a
> demonstrable use case.  But that's just adding on another special
> case, so it might be simpler to allow arbitrary expressions.

It's not the same issue, of course, but note that str.format's object 
specification "language" supports item and attribute access. And that's 
the extent of what it allows.

If you wanted to do this for decorators, I'm not sure how easy it would 
be to restrict the expression inside the brackets, or if you'd even want 
to. str.format has all of this baked-in, it doesn't use any sort of grammar.

Eric

From python at mrabarnett.plus.com  Wed Sep  2 18:36:54 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 02 Sep 2009 17:36:54 +0100
Subject: [Python-Dev] Decorator syntax
In-Reply-To: <E3E4B628-7F91-40C2-9036-16863D836BE7@fuhm.net>
References: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>
	<E3E4B628-7F91-40C2-9036-16863D836BE7@fuhm.net>
Message-ID: <4A9E9F26.8030608@mrabarnett.plus.com>

James Y Knight wrote:
> On Sep 2, 2009, at 6:15 AM, Rob Cliffe wrote:
> 
>> So - the syntax restriction seems not only inconsistent, but 
>> pointless; it doesn't forbid anything, but merely means we have to do 
>> it in a slightly convoluted (unPythonesque) way.  So please, Guido, 
>> will you reconsider?
> 
> Indeed, it's a silly inconsistent restriction. When it was first added I 
> too suggested that any expression be allowed after the @, rather than 
> having a uniquely special restricted syntax. I argued from consistency 
> of grammar standpoint. But Guido was not persuaded. Good luck to you. :)
> 
[snip]
I can see no syntactic reason to restrict what can appear after the @.
If someone chooses to abuse it then that's unPythonic, but not illegal.

From martin at v.loewis.de  Wed Sep  2 20:57:52 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 02 Sep 2009 20:57:52 +0200
Subject: [Python-Dev] Setting up a buildbot
In-Reply-To: <79990c6b0909020716m4148bd46ie7f19a8ac12ec9fb@mail.gmail.com>
References: <79990c6b0908221228y46f2f592web53226cc33473c4@mail.gmail.com>	<20090822194049.GE19190@nexus.in-nomine.org>	<79990c6b0908221308k7d5c87b9m673ea6183946586c@mail.gmail.com>
	<79990c6b0909020716m4148bd46ie7f19a8ac12ec9fb@mail.gmail.com>
Message-ID: <4A9EC030.3010605@v.loewis.de>

> I saw on planet Python that the buildbots have currently been shut
> down. I guess this makes my question fairly irrelevant for the moment,
> then :-(

That was a misunderstanding. It was only the community buildbots, and
Grig Gheorghiu is working on restoring them.

Regards,
Martin

From ideasman42 at gmail.com  Wed Sep  2 21:15:08 2009
From: ideasman42 at gmail.com (Campbell Barton)
Date: Wed, 2 Sep 2009 12:15:08 -0700
Subject: [Python-Dev] Py_CmpToRich removed, alternatives for comparison?
Message-ID: <7c1ab96d0909021215n46e50713v850bd6d5e08fcc8f@mail.gmail.com>

For blender we have a number of types defined in the C/API like
meshes, lamps, metaballs, nurbs etc that dont make sense with some of
richcmp's operations.
A problem I have is that in python 3.1 the Py_CmpToRich function is removed.

Should we copy Py_CmpToRich into our source tree?
Otherwise we have fairly long cmp functions that have Py_CmpToRich inline.

For C extension writers what is the suggested method for comparing
types where Py_LT, Py_GT Py_GE etc are not useful?

Since people sometimes ask why use py3.1, we're doing a rewrite that
wont be ready for quite some time.
-- 
- Campbell

From brett at python.org  Wed Sep  2 22:43:50 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 2 Sep 2009 13:43:50 -0700
Subject: [Python-Dev] Bugs discovered by researchers at Case Western
	Reserve University
In-Reply-To: <3efbea50909020819x5aba605fid2a78887894ddccf@mail.gmail.com>
References: <3efbea50909020819x5aba605fid2a78887894ddccf@mail.gmail.com>
Message-ID: <bbaeab100909021343r3d31bb6xf2a1bd7c2cac49a5@mail.gmail.com>

On Wed, Sep 2, 2009 at 08:19, Boya Sun<bxs187 at case.edu> wrote:
> Dear Developers,
>
> I am a Ph.D student from Case Western Reserve University, specialized at
> software engineering.? Our recent approach analyzes bugs that are being
> fixed in the issue database, and tries to discover any latent bug instances
> that are the same as the fixed bug but are left unfixed.? We have found some
> spurious code in your project in this approach, and pointed out these code
> by comments (sometimes also with patches) to the fixed bugs in the issue DB
> from which it is discovered as follows:
>
> Issue 6817: http://bugs.python.org/issue6817 (A new issue, created following
> Amaury's comments)
> Issue 2620: http://bugs.python.org/issue2620
> Issue 3139: http://bugs.python.org/issue3139
> Issue 5705: http://bugs.python.org/issue5705

I quickly re-opened 2620 and 5705 so we don't lose track of the fact
that new code has been attached.

-Brett

From brett at python.org  Wed Sep  2 22:47:53 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 2 Sep 2009 13:47:53 -0700
Subject: [Python-Dev] Decorator syntax
In-Reply-To: <3bdda690909020334m4c81cecdm8f57dce54c2da398@mail.gmail.com>
References: <A5DBADE2151C40D8BF1E590DF868DA8D@robslaptop>
	<3bdda690909020334m4c81cecdm8f57dce54c2da398@mail.gmail.com>
Message-ID: <bbaeab100909021347s63905e46u74192edf63b75e16@mail.gmail.com>

People, please note that the discussion has shifted to python-ideas
and further comments should happen over there. Carl did the right
thing to shift it there, although cross-posting once the conversation
redirection has occurred is not needed.

hoping-google-wave-will-have-a-permanent-redirector-for-conversations-ly yrs,
Brett

On Wed, Sep 2, 2009 at 03:34, Carl
Johnson<cmjohnson.mailinglist at gmail.com> wrote:
> Crossposting to Python-ideas,
>
> I asked for the same change to the grammar a couple months back on
> python-ideas.
>
> See http://mail.python.org/pipermail/python-ideas/2009-February/thread.html#2787
>
> I'm all for it, but you'll have to convince Guido that this won't
> result in confusing to read code. My own examples, unfortunately did
> not advance your cause, as Guido explained, "My brain hurts trying to
> understand all this. I don't think this bodes well as a use case for a
> proposed feature." :-D The trouble is that I was using lambdas upon
> lambdas to do all kinds of Ruby block-esque tricks. OTOH, if you come
> up with some simple, clear use cases though, and I think he might
> still be persuadable to make a simple change to the grammar.
>
>
> ? Carl Johnson
>
> Rob Cliffe <rob.cliffe at btinternet.com> wrote:
>
>> Hi All,
>> This is my first post to python-dev so I will briefly introduce myself:? My
>> name is Rob Cliffe and I am a commercial programmer living in London, UK.? I
>> have some 30 years of programming experience but have only been using Python
>> for a couple of years.
>> First I want to say what a fantastic language Python is.? It is THE best
>> language for development in my opinion, and a joy to use.
>>
>> My specific issue:
>> I eventually got my head round decorator syntax and realised that what came
>> after the '@' was (basically) a function that took a function as argument
>> and returned a function as result.
>> However it seems to me?unPythonesque (i.e. an exception to Python's normal
>> consistency) that the syntax of what follows the '@' should be restricted to
>> either a single (function) identifier or a single (function) identifier with
>> an argument list.
>> The example I tried, which seems not an unreasonable sort of thing to do,
>> was along the lines of:
>>
>> def deco1(func):
>> ??? <deco1-suite>
>> def deco2(func):
>> ??? <deco2-suite>
>>
>> DecoList = [deco1, deco2]
>>
>> @DecoList[0]??? # NO - CAUSES SYNTAX ERROR
>> def foo():
>> ??? pass
>>
>> I am sure other guys have their own examples.
>>
>> I am of course not the first person to raise this issue, and I see that
>> Guido has a "gut feeling" against allowing a general expression after the
>> '@'.
>>
>> BUT - a general expression can be "smuggled in" very easily as a function
>> argument:
>>
>> def Identity(x): return x
>>
>> @Identity(DecoList[0])??? # THIS WORKS
>> def foo():
>> ??? pass
>>
>> So - the syntax restriction seems not only?inconsistent, but?pointless; it
>> doesn't forbid anything, but merely means we have to do it in a slightly
>> convoluted (unPythonesque) way.? So please, Guido, will you reconsider?
>>
>> Best wishes
>> Rob Cliffe
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From benjamin at python.org  Thu Sep  3 00:19:46 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 2 Sep 2009 17:19:46 -0500
Subject: [Python-Dev] Py_CmpToRich removed, alternatives for comparison?
In-Reply-To: <7c1ab96d0909021215n46e50713v850bd6d5e08fcc8f@mail.gmail.com>
References: <7c1ab96d0909021215n46e50713v850bd6d5e08fcc8f@mail.gmail.com>
Message-ID: <1afaf6160909021519w35c567a9r36aebf203ae64392@mail.gmail.com>

2009/9/2 Campbell Barton <ideasman42 at gmail.com>:
> For blender we have a number of types defined in the C/API like
> meshes, lamps, metaballs, nurbs etc that dont make sense with some of
> richcmp's operations.
> A problem I have is that in python 3.1 the Py_CmpToRich function is removed.
>
> Should we copy Py_CmpToRich into our source tree?
> Otherwise we have fairly long cmp functions that have Py_CmpToRich inline.
>
> For C extension writers what is the suggested method for comparing
> types where Py_LT, Py_GT Py_GE etc are not useful?

Return Py_NotImplemented when you get them.



-- 
Regards,
Benjamin

From olemis at gmail.com  Thu Sep  3 20:14:38 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 3 Sep 2009 13:14:38 -0500
Subject: [Python-Dev] OT : Cannot login to PyPI using MyOpenId
Message-ID: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com>

I realized that PyPI accepts MyOpenId and tried to login to the site.
In the process I get this message :

{{{
OpenID provider did not provide your email address
}}}

I mean, is it mandatory to provide the e-mail address in order to bind
a user to an OpenId ?
I'm curious : I'd like to know if there's a reason to do that.

Thnx in advance !

PS: I know is a little bit OT. Hope you dont mind ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From martin at v.loewis.de  Thu Sep  3 20:49:48 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 03 Sep 2009 20:49:48 +0200
Subject: [Python-Dev] OT : Cannot login to PyPI using MyOpenId
In-Reply-To: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com>
References: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com>
Message-ID: <4AA00FCC.5080606@v.loewis.de>

Olemis Lang wrote:
> I realized that PyPI accepts MyOpenId and tried to login to the site.
> In the process I get this message :
> 
> {{{
> OpenID provider did not provide your email address
> }}}
> 
> I mean, is it mandatory to provide the e-mail address in order to bind
> a user to an OpenId ?
> I'm curious : I'd like to know if there's a reason to do that.

For an existing user? No, there is no reason.
For a new user, yes, we mandate a valid, verified email address.

> PS: I know is a little bit OT. Hope you dont mind ...

For general PyPI discussions, please use catalog-sig. For PyPI bug
reports, use the bug tracker.

Regards,
Martin


From greg.ewing at canterbury.ac.nz  Thu Sep  3 23:33:14 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 04 Sep 2009 09:33:14 +1200
Subject: [Python-Dev] [OT] implicit return values
In-Reply-To: <E8E6089B-B8E6-4164-95DC-2B6D7F8CC0B0@masklinn.net>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
	<52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com>
	<7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net>
	<1251811506.5631.44.camel@localhost>
	<B928ABA7-8AA6-43E1-8AA5-F21F3F9C75E6@masklinn.net>
	<4A9D9BD1.8000501@canterbury.ac.nz>
	<E8E6089B-B8E6-4164-95DC-2B6D7F8CC0B0@masklinn.net>
Message-ID: <4AA0361A.8020904@canterbury.ac.nz>

Xavier Morel wrote:

> Methods yes (and that's one of the few Smalltalk design "features" I  
> consider truly dumb, considering it has message cascading)

Cascading is something different -- it's for sending
multiple messages to the *same* receiver. It's not
dumb to have both.

-- 
Greg

From ben+python at benfinney.id.au  Fri Sep  4 02:22:52 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 04 Sep 2009 10:22:52 +1000
Subject: [Python-Dev] OT : Cannot login to PyPI using MyOpenId
References: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com>
	<4AA00FCC.5080606@v.loewis.de>
Message-ID: <87ab1b7e3n.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> For general PyPI discussions, please use catalog-sig. For PyPI bug
> reports, use the bug tracker.

The Python bug tracker <URL:http://bugs.python.org/>? Or the tracker
linked from <URL:http://pypi.python.org/pypi?:action=openid> which leads
to <URL:http://sourceforge.net/tracker/?group_id=66150&atid=513503>?

The Python bug tracker has issues with OpenID (it doesn't allow OpenID
login at all, AFAICT), and the latter times out attempting a connection
for me.

-- 
 \         ?We must become the change we want to see.? ?Mahatma Gandhi |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From aahz at pythoncraft.com  Fri Sep  4 05:40:53 2009
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 3 Sep 2009 20:40:53 -0700
Subject: [Python-Dev] REMINDER: PyCon 2010: Call for Proposals
Message-ID: <20090904034053.GA6855@panix.com>

Call for proposals -- PyCon 2010 -- <http://us.pycon.org/2010/>
===============================================================

Due date: October 1st, 2009

Want to showcase your skills as a Python Hacker? Want to have
hundreds of people see your talk on the subject of your choice? Have some
hot button issue you think the community needs to address, or have some
package, code or project you simply love talking about? Want to launch
your master plan to take over the world with python?

PyCon is your platform for getting the word out and teaching something
new to hundreds of people, face to face.

Previous PyCon conferences have had a broad range of presentations,
from reports on academic and commercial projects, tutorials on a broad
range of subjects and case studies. All conference speakers are volunteers
and come from a myriad of backgrounds. Some are new speakers, some
are old speakers. Everyone is welcome so bring your passion and your
code! We're looking to you to help us top the previous years of success
PyCon has had.

PyCon 2010 is looking for proposals to fill the formal presentation tracks.
The PyCon conference days will be February 19-22, 2010 in Atlanta,
Georgia, preceded by the tutorial days (February 17-18), and followed
by four days of development sprints (February 22-25).

Online proposal submission is open now! Proposals  will be accepted
through October 1st, with acceptance notifications coming out on
November 15th. For the detailed call for proposals, please see:

 <http://us.pycon.org/2010/conference/proposals/>

For videos of talks from previous years - check out:

<http://pycon.blip.tv>

We look forward to seeing you in Atlanta!
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Look, it's your affair if you want to play with five people, but don't
go calling it doubles."  --John Cleese anticipates Usenet

From martin at v.loewis.de  Fri Sep  4 07:18:48 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 04 Sep 2009 07:18:48 +0200
Subject: [Python-Dev] OT : Cannot login to PyPI using MyOpenId
In-Reply-To: <87ab1b7e3n.fsf@benfinney.id.au>
References: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com>	<4AA00FCC.5080606@v.loewis.de>
	<87ab1b7e3n.fsf@benfinney.id.au>
Message-ID: <4AA0A338.2070506@v.loewis.de>

>> For general PyPI discussions, please use catalog-sig. For PyPI bug
>> reports, use the bug tracker.
> 
> The Python bug tracker <URL:http://bugs.python.org/>? Or the tracker
> linked from <URL:http://pypi.python.org/pypi?:action=openid> which leads
> to <URL:http://sourceforge.net/tracker/?group_id=66150&atid=513503>?

The latter. The former is for bugs in Python only.

> The Python bug tracker has issues with OpenID (it doesn't allow OpenID
> login at all, AFAICT)

That alone wouldn't make it unsuitable for reporting problems with PyPI,
including problems with OpenID, right?

> and the latter times out attempting a connection for me.

Please try again; it does work most of the time (if it still doesn't
work, check your internet connection).

Regards,
Martin

From ben+python at benfinney.id.au  Fri Sep  4 07:53:31 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Fri, 04 Sep 2009 15:53:31 +1000
Subject: [Python-Dev] OT : Cannot login to PyPI using MyOpenId
References: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com>
	<4AA00FCC.5080606@v.loewis.de> <87ab1b7e3n.fsf@benfinney.id.au>
	<4AA0A338.2070506@v.loewis.de>
Message-ID: <87ws4f5k84.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> > The Python bug tracker <URL:http://bugs.python.org/>? Or the tracker
> > linked from <URL:http://pypi.python.org/pypi?:action=openid> which
> > leads to
> > <URL:http://sourceforge.net/tracker/?group_id=66150&atid=513503>?
>
> The latter. The former is for bugs in Python only.
[?]

> > and the latter times out attempting a connection for me.
>
> Please try again; it does work most of the time (if it still doesn't
> work, check your internet connection).

My internet connection is fine, I'm able to visit various sites without
a problem.

That tracker loads fine, but when I select ?Log In?, it takes a very
long time and eventually times out. This is repeatable (for me) multiple
times over the past 12 hours or so.

This is probably a SourceForge problem and nothing you can fix directly,
but those wanting to receive bug reports into that tracker might be
interested to know.

-- 
 \                ?If you can't hear me sometimes, it's because I'm in |
  `\                                      parentheses.? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From catch-all at masklinn.net  Fri Sep  4 08:34:44 2009
From: catch-all at masklinn.net (Xavier Morel)
Date: Fri, 4 Sep 2009 08:34:44 +0200
Subject: [Python-Dev] [OT] implicit return values
In-Reply-To: <4AA0361A.8020904@canterbury.ac.nz>
References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com>
	<loom.20090831T231829-502@post.gmane.org>
	<4A9C6457.2060609@canterbury.ac.nz>
	<52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com>
	<7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net>
	<1251811506.5631.44.camel@localhost>
	<B928ABA7-8AA6-43E1-8AA5-F21F3F9C75E6@masklinn.net>
	<4A9D9BD1.8000501@canterbury.ac.nz>
	<E8E6089B-B8E6-4164-95DC-2B6D7F8CC0B0@masklinn.net>
	<4AA0361A.8020904@canterbury.ac.nz>
Message-ID: <C0101DDA-EF19-47B7-94CA-0D70D95CD5F0@masklinn.net>

On 3 Sep 2009, at 23:33 , Greg Ewing wrote:
Xavier Morel wrote:
>
>> Methods yes (and that's one of the few Smalltalk design "features"  
>> I  consider truly dumb, considering it has message cascading)
>
> Cascading is something different -- it's for sending
> multiple messages to the *same* receiver. It's not
> dumb to have both.
I know what cascading is for. The issue is that with message cascading  
+ the "yourself" message, you *never* need to chain on self (you can  
just cascade and -- if you end up needing the instance to drop down at  
the end of the cascade -- send `yourself`).

Chaining on self is completely redundant in smalltalk as the purpose  
of this pattern is *also* to send a sequence of messages to the same  
receiver (something message cascading already handles & guarantees).

Therefore defaulting method to self-chaining is very dumb and serves  
no purpose whatsoever. It doesn't make the language easier to use,  
less verbose or more practical. It just wastes return values.

From kristjan at ccpgames.com  Fri Sep  4 11:19:09 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Fri, 4 Sep 2009 09:19:09 +0000
Subject: [Python-Dev] PyMem_Malloc() vs PyObject_Malloc()
Message-ID: <930F189C8A437347B80DF2C156F7EC7F098F164308@exchis.ccp.ad.local>


I noticed something (in 2.5) yesterday, which may be a feature, but is more likely a bug.
In tokenizer.c, tok->encoding is allocated using PyMem_MALLOC().
However, this then gets handed to a node->r_str in parsetok.c, and then released in node.c using PyObject_Free().

Now, by coincidence, PyObject_Free() will default to free() for objects that it doesn't recognize, so this works.  But is this documented behavior?  The reason I ran into this was that I had redirect the PyMem_* API to a different allocator, but left the PyObject_* one alone.

My feeling Is that these two APIs shouldn't be interchangeable.  Especially since you can't hand a PyObject_Malloc'd object to PyMem_Free() so the inverse shouldn't be expected to work.

Any thoughts?

Kristj?n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090904/fdb0a03a/attachment.htm>

From mal at egenix.com  Fri Sep  4 11:27:58 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 04 Sep 2009 11:27:58 +0200
Subject: [Python-Dev] PyMem_Malloc() vs PyObject_Malloc()
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098F164308@exchis.ccp.ad.local>
References: <930F189C8A437347B80DF2C156F7EC7F098F164308@exchis.ccp.ad.local>
Message-ID: <4AA0DD9E.9090501@egenix.com>

Kristj?n Valur J?nsson wrote:
> 
> I noticed something (in 2.5) yesterday, which may be a feature, but is more likely a bug.
> In tokenizer.c, tok->encoding is allocated using PyMem_MALLOC().
> However, this then gets handed to a node->r_str in parsetok.c, and then released in node.c using PyObject_Free().
> 
> Now, by coincidence, PyObject_Free() will default to free() for objects that it doesn't recognize, so this works.  But is this documented behavior?  The reason I ran into this was that I had redirect the PyMem_* API to a different allocator, but left the PyObject_* one alone.
> 
> My feeling Is that these two APIs shouldn't be interchangeable.  Especially since you can't hand a PyObject_Malloc'd object to PyMem_Free() so the inverse shouldn't be expected to work.
> 
> Any thoughts?

This is a bug. Please file a bug report for this.

In general, either PyObject_* xor PyMem_* memory API should used.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 04 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ncoghlan at gmail.com  Fri Sep  4 11:45:08 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 04 Sep 2009 19:45:08 +1000
Subject: [Python-Dev] PyMem_Malloc() vs PyObject_Malloc()
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098F164308@exchis.ccp.ad.local>
References: <930F189C8A437347B80DF2C156F7EC7F098F164308@exchis.ccp.ad.local>
Message-ID: <4AA0E1A4.2060906@gmail.com>

Kristj?n Valur J?nsson wrote:
> My feeling Is that these two APIs shouldn?t be interchangeable. 
> Especially since you can?t hand a PyObject_Malloc?d object to
> PyMem_Free() so the inverse shouldn?t be expected to work.

I thought this had officially been deemed illegal for a while, and
Google found the reference I was looking for in the What's New for 2.5:

"""Previously these different families all reduced to the platform's
malloc() and free() functions. This meant it didn't matter if you got
things wrong and allocated memory with the PyMem function but freed it
with the PyObject function. With 2.5's changes to obmalloc, these
families now do different things and mismatches will probably result in
a segfault. You should carefully test your C extension modules with
Python 2.5."""

So either the allocation or the release needs to change here.

The behaviour of PyObject_Del when handed a pointer it doesn't recognise
is currently undocumented. It may be best to make it officially
undefined in order to further discourage people from relying on the
implicit delegation to PyMem_Free.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From chris at simplistix.co.uk  Fri Sep  4 13:11:46 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 04 Sep 2009 12:11:46 +0100
Subject: [Python-Dev] how to debug httplib slowness
In-Reply-To: <ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>
References: <4A82A20A.1070704@simplistix.co.uk>
	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>
	<4A82E120.4030105@simplistix.co.uk>
	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>
Message-ID: <4AA0F5F2.6030903@simplistix.co.uk>

Guido van Rossum wrote:
>>> You might see a pattern. Is this on Windows?
>> Well, yes, but I'm not 100%. The problematic machine is a Windows box, but
>> there are no non-windows boxes on that network and vpn'ing from one of my
>> non-windows boxes slows things down enough that I'm not confident what I'd
>> be seeing was indicative of the same problem...
> 
> Time to set up a more conclusive test. Do you have something like curl
> or wget available on the same box?

Time taken with IE: ~2 seconds
Time taken with wget: 2.2 seconds
Time taken with Python [1]: 20-30 minutes

I did a run of the script through cProfile and got the following:

pstats.Stats('download.profile').strip_dirs().sort_stats('time').print_stats(10)

          1604545 function calls in 1956.057 CPU seconds

    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
         1 1950.767 1950.767 1955.952 1955.952 httplib.py:544(_read_chunked)
     85125    1.235    0.000    1.235    0.000 {method 'recv' of 
'_socket.socket' objects}
     85838    1.031    0.000    2.246    0.000 socket.py:313(read)
     85838    0.787    0.000    3.386    0.000 httplib.py:601(_safe_read)
     42928    0.614    0.000    1.779    0.000 socket.py:373(readline)
    128775    0.344    0.000    0.344    0.000 {method 'write' of 
'cStringIO.StringO' objects}
    200796    0.206    0.000    0.206    0.000 {method 'seek' of 
'cStringIO.StringO' objects}
     85838    0.179    0.000    0.179    0.000 {min}
    128767    0.135    0.000    0.135    0.000 {cStringIO.StringIO}
     72735    0.116    0.000    0.116    0.000 {method 'read' of 
'cStringIO.StringO' objects}

...which isn't what I was expecting!

Am I right in reading this as most of the time is being spent in 
httplib's HTTPResponse._read_chunked and none of the methods it calls?

If so, is there a better way that a bunch of print statements to find 
where in that method the time is being spent?

cheers,

Chris

[1] Python 2.6.2 on Windows Server 2003 R2 running this script:

from base64 import encodestring
from httplib import HTTPConnection
from datetime import datetime

conn = HTTPConnection('servername')
headers = {}
a = 'Basic '+encodestring('username:password').strip()
headers['Authorization']=a
t = datetime.now()
print t
conn.request('GET','/some/big/file',None,headers)
print 'request:',datetime.now()-t
response = conn.getresponse()
print 'response:',datetime.now()-t
data = response.read()
if len(data)<2000: print data
print 'read:',datetime.now()-t


-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From hodgestar+pythondev at gmail.com  Fri Sep  4 13:28:35 2009
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Fri, 4 Sep 2009 13:28:35 +0200
Subject: [Python-Dev] how to debug httplib slowness
In-Reply-To: <4AA0F5F2.6030903@simplistix.co.uk>
References: <4A82A20A.1070704@simplistix.co.uk>
	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>
	<4A82E120.4030105@simplistix.co.uk>
	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>
	<4AA0F5F2.6030903@simplistix.co.uk>
Message-ID: <fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>

On Fri, Sep 4, 2009 at 1:11 PM, Chris Withers<chris at simplistix.co.uk> wrote:
> Am I right in reading this as most of the time is being spent in httplib's
> HTTPResponse._read_chunked and none of the methods it calls?
>
> If so, is there a better way that a bunch of print statements to find where
> in that method the time is being spent?

Well, since the source for _read_chunked includes the comment

        # XXX This accumulates chunks by repeated string concatenation,
        # which is not efficient as the number or size of chunks gets big.

you might gain some speed improvement with minimal effort by gathering
the read data chunks into a list and then returning "".join(chunks) at
the end.

Schiavo
Simon

From solipsis at pitrou.net  Fri Sep  4 13:40:32 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 4 Sep 2009 11:40:32 +0000 (UTC)
Subject: [Python-Dev] how to debug httplib slowness
References: <4A82A20A.1070704@simplistix.co.uk>
	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>
	<4A82E120.4030105@simplistix.co.uk>
	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>
	<4AA0F5F2.6030903@simplistix.co.uk>
	<fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
Message-ID: <loom.20090904T133126-602@post.gmane.org>

Simon Cross <hodgestar+pythondev <at> gmail.com> writes:
> 
> Well, since the source for _read_chunked includes the comment
> 
>         # XXX This accumulates chunks by repeated string concatenation,
>         # which is not efficient as the number or size of chunks gets big.
> 
> you might gain some speed improvement with minimal effort by gathering
> the read data chunks into a list and then returning "".join(chunks) at
> the end.

+1 for trying this. Given differences between platforms in realloc()
performance, it might be the reason why it goes unnoticed under Linux but
degenerates under Windows.

As a sidenote, it is interesting that even an stdlib module makes this mistake
and acknowledges it without trying to fix it.



From chris at simplistix.co.uk  Fri Sep  4 17:02:39 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 04 Sep 2009 16:02:39 +0100
Subject: [Python-Dev] how to debug httplib slowness
In-Reply-To: <fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
References: <4A82A20A.1070704@simplistix.co.uk>	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>	<4A82E120.4030105@simplistix.co.uk>	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>	<4AA0F5F2.6030903@simplistix.co.uk>
	<fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
Message-ID: <4AA12C0F.1090305@simplistix.co.uk>

Simon Cross wrote:
> Well, since the source for _read_chunked includes the comment
> 
>         # XXX This accumulates chunks by repeated string concatenation,
>         # which is not efficient as the number or size of chunks gets big.
> 
> you might gain some speed improvement with minimal effort by gathering
> the read data chunks into a list and then returning "".join(chunks) at
> the end.

True, I'll be trying that and reporting back, but, more interestingly, I 
did some analysis with wireshark (only 200MB-odd of .pcap logs that was 
fun ;-) to see the differences in the http conversation and noticed more 
interestingness...

So, httplib does this:

GET /<blah> HTTP/1.1
Host: <blah>
Accept-Encoding: identity
Authorization: Basic <blah>

HTTP/1.1 200 OK
Date: Fri, 04 Sep 2009 11:44:22 GMT
Server: Apache-Coyote/1.1
ContentLength: 116245504
Content-Type: application/vnd.excel
Transfer-Encoding: chunked

While wget does this:

<snip 401 conversation>
GET /<blah> HTTP/1.0
User-Agent: Wget/1.11.4
Accept: */*
Host: <blah>
Connection: Keep-Alive
Authorization: Basic <blah>

HTTP/1.1 200 OK
Date: Fri, 04 Sep 2009 11:35:19 GMT
Server: Apache-Coyote/1.1
ContentLength: 116245504
Content-Type: application/vnd.excel
Connection: close

Interesting points:

- Apache in this instance responds with HTTP 1.1, even though the wget 
request was 1.0, is that allowed?

- Apache responds with a chunked response only to httplib. Why is that?

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From phd at phd.pp.ru  Fri Sep  4 17:32:41 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Fri, 4 Sep 2009 19:32:41 +0400
Subject: [Python-Dev] how to debug httplib slowness
In-Reply-To: <4AA12C0F.1090305@simplistix.co.uk>
References: <4A82A20A.1070704@simplistix.co.uk>
	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>
	<4A82E120.4030105@simplistix.co.uk>
	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>
	<4AA0F5F2.6030903@simplistix.co.uk>
	<fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
	<4AA12C0F.1090305@simplistix.co.uk>
Message-ID: <20090904153241.GB7342@phd.pp.ru>

On Fri, Sep 04, 2009 at 04:02:39PM +0100, Chris Withers wrote:
> So, httplib does this:
>
> GET /<blah> HTTP/1.1
[skip]
> While wget does this:
>
> GET /<blah> HTTP/1.0
[skip]
> - Apache responds with a chunked response only to httplib. Why is that?

   Probably because wget uses HTTP/1.0?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From scott+python-dev at scottdial.com  Fri Sep  4 18:03:16 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Fri, 04 Sep 2009 12:03:16 -0400
Subject: [Python-Dev] how to debug httplib slowness
In-Reply-To: <4AA12C0F.1090305@simplistix.co.uk>
References: <4A82A20A.1070704@simplistix.co.uk>	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>	<4A82E120.4030105@simplistix.co.uk>	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>	<4AA0F5F2.6030903@simplistix.co.uk>	<fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
	<4AA12C0F.1090305@simplistix.co.uk>
Message-ID: <4AA13A44.3010008@scottdial.com>

Chris Withers wrote:
> - Apache in this instance responds with HTTP 1.1, even though the wget
> request was 1.0, is that allowed?
> 
> - Apache responds with a chunked response only to httplib. Why is that?
> 

I find it very confusing that you say "Apache" since your really want to
say "Coyote" which is to say "Tomcat".

Http11Processor.java at 1547:
    if (entityBody && http11 && keepAlive) {
        outputBuffer.addActiveFilter
            (outputFilters[Constants.CHUNKED_FILTER]);
        contentDelimitation = true;
        headers.addValue(
            Constants.TRANSFERENCODING).setString(Constants.CHUNKED);
    } else {
        outputBuffer.addActiveFilter
            (outputFilters[Constants.IDENTITY_FILTER]);
    }

So, as Oleg said, it's because httplib talks HTTP/1.1 and wget talks
HTTP/1.0. All HTTP/1.1 client must support chunked transfer-encoding,
and apparently Tomcat/Coyote defaults to that unless it is either an
empty message, not a HTTP/1.1 client, or the request is not to be kept
alive ("Connection: close" or no more keep-alive slots on the server).

As Simon said, changing this to do ''.join(chunks) is really the best
first step to take.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From status at bugs.python.org  Fri Sep  4 18:08:26 2009
From: status at bugs.python.org (Python tracker)
Date: Fri,  4 Sep 2009 18:08:26 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090904160826.85D3C785E8@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (08/28/09 - 09/04/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2374 open (+24) / 16285 closed (+18) / 18659 total (+42)

Open issues with patches:   939

Average duration of open issues: 661 days.
Median duration of open issues: 415 days.

Open Issues Breakdown
   open  2341 (+23)
pending    32 ( +1)

Issues Created Or Reopened (45)
_______________________________

Multiple buffer overflows in unicode processing                  09/02/09
       http://bugs.python.org/issue2620    reopened brett.cannon                  
       patch                                                                   

os.getpwent returns unsigned 32bit value, os.setuid refuses it   09/02/09
       http://bugs.python.org/issue5705    reopened brett.cannon                  
       patch, 64bit                                                            

math.log, log10 inconsistency                                    08/28/09
CLOSED http://bugs.python.org/issue6765    reopened tjreedy                       
                                                                               

invalid print in tkinter\test\test_ttk\test_widgets.py           08/28/09
CLOSED http://bugs.python.org/issue6796    created  keithc                        
                                                                               

When Beginning Expression with Lookahead Assertion I get no Matc 08/28/09
CLOSED http://bugs.python.org/issue6797    created  jwindle                       
                                                                               

Argument for sys.settrace() callbacks documented incorrectly     08/28/09
       http://bugs.python.org/issue6798    created  robert.kern                   
                                                                               

mimetypes does not give cannonical extension for guess_extension 08/29/09
       http://bugs.python.org/issue6799    created  ptarjan                       
                                                                               

os.exec* raises "OSError: [Errno 45] Operation not supported" in 08/29/09
       http://bugs.python.org/issue6800    created  rnk                           
                                                                               

symmetric_difference_update documentation fix                    08/30/09
CLOSED http://bugs.python.org/issue6801    reopened r.david.murray                
                                                                               

build fails on Snow Leopard                                      08/29/09
       http://bugs.python.org/issue6802    created  jmr                           
       patch                                                                   

Context manager docs refer to contextlib.contextmanager as conte 08/29/09
CLOSED http://bugs.python.org/issue6803    created  dhaffey                       
       patch                                                                   

IDLE: Detect Python files even if name doesn't end in .py        08/29/09
       http://bugs.python.org/issue6804    created  gagenellina                   
       patch                                                                   

Should be URL for documentation of current release of Python 3 ( 08/30/09
CLOSED http://bugs.python.org/issue6805    created  MLModel                       
                                                                               

test_platform fails under Snow Leopard                           08/30/09
CLOSED http://bugs.python.org/issue6806    created  brett.cannon                  
       easy                                                                    

No such file or directory: 'msisupport.dll' in msi.py            08/30/09
       http://bugs.python.org/issue6807    created  pds                           
                                                                               

python 3.1 documentation tutorial classes                        08/30/09
CLOSED http://bugs.python.org/issue6808    created  tom_morse                     
                                                                               

Python string.lstrip bug?                                        08/31/09
CLOSED http://bugs.python.org/issue6809    created  mushywushy                    
                                                                               

add link to the documentation of signal.signal                   08/31/09
CLOSED http://bugs.python.org/issue6810    created  Yinon                         
                                                                               

add a filename argument to marshal.load*                         08/31/09
       http://bugs.python.org/issue6811    created  brett.cannon                  
                                                                               

Snow Leopard python program fails because _PyType_Modified is mi 08/31/09
       http://bugs.python.org/issue6812    created  soshea                        
                                                                               

update format() documentation and tutorial                       09/01/09
CLOSED http://bugs.python.org/issue6813    created  grooverdan                    
       patch                                                                   

xrange removal from documentation                                09/01/09
CLOSED http://bugs.python.org/issue6814    created  grooverdan                    
       patch                                                                   

UnicodeDecodeError in os.path.expandvars                         09/01/09
       http://bugs.python.org/issue6815    created  shura_zam                     
                                                                               

Provide CPython command line functionality via runpy module      09/01/09
       http://bugs.python.org/issue6816    created  ncoghlan                      
                                                                               

char buffer in function posix_getcwdu should not be fix length   09/02/09
       http://bugs.python.org/issue6817    created  boya                          
       patch                                                                   

remove/delete method for zipfile/tarfile objects                 09/02/09
       http://bugs.python.org/issue6818    created  rossmclendon                  
                                                                               

Typo in datamodel.rst ("Custon" -> "Custom")                     09/02/09
CLOSED http://bugs.python.org/issue6819    created  jon                           
       patch                                                                   

Redefinition of HAVE_STRFTIME can cause compiler errors.         09/02/09
       http://bugs.python.org/issue6820    created  rgpitts                       
                                                                               

incorrect doc for PyBuffer_Release                               09/02/09
CLOSED http://bugs.python.org/issue6821    created  egreen                        
                                                                               

Error calling .storlines from ftplib                             09/02/09
       http://bugs.python.org/issue6822    created  rdevaughn                     
                                                                               

time.strftime does unnecessary range check                       09/02/09
       http://bugs.python.org/issue6823    created  rshapiro                      
       easy                                                                    

help for a module should list supported platforms                09/02/09
       http://bugs.python.org/issue6824    created  rickysarraf                   
                                                                               

Minor documentation bug with os.path.split                       09/02/09
       http://bugs.python.org/issue6825    created  evaned                        
                                                                               

Add __contains__ to range type                                   09/02/09
CLOSED http://bugs.python.org/issue6826    created  hpesoj                        
                                                                               

deepcopy erroneously doesn't call __setstate__ if __getstate__ r 09/02/09
       http://bugs.python.org/issue6827    created  shauncutts                    
                                                                               

wrongly highlighted blocks in the Tutorial                       09/03/09
CLOSED http://bugs.python.org/issue6828    created  gagenellina                   
       patch                                                                   

Frendly error message when inheriting from function              09/04/09
       http://bugs.python.org/issue6829    reopened techtonik                     
       easy                                                                    

Some uniformness in defaultdict                                  09/03/09
CLOSED http://bugs.python.org/issue6830    created  t-kamiya                      
                                                                               

2to3 assignment division conversion                              09/03/09
       http://bugs.python.org/issue6831    created  highfestiva                   
                                                                               

Outputting unicode crushes when printing to file on Linux        09/03/09
CLOSED http://bugs.python.org/issue6832    created  Orlowski                      
                                                                               

incorrect: failed local variable referenced before assignment    09/03/09
CLOSED http://bugs.python.org/issue6833    created  paultjuhatwork                
                                                                               

use different mechanism for pythonw on osx                       09/03/09
       http://bugs.python.org/issue6834    created  ronaldoussoren                
       easy                                                                    

doctest problem with decorated function when decorator is define 09/03/09
       http://bugs.python.org/issue6835    created  goetzpf                       
                                                                               

Mismatching use of memory APIs                                   09/04/09
       http://bugs.python.org/issue6836    created  krisvale                      
       patch                                                                   

Mark the compiler package as deprecated                          09/04/09
       http://bugs.python.org/issue6837    created  exarkun                       
                                                                               



Issues Now Closed (45)
______________________

poss. patch for fnmatch.py to add {.htm,html} style globbing      505 days
       http://bugs.python.org/issue2649    georg.brandl                  
                                                                               

webbrowser.py doesn't properly handle BROWSER env var             500 days
       http://bugs.python.org/issue2666    georg.brandl                  
       patch                                                                   

os.getcwd fails for long path names on linux                      490 days
       http://bugs.python.org/issue2722    boya                          
       patch, easy                                                             

bytearrays are not thread safe                                    440 days
       http://bugs.python.org/issue3139    loewis                        
       patch                                                                   

math.log(x, 10) gives different result than math.log10(x)         368 days
       http://bugs.python.org/issue3724    tjreedy                       
       patch                                                                   

library.pdf - Section 12.13.2 Connection Objects - example cut o  316 days
       http://bugs.python.org/issue4160    ezio.melotti                  
                                                                               

Remove Monterey support from configure.in                         222 days
       http://bugs.python.org/issue5047    georg.brandl                  
       patch                                                                   

compiler.parse raises SyntaxErrors without line number informati  221 days
       http://bugs.python.org/issue5064    georg.brandl                  
                                                                               

test_funcattrs truncated during unittest conversion               218 days
       http://bugs.python.org/issue5101    georg.brandl                  
                                                                               

help  related topic doesn't exist                                 204 days
       http://bugs.python.org/issue5221    georg.brandl                  
       easy                                                                    

BaseCookie.load doesn't create Morsel objects for mappings        200 days
       http://bugs.python.org/issue5275    georg.brandl                  
                                                                               

add conversion table to time module docs                           13 days
       http://bugs.python.org/issue5365    georg.brandl                  
                                                                               

tarfile normalizes arcname                                        102 days
       http://bugs.python.org/issue6054    lars.gustaebel                
                                                                               

should we include argparse                                         81 days
       http://bugs.python.org/issue6247    r.david.murray                
                                                                               

Added Misc/python.pc to 'distclean' Rule                           75 days
       http://bugs.python.org/issue6297    marketdickinson               
       patch                                                                   

inserting None into sys.modules does not raise ImportError with    36 days
       http://bugs.python.org/issue6563    brett.cannon                  
                                                                               

No update about automatic numbering of fields in format strings    38 days
       http://bugs.python.org/issue6579    eric.smith                    
                                                                               

test_telnetlib doesn't test Telnet.write                           39 days
       http://bugs.python.org/issue6582    rwanderley                    
       patch, easy                                                             

optparse parse_args argument references wrong                      28 days
       http://bugs.python.org/issue6638    georg.brandl                  
                                                                               

PyInit_shoddy() in shoddy.c does not return anything on success    13 days
       http://bugs.python.org/issue6732    georg.brandl                  
                                                                               

threading issue in __builtins__.print                               9 days
       http://bugs.python.org/issue6750    amaury.forgeotdarc            
       patch, needs review                                                     

Non-existent member 'nb_inplace_divide' in PyNumberMethods         11 days
       http://bugs.python.org/issue6754    georg.brandl                  
                                                                               

ftplib documentation does not document what the acct parameter i   14 days
       http://bugs.python.org/issue6756    georg.brandl                  
                                                                               

Marshal's documentation incomplete (Bools)                         13 days
       http://bugs.python.org/issue6757    georg.brandl                  
                                                                               

math.log, log10 inconsistency                                       3 days
       http://bugs.python.org/issue6765    georg.brandl                  
                                                                               

Python 2.6 tutorial still recommends using Exception.message att   11 days
       http://bugs.python.org/issue6777    georg.brandl                  
                                                                               

invalid print in tkinter\test\test_ttk\test_widgets.py              5 days
       http://bugs.python.org/issue6796    keithc                        
                                                                               

When Beginning Expression with Lookahead Assertion I get no Matc    1 days
       http://bugs.python.org/issue6797    marketdickinson               
                                                                               

symmetric_difference_update documentation fix                       1 days
       http://bugs.python.org/issue6801    georg.brandl                  
                                                                               

Context manager docs refer to contextlib.contextmanager as conte    0 days
       http://bugs.python.org/issue6803    georg.brandl                  
       patch                                                                   

Should be URL for documentation of current release of Python 3 (    0 days
       http://bugs.python.org/issue6805    georg.brandl                  
                                                                               

test_platform fails under Snow Leopard                              5 days
       http://bugs.python.org/issue6806    brett.cannon                  
       easy                                                                    

python 3.1 documentation tutorial classes                           2 days
       http://bugs.python.org/issue6808    georg.brandl                  
                                                                               

Python string.lstrip bug?                                           0 days
       http://bugs.python.org/issue6809    marketdickinson               
                                                                               

add link to the documentation of signal.signal                      1 days
       http://bugs.python.org/issue6810    georg.brandl                  
                                                                               

update format() documentation and tutorial                          0 days
       http://bugs.python.org/issue6813    eric.smith                    
       patch                                                                   

xrange removal from documentation                                   0 days
       http://bugs.python.org/issue6814    georg.brandl                  
       patch                                                                   

Typo in datamodel.rst ("Custon" -> "Custom")                        1 days
       http://bugs.python.org/issue6819    georg.brandl                  
       patch                                                                   

incorrect doc for PyBuffer_Release                                  0 days
       http://bugs.python.org/issue6821    georg.brandl                  
                                                                               

Add __contains__ to range type                                      0 days
       http://bugs.python.org/issue6826    marketdickinson               
                                                                               

wrongly highlighted blocks in the Tutorial                          0 days
       http://bugs.python.org/issue6828    georg.brandl                  
       patch                                                                   

Some uniformness in defaultdict                                     0 days
       http://bugs.python.org/issue6830    georg.brandl                  
                                                                               

Outputting unicode crushes when printing to file on Linux           1 days
       http://bugs.python.org/issue6832    Orlowski                      
                                                                               

incorrect: failed local variable referenced before assignment       0 days
       http://bugs.python.org/issue6833    marketdickinson               
                                                                               

Deleted files are reinstalled                                    1900 days
       http://bugs.python.org/issue977470  georg.brandl                  
                                                                               



Top Issues Most Discussed (10)
______________________________

 15 build fails on Snow Leopard                                        6 days
open    http://bugs.python.org/issue6802   

  8 Outputting unicode crushes when printing to file on Linux          1 days
closed  http://bugs.python.org/issue6832   

  6 remove/delete method for zipfile/tarfile objects                   2 days
open    http://bugs.python.org/issue6818   

  6 Support TLS SNI extension in ssl module                          156 days
open    http://bugs.python.org/issue5639   

  6 improve xrange.__contains__                                      764 days
open    http://bugs.python.org/issue1766304

  5 symmetric_difference_update documentation fix                      1 days
closed  http://bugs.python.org/issue6801   

  4 help for a module should list supported platforms                  2 days
open    http://bugs.python.org/issue6824   

  4 bytearrays are not thread safe                                   440 days
closed  http://bugs.python.org/issue3139   

  4 Add os.link() and os.symlink() and	os.path.islink() support for 1054 days
open    http://bugs.python.org/issue1578269

  3 attribute error after non-from import                           1876 days
open    http://bugs.python.org/issue992389 




From chris at simplistix.co.uk  Fri Sep  4 18:16:18 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 04 Sep 2009 17:16:18 +0100
Subject: [Python-Dev] httplib slowness solved - which branches to merge to?
In-Reply-To: <loom.20090904T133126-602@post.gmane.org>
References: <4A82A20A.1070704@simplistix.co.uk>	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>	<4A82E120.4030105@simplistix.co.uk>	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>	<4AA0F5F2.6030903@simplistix.co.uk>	<fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
	<loom.20090904T133126-602@post.gmane.org>
Message-ID: <4AA13D52.9060601@simplistix.co.uk>

Antoine Pitrou wrote:
> Simon Cross <hodgestar+pythondev <at> gmail.com> writes:
>> Well, since the source for _read_chunked includes the comment
>>
>>         # XXX This accumulates chunks by repeated string concatenation,
>>         # which is not efficient as the number or size of chunks gets big.
>>
>> you might gain some speed improvement with minimal effort by gathering
>> the read data chunks into a list and then returning "".join(chunks) at
>> the end.
> 
> +1 for trying this. Given differences between platforms in realloc()
> performance, it might be the reason why it goes unnoticed under Linux but
> degenerates under Windows.

And how! The following change dropped the download time using httplib to 
2.3 seconds:

http://svn.python.org/view/python/trunk/Lib/httplib.py?r1=74523&r2=74655

> As a sidenote, it is interesting that even an stdlib module makes this mistake
> and acknowledges it without trying to fix it.

No longer in this case ;-)

The fix is applied on the trunk, but the problem still exists on the 2.6 
branch, 3.1 branch and 3.2 branch.

Which of these should I merge to? I assume all of them?

Do I need to update any changelog files or similar to indicate this bug 
has been fixed?

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From solipsis at pitrou.net  Fri Sep  4 18:26:03 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 4 Sep 2009 16:26:03 +0000 (UTC)
Subject: [Python-Dev] httplib slowness solved - which branches to merge
	to?
References: <4A82A20A.1070704@simplistix.co.uk>	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>	<4A82E120.4030105@simplistix.co.uk>	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>	<4AA0F5F2.6030903@simplistix.co.uk>	<fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
	<loom.20090904T133126-602@post.gmane.org>
	<4AA13D52.9060601@simplistix.co.uk>
Message-ID: <loom.20090904T182345-904@post.gmane.org>

Chris Withers <chris <at> simplistix.co.uk> writes:
> 
> The fix is applied on the trunk, but the problem still exists on the 2.6 
> branch, 3.1 branch and 3.2 branch.
> 
> Which of these should I merge to? I assume all of them?

The performance problem is sufficiently serious that it should be considered a
bug so, yes, you should merge to all remaining branches (3.2, 2.6 and 3.1).

> Do I need to update any changelog files or similar to indicate this bug 
> has been fixed?

Yes, add an entry to Misc/NEWS under the "Library" section using the appropriate
conventions.

Thanks

Antoine.



From chris at simplistix.co.uk  Fri Sep  4 19:05:36 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 04 Sep 2009 18:05:36 +0100
Subject: [Python-Dev] test failure on py3k branch in test___all__
Message-ID: <4AA148E0.9050806@simplistix.co.uk>

Hi All,

Anyone know what's causing this failure?

test test___all__ failed -- Traceback (most recent call last):
   File "Lib/test/test___all__.py", line 106, in test_all
     self.check_all("profile")
   File "Lib/test/test___all__.py", line 23, in check_all
     exec("from %s import *" % modname, names)
   File "<string>", line 1, in <module>
AttributeError: 'module' object has no attribute 'help'

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From g.brandl at gmx.net  Fri Sep  4 19:14:04 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 04 Sep 2009 19:14:04 +0200
Subject: [Python-Dev] test failure on py3k branch in test___all__
In-Reply-To: <4AA148E0.9050806@simplistix.co.uk>
References: <4AA148E0.9050806@simplistix.co.uk>
Message-ID: <h7ri06$fsi$1@ger.gmane.org>

Chris Withers schrieb:
> Hi All,
> 
> Anyone know what's causing this failure?
> 
> test test___all__ failed -- Traceback (most recent call last):
>    File "Lib/test/test___all__.py", line 106, in test_all
>      self.check_all("profile")
>    File "Lib/test/test___all__.py", line 23, in check_all
>      exec("from %s import *" % modname, names)
>    File "<string>", line 1, in <module>
> AttributeError: 'module' object has no attribute 'help'

My fault -- fixed in r74661.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From guido at python.org  Fri Sep  4 20:22:21 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 4 Sep 2009 11:22:21 -0700
Subject: [Python-Dev] how to debug httplib slowness
In-Reply-To: <fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
References: <4A82A20A.1070704@simplistix.co.uk>
	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com> 
	<4A82E120.4030105@simplistix.co.uk>
	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com> 
	<4AA0F5F2.6030903@simplistix.co.uk>
	<fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
Message-ID: <ca471dc20909041122h2025f346h6fd4e6c8453fa420@mail.gmail.com>

On Fri, Sep 4, 2009 at 4:28 AM, Simon
Cross<hodgestar+pythondev at gmail.com> wrote:
> On Fri, Sep 4, 2009 at 1:11 PM, Chris Withers<chris at simplistix.co.uk> wrote:
>> Am I right in reading this as most of the time is being spent in httplib's
>> HTTPResponse._read_chunked and none of the methods it calls?
>>
>> If so, is there a better way that a bunch of print statements to find where
>> in that method the time is being spent?
>
> Well, since the source for _read_chunked includes the comment
>
> ? ? ? ?# XXX This accumulates chunks by repeated string concatenation,
> ? ? ? ?# which is not efficient as the number or size of chunks gets big.
>
> you might gain some speed improvement with minimal effort by gathering
> the read data chunks into a list and then returning "".join(chunks) at
> the end.

+1 on trying this. Constructing a 116MB string by concatenating 1KB
buffers surely must take forever. (116MB divided by 85125 recv() calls
give 1365 byte per chunk, which is awful.) The HTTP/1.0 business looks
like a red herring.

Also agreed that this is an embarrassment.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From chris at simplistix.co.uk  Fri Sep  4 21:04:49 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 04 Sep 2009 20:04:49 +0100
Subject: [Python-Dev] how to debug httplib slowness
In-Reply-To: <ca471dc20909041122h2025f346h6fd4e6c8453fa420@mail.gmail.com>
References: <4A82A20A.1070704@simplistix.co.uk>	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>
	<4A82E120.4030105@simplistix.co.uk>	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>
	<4AA0F5F2.6030903@simplistix.co.uk>	<fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
	<ca471dc20909041122h2025f346h6fd4e6c8453fa420@mail.gmail.com>
Message-ID: <4AA164D1.3060702@simplistix.co.uk>

Guido van Rossum wrote:
> +1 on trying this. Constructing a 116MB string by concatenating 1KB
> buffers surely must take forever. (116MB divided by 85125 recv() calls
> give 1365 byte per chunk, which is awful.) The HTTP/1.0 business looks
> like a red herring.
> 
> Also agreed that this is an embarrassment.

Embarrassment eradicated ;-)

http://bugs.python.org/issue6838

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From solipsis at pitrou.net  Fri Sep  4 21:15:18 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 4 Sep 2009 19:15:18 +0000 (UTC)
Subject: [Python-Dev] how to debug httplib slowness
References: <4A82A20A.1070704@simplistix.co.uk>
	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>
	<4A82E120.4030105@simplistix.co.uk>
	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>
	<4AA0F5F2.6030903@simplistix.co.uk>
	<fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
	<ca471dc20909041122h2025f346h6fd4e6c8453fa420@mail.gmail.com>
Message-ID: <loom.20090904T211403-966@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> +1 on trying this. Constructing a 116MB string by concatenating 1KB
> buffers surely must take forever. (116MB divided by 85125 recv() calls
> give 1365 byte per chunk, which is awful.)

It certainly is quite small but perhaps the server tries to stay below the
detected MTU?
(not that there is necessarily any point in doing so for most HTTP content,
IMO... except perhaps when the client does some progressive decoding)



From skippy.hammond at gmail.com  Sat Sep  5 02:30:16 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Sat, 05 Sep 2009 10:30:16 +1000
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <877hwlwmzv.fsf@hbox.dyndns.org>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>
	<877hwlwmzv.fsf@hbox.dyndns.org>
Message-ID: <4AA1B118.3030404@gmail.com>

On 30/08/2009 9:37 PM, Martin Geisler wrote:
> Mark Hammond<skippy.hammond at gmail.com>  writes:
>
>> 1) I've stalled on the 'none:' patch I promised to resurrect.  While
>> doing this, I re-discovered that the tests for win32text appear to
>> check win32 line endings are used by win32text on *all* platforms, not
>> just Windows.
>
> I think it is only Patrick Mezard who knows how to run (parts of) the
> test suite on Windows.
>
>> I asked for advice from Dirkjan who referred me to the mercurual-devel
>> list, but my request of slightly over a week ago remains unanswered
>> (http://selenic.com/pipermail/mercurial-devel/2009-August/014873.html)
>> -
>> maybe I just need to be more patient...
>
> Oh no, that's usually the wrong tactic :-) I've been too busy for real
> Mercurial work the last couple of weeks, but you should not feel bad
> about poking us if you don't get a reply. Or come to the IRC channel
> (#mercurial on irc.freenode.net) where Dirkjan (djc) and myself (mg)
> hang out when it's daytime in Europe.

To be fair, I did mail Dirkjan directly who referred me to the -develop 
list, which I did with a CC to him and a private mail asking for some 
help should the mail fall on deaf ears as I feared it would.  There 
really is only so far I'm willing to poke and prod people when I'm well 
aware we are all volunteers.

>> Further, Martin's comments in this thread indicate he believes a new
>> extension will be necessary rather than 'fixing' win32text.  If this
>> is the direction we take, it may mean the none: patch, which targets
>> the implementation of win32text, is no longer necessary anyway.
>
> I suggested a new extension for two reasons:

...

Thanks, and that does indeed sound excellent.  However, this is going a 
fair way beyond the original scope I signed up for.  While I was willing 
to help implement some new features into an existing extension, taking 
on the design and implementation of an entire new extension is something 
I'm not willing to undertake.  I don't think such an extension should 
even come from the Python community or it will end up being a 
python-only extension - or at best, will need to run the gauntlet of 2 
bike-shedding sessions from both the Python and hg communities which 
will waste much time.

What is the hope of an EOL extension which meets our requirements coming 
directly out of the hg community?  If that hope is small, where does 
that leave us?

Cheers,

Mark

From martin at v.loewis.de  Sat Sep  5 09:15:57 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 05 Sep 2009 09:15:57 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA1B118.3030404@gmail.com>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>	<877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com>
Message-ID: <4AA2102D.3080600@v.loewis.de>

> What is the hope of an EOL extension which meets our requirements coming
> directly out of the hg community?  If that hope is small, where does
> that leave us?

As before. I'll repost my request for help, and we stay with subversion
meanwhile.

Perhaps I'll post it to some mercurial list as well.

Regards,
Martin

From p.f.moore at gmail.com  Sat Sep  5 11:24:38 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 5 Sep 2009 10:24:38 +0100
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA2102D.3080600@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de>
Message-ID: <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com>

2009/9/5 "Martin v. L?wis" <martin at v.loewis.de>:
>> What is the hope of an EOL extension which meets our requirements coming
>> directly out of the hg community? ?If that hope is small, where does
>> that leave us?
>
> As before. I'll repost my request for help, and we stay with subversion
> meanwhile.
>
> Perhaps I'll post it to some mercurial list as well.

Can anyone (re-) post the specification of the proposed extension, to
the level that it is currently defined? I'm willing to make an attempt
to put together an extension, on the assumption that it'll be easier
to refine an implementation than continue discussing possibilities...

Paul.

From solipsis at pitrou.net  Sat Sep  5 13:44:36 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 5 Sep 2009 11:44:36 +0000 (UTC)
Subject: [Python-Dev] Mercurial migration: help needed
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>
	<877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com>
Message-ID: <loom.20090905T134046-455@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:
> 
> What is the hope of an EOL extension which meets our requirements coming 
> directly out of the hg community?  If that hope is small, where does 
> that leave us?

I'm starting to wonder what the problem really is that makes it so
Python-specific. If I understood correctly, it's about a couple of files which
must be stored using non-Unix line endings, right? (in the PC and PCbuild
directories?)

These files are hardly modified often and by many people (and even more rarely
by non-Windows people), so why not just put a verification hook on the server
and let the offending committer repair his mistake manually, if it ever happens?
(we can even provide a script to help repairing the EOL mistake, like
Tools/reindent.py does for indentation mistakes)



From martin at v.loewis.de  Sat Sep  5 15:19:43 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 05 Sep 2009 15:19:43 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <loom.20090905T134046-455@post.gmane.org>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>	<877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>
Message-ID: <4AA2656F.4040805@v.loewis.de>

> I'm starting to wonder what the problem really is that makes it so
> Python-specific. If I understood correctly, it's about a couple of files which
> must be stored using non-Unix line endings, right? (in the PC and PCbuild
> directories?)

No. It's about files that must, when checked out on Windows, have CRLF
endings, and, when checked out on Unix, have LF endings - i.e. all the
.py, .c, .h, and .rst files, plus a couple of others which don't require
specific treatment.

IOW, it's about the default behavior, and the majority of new files.

> These files are hardly modified often and by many people (and even more rarely
> by non-Windows people), so why not just put a verification hook on the server
> and let the offending committer repair his mistake manually, if it ever happens?
> (we can even provide a script to help repairing the EOL mistake, like
> Tools/reindent.py does for indentation mistakes)

This was Dirkjan's original proposal, and it is the proposal that brings
so much heat into the discussion, claiming that it is a problem of
minorities (I do understand that you were unaware that "the problem"
is really with the many files, not with the few).

In addition, a DVCS brings in another problem dimension: when people
push their changes, they have *already* committed them - and perhaps not
even they, but a contributor from which they had been pulling changes.
The bogus change may have been weeks ago, so the subversion solution
(of rejecting the commit to happen) doesn't quite work that well for
a DVCS.

Regards,
Martin

From solipsis at pitrou.net  Sat Sep  5 15:26:07 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 05 Sep 2009 15:26:07 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA2656F.4040805@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com>	<877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de>
Message-ID: <1252157167.5692.43.camel@localhost>

Le samedi 05 septembre 2009 ? 15:19 +0200, "Martin v. L?wis" a ?crit :
> No. It's about files that must, when checked out on Windows, have CRLF
> endings, and, when checked out on Unix, have LF endings - i.e. all the
> ..py, .c, .h, and .rst files, plus a couple of others which don't require
> specific treatment.
> 
> IOW, it's about the default behavior, and the majority of new files.

Ok, sorry for the misunderstanding and the lost bandwidth.

> In addition, a DVCS brings in another problem dimension: when people
> push their changes, they have *already* committed them - and perhaps not
> even they, but a contributor from which they had been pulling changes.
> The bogus change may have been weeks ago, so the subversion solution
> (of rejecting the commit to happen) doesn't quite work that well for
> a DVCS.

I don't think this problem is really serious.
If the push fails, you can just commit (locally) a new changeset that
repairs the EOL or indentation problems, and push the whole bunch of
changesets again (I assume the server-side hook will not examine
changesets individually, but only the last of them?).



From martin at v.loewis.de  Sat Sep  5 16:18:02 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 05 Sep 2009 16:18:02 +0200
Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help needed)
In-Reply-To: <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>
	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>
	<4AA2102D.3080600@v.loewis.de>
	<79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com>
Message-ID: <4AA2731A.9060001@v.loewis.de>

> Can anyone (re-) post the specification of the proposed extension, to
> the level that it is currently defined?

For reference, here are the original specification, mine and Martin
Geisler's:

http://mail.python.org/pipermail/python-dev/2009-August/090984.html
http://mail.python.org/pipermail/python-dev/2009-August/091453.html

Here is my attempt at summarizing it:

- name of versioned configuration file (in root of tree): .hgeol
- names of conversion modes: native, LF, CRLF
In the configuration file, there is a section [patterns] which
maps file name patterns to conversion modes, e.g.

[patterns]
**.txt = native
**.py = native
**.dsp = CRLF
**.bat = CRLF
Tools/bgen/README = native
Lib/email/test/data/msg_26.txt = CRLF

- Martin Geisler also proposes that there is a section
[repository]
native = <conversionmode>
I personally feel YAGNI; it should only support LF (adding such
a feature later may be considered)

Open issues:
- name of extension
- what should happen if the file on disk doesn't have the "expected"
line endings, or mixed line endings? E.g. a file declared as native
"should" have CRLF on Windows - what if it doesn't, on commit?
My proposal: do what svn does (whatever that is).

That's it, AFAICT. Martin Geisler also discussed something that
I read as an implementation strategy, by mapping the patterns to
into the (apparently existing) encode/decode configuration setting.

HTH,
Martin

P.S. If you decide that you will or will not work on it, please let
us know.

From stephen at xemacs.org  Sat Sep  5 16:59:26 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 05 Sep 2009 23:59:26 +0900
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <1252157167.5692.43.camel@localhost>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost>
Message-ID: <87ws4dcu9d.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > > In addition, a DVCS brings in another problem dimension: when people
 > > push their changes, they have *already* committed them - and perhaps not
 > > even they, but a contributor from which they had been pulling changes.
 > > The bogus change may have been weeks ago, so the subversion solution
 > > (of rejecting the commit to happen) doesn't quite work that well for
 > > a DVCS.

git has a nice filter-branch command, which would allow you to
automatically repair the problem (it works basically by checking out
each changeset and rerecording it with the appropriate commands).  I
know bzr is growing something similar, so presumably it is or will
soon be available in hg.

 > I don't think this problem is really serious.
 > If the push fails, you can just commit (locally) a new changeset that
 > repairs the EOL or indentation problems, and push the whole bunch of
 > changesets again (I assume the server-side hook will not examine
 > changesets individually, but only the last of them?).

That's not a very good solution.  Especially with typical Mercurial
workflows[1], it's quite possible that you'll have a number of bogus
changesets interleaved with good one.  I don't think recording a
repair is satisfactory.

Footnotes: 
[1]  Note that DVCS means you do *not* have to follow Python workflows
in your private branches.



From tjreedy at udel.edu  Sat Sep  5 16:26:36 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 05 Sep 2009 10:26:36 -0400
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA2656F.4040805@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>	<loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de>
Message-ID: <h7tser$tib$1@ger.gmane.org>

Martin v. L?wis wrote:
>> I'm starting to wonder what the problem really is that makes it so
>> Python-specific. If I understood correctly, it's about a couple of files which
>> must be stored using non-Unix line endings, right? (in the PC and PCbuild
>> directories?)
> 
> No. It's about files that must, when checked out on Windows, have CRLF
> endings, and, when checked out on Unix, have LF endings - i.e. all the
> .py, .c, .h, and .rst files, plus a couple of others which don't require
> specific treatment.
> 
> IOW, it's about the default behavior, and the majority of new files.

FWIW, I had the same impression as Antoine. I am aware that 'stupid'pad 
requires /r/n, but do IDLE and other editors (on Windows) that people 
would actually use to create/edit such files? I would personally be 
willing to install a notepad replacement if needed to quickview such files.

If essentially all text files need fixed line endings on Windows, then 
hg really needs this built in. Has it really not been used much on Windows?

tjr


From mg at lazybytes.net  Sat Sep  5 16:27:28 2009
From: mg at lazybytes.net (Martin Geisler)
Date: Sat, 05 Sep 2009 16:27:28 +0200
Subject: [Python-Dev] Mercurial migration: help needed
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost>
Message-ID: <87ws4dpiun.fsf@hbox.dyndns.org>

Antoine Pitrou <solipsis at pitrou.net> writes:

> Le samedi 05 septembre 2009 ? 15:19 +0200, "Martin v. L?wis" a ?crit :
>
>> In addition, a DVCS brings in another problem dimension: when people
>> push their changes, they have *already* committed them - and perhaps
>> not even they, but a contributor from which they had been pulling
>> changes. The bogus change may have been weeks ago, so the subversion
>> solution (of rejecting the commit to happen) doesn't quite work that
>> well for a DVCS.
>
> I don't think this problem is really serious. If the push fails, you
> can just commit (locally) a new changeset that repairs the EOL or
> indentation problems, and push the whole bunch of changesets again (I
> assume the server-side hook will not examine changesets individually,
> but only the last of them?).

Yes, the server-side hook will have to work like this in order for
people to fix mistakes like you just described.

-- 
Martin Geisler

VIFF (Virtual Ideal Functionality Framework) brings easy and efficient
SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090905/ade522ec/attachment-0001.pgp>

From solipsis at pitrou.net  Sat Sep  5 17:09:39 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 5 Sep 2009 15:09:39 +0000 (UTC)
Subject: [Python-Dev] Mercurial migration: help needed
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>	<loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <h7tser$tib$1@ger.gmane.org>
Message-ID: <loom.20090905T170751-92@post.gmane.org>

Terry Reedy <tjreedy <at> udel.edu> writes:
> 
> If essentially all text files need fixed line endings on Windows, then 
> hg really needs this built in. Has it really not been used much on Windows?

Mercurial is used by e.g. Mozilla, which is not really known for poor Windows
support (chances are many Firefox developers are Windows-based). I wonder
whether they have written their own extension, or if they simply rely on their
text editors to do the right thing.




From gzlist at googlemail.com  Sat Sep  5 17:19:18 2009
From: gzlist at googlemail.com (Martin (gzlist))
Date: Sat, 5 Sep 2009 16:19:18 +0100
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <loom.20090905T170751-92@post.gmane.org>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <h7tser$tib$1@ger.gmane.org>
	<loom.20090905T170751-92@post.gmane.org>
Message-ID: <d80792120909050819u51763e7as80b6da10ffbb2416@mail.gmail.com>

On 05/09/2009, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Terry Reedy <tjreedy <at> udel.edu> writes:
>>
>> If essentially all text files need fixed line endings on Windows, then
>> hg really needs this built in. Has it really not been used much on
>> Windows?
>
> Mercurial is used by e.g. Mozilla, which is not really known for poor
> Windows
> support (chances are many Firefox developers are Windows-based). I wonder
> whether they have written their own extension, or if they simply rely on
> their
> text editors to do the right thing.

Actually, most Firefox developers are mac based.

Mozilla isn't a great example of windows integration, they install
half-a-unix-system in order to just build under windows, including
msys, python 2.5, mercurial, and xemacs. See:
<https://developer.mozilla.org/en/Windows_Build_Prerequisites>

Martin

From dirkjan at ochtman.nl  Sat Sep  5 17:26:12 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sat, 05 Sep 2009 17:26:12 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <loom.20090905T170751-92@post.gmane.org>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>
	<h7tser$tib$1@ger.gmane.org>
	<loom.20090905T170751-92@post.gmane.org>
Message-ID: <4AA28314.6020205@ochtman.nl>

On 05/09/2009 17:09, Antoine Pitrou wrote:
> Mercurial is used by e.g. Mozilla, which is not really known for poor Windows
> support (chances are many Firefox developers are Windows-based). I wonder
> whether they have written their own extension, or if they simply rely on their
> text editors to do the right thing.

I'm pretty sure they don't have a specific extension for it. I don't 
know if many of their developers use the win32text extension, but I 
would guess not (I have been somewhat involved in Mozilla's migration).

Cheers,

Dirkjan

From dirkjan at ochtman.nl  Sat Sep  5 17:27:24 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sat, 05 Sep 2009 17:27:24 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <87ws4dcu9d.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>
	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>
	<1252157167.5692.43.camel@localhost>
	<87ws4dcu9d.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4AA2835C.6000002@ochtman.nl>

On 05/09/2009 16:59, Stephen J. Turnbull wrote:
> git has a nice filter-branch command, which would allow you to
> automatically repair the problem (it works basically by checking out
> each changeset and rerecording it with the appropriate commands).  I
> know bzr is growing something similar, so presumably it is or will
> soon be available in hg.

That means you change hashes on the server side, without human feedback. 
Let's try not to subvert the immutability design that Mercurial tries to 
encourage.

Cheers,

Dirkjan

From martin at v.loewis.de  Sat Sep  5 18:18:33 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 05 Sep 2009 18:18:33 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <1252157167.5692.43.camel@localhost>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>
	<1252157167.5692.43.camel@localhost>
Message-ID: <4AA28F59.5030805@v.loewis.de>

> I don't think this problem is really serious.
> If the push fails, you can just commit (locally) a new changeset that
> repairs the EOL or indentation problems

I would find that unfortunate. It's a fairly irrelevant change, yet
it may manage to corrupt the history (hg blame).

> and push the whole bunch of
> changesets again (I assume the server-side hook will not examine
> changesets individually, but only the last of them?).

That is for us to decide. I can see arguments either way.

But it shouldn't happen often that the server refuses a push;
all errors should already be caught on the clients.

Regards,
Martin

From martin at v.loewis.de  Sat Sep  5 18:20:16 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 05 Sep 2009 18:20:16 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <87ws4dpiun.fsf@hbox.dyndns.org>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>
	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>
	<1252157167.5692.43.camel@localhost>
	<87ws4dpiun.fsf@hbox.dyndns.org>
Message-ID: <4AA28FC0.9010505@v.loewis.de>

>> I don't think this problem is really serious. If the push fails, you
>> can just commit (locally) a new changeset that repairs the EOL or
>> indentation problems, and push the whole bunch of changesets again (I
>> assume the server-side hook will not examine changesets individually,
>> but only the last of them?).
> 
> Yes, the server-side hook will have to work like this in order for
> people to fix mistakes like you just described.

Not necessarily. People could also be required to go back and replay
all changes.

Regards,
Martin

From dirkjan at ochtman.nl  Sat Sep  5 18:25:36 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sat, 5 Sep 2009 18:25:36 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA28F59.5030805@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost>
	<4AA28F59.5030805@v.loewis.de>
Message-ID: <ea2499da0909050925iaf26fcdm455a547a94bb079@mail.gmail.com>

On Sat, Sep 5, 2009 at 18:18, "Martin v. L?wis"<martin at v.loewis.de> wrote:
> But it shouldn't happen often that the server refuses a push;
> all errors should already be caught on the clients.

We could just mandate the same hook code as a commit hook.

Cheers,

Dirkjan

From martin at v.loewis.de  Sat Sep  5 18:25:46 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 05 Sep 2009 18:25:46 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <h7tser$tib$1@ger.gmane.org>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>
	<h7tser$tib$1@ger.gmane.org>
Message-ID: <4AA2910A.4070504@v.loewis.de>

> FWIW, I had the same impression as Antoine. I am aware that 'stupid'pad
> requires /r/n, but do IDLE and other editors (on Windows) that people
> would actually use to create/edit such files? I would personally be
> willing to install a notepad replacement if needed to quickview such files.

Visual Studio will create files with CRLF endings. Please don't talk
people out of using Visual Studio for development. More generally:
please accept that it is consensus that this *has* to be fixed, and
that it *is* a problem on Windows.

> If essentially all text files need fixed line endings on Windows, then
> hg really needs this built in. Has it really not been used much on Windows?

I think that's the case. It's pretty much a Unix-only tool, like most of
the other DVCS implementations.

FWIW, I tried to check out Mozilla (which is in hg), and the check out
would always abort with a timeout. Then I downloaded a bundle that they
had produced, and try to unbundle it. It took all night, but was
complete the other morning. Trying to update the checkout would again
make me run into http timeouts. I tried the same on Linux, and it
completed within a few minutes. So I conclude that, from a certain
project size on, hg is unusable on Windows, atleast on my office
machine, running Windows 7.

Regards,
Martin

From martin at v.loewis.de  Sat Sep  5 18:28:00 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 05 Sep 2009 18:28:00 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <ea2499da0909050925iaf26fcdm455a547a94bb079@mail.gmail.com>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>
	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>
	<1252157167.5692.43.camel@localhost>	<4AA28F59.5030805@v.loewis.de>
	<ea2499da0909050925iaf26fcdm455a547a94bb079@mail.gmail.com>
Message-ID: <4AA29190.90500@v.loewis.de>

>> But it shouldn't happen often that the server refuses a push;
>> all errors should already be caught on the clients.
> 
> We could just mandate the same hook code as a commit hook.

I would be in favor (although, IIUC, "mandate" here would be a
social thing, not a technical one).

Regards,
Martin

From dirkjan at ochtman.nl  Sat Sep  5 18:38:55 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sat, 05 Sep 2009 18:38:55 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA29190.90500@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>
	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>
	<1252157167.5692.43.camel@localhost>	<4AA28F59.5030805@v.loewis.de>
	<ea2499da0909050925iaf26fcdm455a547a94bb079@mail.gmail.com>
	<4AA29190.90500@v.loewis.de>
Message-ID: <4AA2941F.3020000@ochtman.nl>

On 05/09/2009 18:28, "Martin v. L?wis" wrote:
> I would be in favor (although, IIUC, "mandate" here would be a
> social thing, not a technical one).

Right, but that would be the same for the extension.

Cheers,

Dirkjan

From g.brandl at gmx.net  Sat Sep  5 18:56:12 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 05 Sep 2009 18:56:12 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA28F59.5030805@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>	<1252157167.5692.43.camel@localhost>
	<4AA28F59.5030805@v.loewis.de>
Message-ID: <h7u5ao$js2$2@ger.gmane.org>

Martin v. L?wis schrieb:
>> I don't think this problem is really serious.
>> If the push fails, you can just commit (locally) a new changeset that
>> repairs the EOL or indentation problems
> 
> I would find that unfortunate. It's a fairly irrelevant change, yet
> it may manage to corrupt the history (hg blame).

I'm for a per-changeset check as well.  In the common case, the client will
have the "required" extension, and errors will be caught there.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From dirkjan at ochtman.nl  Sat Sep  5 19:02:39 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sat, 5 Sep 2009 19:02:39 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA2910A.4070504@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <h7tser$tib$1@ger.gmane.org>
	<4AA2910A.4070504@v.loewis.de>
Message-ID: <ea2499da0909051002p20dc2270vd74fbf516644f5db@mail.gmail.com>

On Sat, Sep 5, 2009 at 18:25, "Martin v. L?wis"<martin at v.loewis.de> wrote:
> I think that's the case. It's pretty much a Unix-only tool, like most of
> the other DVCS implementations.

I know a lot of projects use Mercurial on Windows as well, I'm not
aware of any big problems with it.

> FWIW, I tried to check out Mozilla (which is in hg), and the check out
> would always abort with a timeout. Then I downloaded a bundle that they
> had produced, and try to unbundle it. It took all night, but was
> complete the other morning. Trying to update the checkout would again
> make me run into http timeouts. I tried the same on Linux, and it
> completed within a few minutes. So I conclude that, from a certain
> project size on, hg is unusable on Windows, atleast on my office
> machine, running Windows 7.

That sounds pretty bad. By check out, do you mean the clone (getting
data over the wire) or the actual check out (setting up a working
directory)? I think I've heard of problems with the clone part before,
for them. We're actually working on improving clone size, though it
also seems to have to do with network reliability.

Cheers,

Dirkjan

From p.f.moore at gmail.com  Sat Sep  5 19:26:23 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 5 Sep 2009 18:26:23 +0100
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA2910A.4070504@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <h7tser$tib$1@ger.gmane.org>
	<4AA2910A.4070504@v.loewis.de>
Message-ID: <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com>

2009/9/5 "Martin v. L?wis" <martin at v.loewis.de>:
>> FWIW, I had the same impression as Antoine. I am aware that 'stupid'pad
>> requires /r/n, but do IDLE and other editors (on Windows) that people
>> would actually use to create/edit such files? I would personally be
>> willing to install a notepad replacement if needed to quickview such files.
>
> Visual Studio will create files with CRLF endings. Please don't talk
> people out of using Visual Studio for development. More generally:
> please accept that it is consensus that this *has* to be fixed, and
> that it *is* a problem on Windows.

(Disclaimer: I have no problem with accepting that the extension is
needed - the following is for clarity, in particular to help me
understand how the hook will be used)

There are 2 separate questions - (1) what is held in the repository,
and (2) what is in the user's workspace. The two clearly interact.

Taking (2) first, there are *some* files (very few, I believe) that
require specific line endings (CRLF - Visual Studio build files, is my
understanding). There are also tools that require fixed line endings
for input (notepad, Visual Studio). Finally, tools create new files
with certain line endings by default (pretty much guaranteed to be
platform-native, I'd expect). The result is that user workspaces *may*
(quite probably, will) contain files with a mixture of line endings if
care is not taken.

As regards (1), I assume that for "text" files, a consistent EOL
convention (assumed LF) should be used in the repository. It's not
clear to me what should be held in the repo for the files requiring
specific line endings - my instinct is that they should be treated as
"binary" and stored in the repo with the mandated line endings, and
checked out unchanged.

So we have the following situation:

- Some "binary" files which should never be converted
- Some "text" files, which must be held in LF form in the repo

My view is that how I store text files in my workspace is entirely up
to me (and the capabilities of my tools). So, how files get checked
out should not be centrally mandated. (Hmm, that may complicate
matters). How files are checked in is crucial, so a setting is
required which ensures that each file so marked is converted to LF
format on checking - effectively working like universal newline mode.

So, the issues:

1. Given that the "problematic" tools (notepad and Visual Studio) are
Windows tools, we seem to be back to the idea that this extension is
only needed by Windows developers. As I understood the consensus to be
that the extension should be for all users, I suspect I've missed
something.
2. Allowing text files to be checked out in whatever form the user
prefers seems complicated. The alternative would likely be to say test
files are checked out in "native" form. That works, but would irritate
me as I work on Windows, but prefer strongly to use LF line endings
(yes, I realise that makes me an oddball...) I'd put up with it if it
was the consensus to do this, of course.
3. Is there a realistic possibility that a user could edit one of the
CRLF-requiring files with a tool that converts it to LF? If so, is
there a need to trap that programmatically (as opposed to simply
accepting that this equates to the individual accidentally breaking
the build, no worse or better than checking in a C file with a syntax
error)?

Is this a fair summary?

Paul

From martin at v.loewis.de  Sat Sep  5 19:37:57 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 05 Sep 2009 19:37:57 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <ea2499da0909051002p20dc2270vd74fbf516644f5db@mail.gmail.com>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>
	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>
	<h7tser$tib$1@ger.gmane.org>	<4AA2910A.4070504@v.loewis.de>
	<ea2499da0909051002p20dc2270vd74fbf516644f5db@mail.gmail.com>
Message-ID: <4AA2A1F5.7090406@v.loewis.de>

> I know a lot of projects use Mercurial on Windows as well, I'm not
> aware of any big problems with it.

I trust that indeed, there are no big problems for most users. I
also trust that the hg developers are, in general, open to incorporating
improvements on Windows.

I'm still skeptical though whether Mercurial is really usable on
Windows. Your statement is a somewhat self-fulfilling prophecy:
people who run into big problems early likely aren't going to use
it. So lack of reports doesn't really mean there aren't any problems.

>> FWIW, I tried to check out Mozilla (which is in hg), and the check out
>> would always abort with a timeout. Then I downloaded a bundle that they
>> had produced, and try to unbundle it. It took all night, but was
>> complete the other morning. Trying to update the checkout would again
>> make me run into http timeouts. I tried the same on Linux, and it
>> completed within a few minutes. So I conclude that, from a certain
>> project size on, hg is unusable on Windows, atleast on my office
>> machine, running Windows 7.
> 
> That sounds pretty bad. By check out, do you mean the clone (getting
> data over the wire) or the actual check out (setting up a working
> directory)?

Creating the clone. ISTM that it leaves the http connection open while
doing stuff locally (or creates multiple of them, and one times out).

It starts cloning, and then, after an hour or so, it reports ABORT,
and rolls back, for no apparent reason.

> I think I've heard of problems with the clone part before,
> for them. We're actually working on improving clone size, though it
> also seems to have to do with network reliability.

Our institute has generally a really good internet connection;
I think hg.mozilla.org does as well. Plus, it worked when doing
it on the very same machine on Linux.

Regards,
Martin

From martin at v.loewis.de  Sat Sep  5 19:56:01 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 05 Sep 2009 19:56:01 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>
	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>
	<h7tser$tib$1@ger.gmane.org>	<4AA2910A.4070504@v.loewis.de>
	<79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com>
Message-ID: <4AA2A631.3010508@v.loewis.de>

> There are 2 separate questions - (1) what is held in the repository,
> and (2) what is in the user's workspace. The two clearly interact.
[...]
> As regards (1), I assume that for "text" files, a consistent EOL
> convention (assumed LF) should be used in the repository.

Correct. I believe Martin (the other one) proposed to make it
configurable. I agree that using LF in the repository is sensible.

Wrt. "should": there is the debate whether intermediate revisions
can deviate from that requirement.

> It's not
> clear to me what should be held in the repo for the files requiring
> specific line endings - my instinct is that they should be treated as
> "binary" and stored in the repo with the mandated line endings, and
> checked out unchanged.

I would say so, yes. One consequence of that is that if you change
your mind in hgeols, you need to commit all files that now fail to
conform. This is what happens with svn, and it may be tricky to
implement as you need to commit files that didn't change on disk
(say you switch from native to CRLF on a Windows checkout).

OTOH, even if you do store all text files in LF in the repo, then
you would still have the problem for files that go from unspecified
eol-style to a specified one. So changing hgeols is tricky, period.

> - Some "binary" files which should never be converted
> - Some "text" files, which must be held in LF form in the repo
> 
> My view is that how I store text files in my workspace is entirely up
> to me (and the capabilities of my tools). So, how files get checked
> out should not be centrally mandated.

Not by technical means, no. In the developer FAQ, there will be clear
directions, and you better follow them - or need to accept the blame
if you made a mistake because you ignored them.

> So, the issues:
> 
> 1. Given that the "problematic" tools (notepad and Visual Studio) are
> Windows tools, we seem to be back to the idea that this extension is
> only needed by Windows developers. As I understood the consensus to be
> that the extension should be for all users, I suspect I've missed
> something.

Technically, yes, it is only needed on Windows. The desire to have
all users use them come from the wish that problems with the setup
will be detected earlier. E.g. if the extension stops working with
a new Mercurial version, and all users use them, there is a larger
motivation to fix them for all users.

Things that the extension can do for you on Unix:
- check that the syntax of .hgeols is correct; this may affect
  Unix users which try to edit it
- check that all text files have consistent line endings, and
  refuse checkin if they don't. This may become relevant if
  a Unix text editor tries to edit a CRLF file, and doesn't
  quite detect that.

> 2. Allowing text files to be checked out in whatever form the user
> prefers seems complicated. The alternative would likely be to say test
> files are checked out in "native" form. That works, but would irritate
> me as I work on Windows, but prefer strongly to use LF line endings
> (yes, I realise that makes me an oddball...) I'd put up with it if it
> was the consensus to do this, of course.

It's consensus, and it's also what subversion does, and CVS did.

> 3. Is there a realistic possibility that a user could edit one of the
> CRLF-requiring files with a tool that converts it to LF? If so, is
> there a need to trap that programmatically (as opposed to simply
> accepting that this equates to the individual accidentally breaking
> the build, no worse or better than checking in a C file with a syntax
> error)?

I think everything you can imagine is also realistic. For the
less-realistic cases, it may be better if the commit is refused rather
than silently fixing it, since the user operated the system in a
surprising way - so he may actually have meant to do it that way.

One specific case is recent autoconf, which put a CR character
into configure, completely breaking svn's eol handling.

Regards,
Martin

From martin at v.loewis.de  Sat Sep  5 20:04:29 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 05 Sep 2009 20:04:29 +0200
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA2A631.3010508@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>	<4A99E77A.5080603@gmail.com>	<877hwlwmzv.fsf@hbox.dyndns.org>	<4AA1B118.3030404@gmail.com>	<loom.20090905T134046-455@post.gmane.org>	<4AA2656F.4040805@v.loewis.de>	<h7tser$tib$1@ger.gmane.org>	<4AA2910A.4070504@v.loewis.de>	<79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com>
	<4AA2A631.3010508@v.loewis.de>
Message-ID: <4AA2A82D.2090100@v.loewis.de>

>> 2. Allowing text files to be checked out in whatever form the user
>> prefers seems complicated. The alternative would likely be to say test
>> files are checked out in "native" form. That works, but would irritate
>> me as I work on Windows, but prefer strongly to use LF line endings
>> (yes, I realise that makes me an oddball...) I'd put up with it if it
>> was the consensus to do this, of course.
> 
> It's consensus, and it's also what subversion does, and CVS did.

Following up to myself: you might want to make it a feature that
"native" is configurable, per user or per repo. It should default
to CRLF on Windows, but you might want to set it to LF on your
system. In that case, the extension would have just its checking
functionality.

Regards,
Martin

From gzlist at googlemail.com  Sat Sep  5 20:06:46 2009
From: gzlist at googlemail.com (Martin (gzlist))
Date: Sat, 5 Sep 2009 19:06:46 +0100
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA2A1F5.7090406@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com>
	<877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <h7tser$tib$1@ger.gmane.org>
	<4AA2910A.4070504@v.loewis.de>
	<ea2499da0909051002p20dc2270vd74fbf516644f5db@mail.gmail.com>
	<4AA2A1F5.7090406@v.loewis.de>
Message-ID: <d80792120909051106x771b7c30sd36f7e464422ef2c@mail.gmail.com>

On 05/09/2009, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> Creating the clone. ISTM that it leaves the http connection open while
> doing stuff locally (or creates multiple of them, and one times out).
>
> It starts cloning, and then, after an hour or so, it reports ABORT,
> and rolls back, for no apparent reason.

I have been tracking mozilla-central, and believe this is a problem
with the repo, that started some time after Aug 04 - which is the last
log entry I have in my clone. I presumed it was just a problem my end
so hadn't got round trying to debug it yet. If it
is a general problem, the fact it's been around for about a month
without being addressed might indicate how well tested most DVCSes are
under windows.

Martin

From mg at lazybytes.net  Sat Sep  5 21:34:05 2009
From: mg at lazybytes.net (Martin Geisler)
Date: Sat, 05 Sep 2009 21:34:05 +0200
Subject: [Python-Dev] Mercurial migration: help needed
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost>
	<87ws4dpiun.fsf@hbox.dyndns.org> <4AA28FC0.9010505@v.loewis.de>
Message-ID: <87pra5p4nm.fsf@hbox.dyndns.org>

"Martin v. L?wis" <martin at v.loewis.de> writes:

>>> I don't think this problem is really serious. If the push fails, you
>>> can just commit (locally) a new changeset that repairs the EOL or
>>> indentation problems, and push the whole bunch of changesets again
>>> (I assume the server-side hook will not examine changesets
>>> individually, but only the last of them?).
>> 
>> Yes, the server-side hook will have to work like this in order for
>> people to fix mistakes like you just described.
>
> Not necessarily. People could also be required to go back and replay
> all changes.

Replaying changes, i.e., editing history is quite easy as long as you
have done no merges. So people can indeed fix their mistakes by cleaning
up history as long as they have a linear history. Both mq and histedit
are available for this:

  http://mercurial.selenic.com/wiki/MqExtension
  http://mercurial.selenic.com/wiki/HisteditExtension

The problem comes if a small group have been working together on a new
feature and have merged changes in from the mainline while doing so.
They will then no longer be able to edit past the most recent merge.

-- 
Martin Geisler

VIFF (Virtual Ideal Functionality Framework) brings easy and efficient
SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090905/13c5153c/attachment.pgp>

From brett at python.org  Sat Sep  5 22:20:14 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 5 Sep 2009 13:20:14 -0700
Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help
	needed)
In-Reply-To: <4AA2731A.9060001@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com> 
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> 
	<4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de> 
	<79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> 
	<4AA2731A.9060001@v.loewis.de>
Message-ID: <bbaeab100909051320l533340c4y2157bfd00e40566c@mail.gmail.com>

On Sat, Sep 5, 2009 at 07:18, "Martin v. L?wis"<martin at v.loewis.de> wrote:
>> Can anyone (re-) post the specification of the proposed extension, to
>> the level that it is currently defined?
>
> For reference, here are the original specification, mine and Martin
> Geisler's:
>
> http://mail.python.org/pipermail/python-dev/2009-August/090984.html
> http://mail.python.org/pipermail/python-dev/2009-August/091453.html
>
> Here is my attempt at summarizing it:
>
> - name of versioned configuration file (in root of tree): .hgeol
> - names of conversion modes: native, LF, CRLF
> In the configuration file, there is a section [patterns] which
> maps file name patterns to conversion modes, e.g.
>
> [patterns]
> **.txt = native
> **.py = native
> **.dsp = CRLF
> **.bat = CRLF
> Tools/bgen/README = native
> Lib/email/test/data/msg_26.txt = CRLF
>
> - Martin Geisler also proposes that there is a section
> [repository]
> native = <conversionmode>
> I personally feel YAGNI; it should only support LF (adding such
> a feature later may be considered)

Do you mean what native is in the repo or what it should be considered
on the user's machine? If it's the former then I actually like it as
it means a clone doesn't need to do anything special when 'native'
matches what is expected in the repo while a commit still does its EOL
validation. I still think we need to have a server-side block which
rejects commits that messes up the line-endings so people can fix
them. Shouldn't mess up 'blame' as the messed up line-endings should
only be from their edits. Otherwise it's just like when Tim used to
run reindent.py over the entire repo on occasion.

And as mentioned in another email by Paul, it would be nice to let the
user specify what they want 'native' to be on their machine if they
happen to be a Windows user who prefers LF.

>
> Open issues:
> - name of extension

StupidLineEndings =)

> - what should happen if the file on disk doesn't have the "expected"
> line endings, or mixed line endings? E.g. a file declared as native
> "should" have CRLF on Windows - what if it doesn't, on commit?
> My proposal: do what svn does (whatever that is).

Or refuse the commit with a message and tell the user to fix it (if
svn doesn't happen to do that).

-Brett

From wiggin15 at gmail.com  Sat Sep  5 23:26:14 2009
From: wiggin15 at gmail.com (Ender Wiggin)
Date: Sun, 6 Sep 2009 00:26:14 +0300
Subject: [Python-Dev] Problems with hex-conversion functions
Message-ID: <8449046c0909051426g2eaa3f94s4c5842c25af17abf@mail.gmail.com>

Hello everyone.

I see several problems with the two hex-conversion function pairs that
Python offers:
1. binascii.hexlify and binascii.unhexlify
2. bytes.fromhex and bytes.hex

Problem #1:
bytes.hex is not implemented, although it was specified in PEP 358.
This means there is no symmetrical function to accompany bytes.fromhex.

Problem #2:
Both pairs perform the same function, although The Zen Of Python suggests
that
"There should be one-- and preferably only one --obvious way to do it."
I do not understand why PEP 358 specified the bytes function pair although
it mentioned the binascii pair...

Problem #3:
bytes.fromhex may receive spaces in the input string, although
binascii.unhexlify may not.
I see no good reason for these two functions to have different features.

Problem #4:
binascii.unhexlify may receive both input types: strings or bytes, whereas
bytes.fromhex raises an exception when given a bytes parameter.
Again there is no reason for these functions to be different.

Problem #5:
binascii.hexlify returns a bytes type - although ideally, converting to hex
should
always return string types and converting from hex should always return
bytes.
IMO there is no meaning of bytes as an output of hexlify, since the output
is a
representation of other bytes.
This is also the suggested behavior of bytes.hex in PEP 358


Problems #4 and #5 call for a decision about the input and output of the
functions being discussed:

Option A : Strict input and output
unhexlify (and bytes.fromhex) may only receives string and may only return
bytes
hexlify (and bytes.hex) may only receives bytes and may only return strings

Option B : Robust input and strict output
unhexlify (and bytes.fromhex) may receive bytes and strings and may only
return bytes
hexlify (and bytes.hex) may receive bytes or strings and may only return
strings

Of course we may also consider a third option, which will allow the return
type of
all functions to be robust (perhaps specified in a keyword argument), but as
I wrote in
the description of problem #5, I see no sense in that.

Note that PEP 3137 describes: "... the more strict definitions of encoding
and decoding in
Python 3000: encoding always takes a Unicode string and returns a bytes
sequence, and decoding
always takes a bytes sequence and returns a Unicode string." - suggesting
option A.

To repeat problems #4 and #5, the current behavior does not match any
option:
* The return type of binascii.hexlify should be string, and this is not the
current behavior.
As for the input:
* Option A is not the current behavior because binascii.unhexlify may
receive both input types.
* Option B is not the current behavior because bytes.fromhex does not allow
bytes as input.


To fix these issues, three changes should be applied:
1. Deprecate bytes.fromhex. This fixes the following problems:
   #4 (go with option B and remove the function that does not allow bytes
input)
   #2 (the binascii functions will be the only way to "do it")
   #1 (bytes.hex should not be implemented)
2. In order to keep the functionality that bytes.fromhex has over unhexlify,
   the latter function should be able to handle spaces in its input (fix #3)
3. binascii.hexlify should return string as its return type (fix #5)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090906/04b585e0/attachment.htm>

From martin at v.loewis.de  Sun Sep  6 00:06:11 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 06 Sep 2009 00:06:11 +0200
Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help
 needed)
In-Reply-To: <bbaeab100909051320l533340c4y2157bfd00e40566c@mail.gmail.com>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de>
	<79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com>
	<4AA2731A.9060001@v.loewis.de>
	<bbaeab100909051320l533340c4y2157bfd00e40566c@mail.gmail.com>
Message-ID: <4AA2E0D3.3050206@v.loewis.de>

>> - Martin Geisler also proposes that there is a section
>> [repository]
>> native = <conversionmode>
>> I personally feel YAGNI; it should only support LF (adding such
>> a feature later may be considered)
> 
> Do you mean what native is in the repo or what it should be considered
> on the user's machine? 

The former.

> If it's the former then I actually like it as
> it means a clone doesn't need to do anything special when 'native'
> matches what is expected in the repo while a commit still does its EOL
> validation.

But the same would be true if the repo format would be always LF:
when "native" matches (which would then be on Unix), the extension
would *still* have to do nothing but validation.

> I still think we need to have a server-side block which
> rejects commits that messes up the line-endings so people can fix
> them.

Certainly.

> Shouldn't mess up 'blame' as the messed up line-endings should
> only be from their edits.

It could be that they had a number of commits that eventually lead
to the version that they push; this will also push the intermediate
versions. So when you then do a blame, it will tell you that the
revision was logged as "fix whitespace", rather than "resolve issue
#9743".

You are mostly right that the committer name would be the same
(except when the committer was pushing some changes pulled from
the actual contributor), however, I still see these whitespace-only
changes as a complication.

Regards,
Martin

From brett at python.org  Sun Sep  6 00:25:05 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 5 Sep 2009 15:25:05 -0700
Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help
	needed)
In-Reply-To: <4AA2E0D3.3050206@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com> 
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> 
	<4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de> 
	<79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> 
	<4AA2731A.9060001@v.loewis.de>
	<bbaeab100909051320l533340c4y2157bfd00e40566c@mail.gmail.com> 
	<4AA2E0D3.3050206@v.loewis.de>
Message-ID: <bbaeab100909051525v726d6fc1he7fc2dd5b0a5c599@mail.gmail.com>

On Sat, Sep 5, 2009 at 15:06, "Martin v. L?wis"<martin at v.loewis.de> wrote:
>>> - Martin Geisler also proposes that there is a section
>>> [repository]
>>> native = <conversionmode>
>>> I personally feel YAGNI; it should only support LF (adding such
>>> a feature later may be considered)
>>
>> Do you mean what native is in the repo or what it should be considered
>> on the user's machine?
>
> The former.
>
>> If it's the former then I actually like it as
>> it means a clone doesn't need to do anything special when 'native'
>> matches what is expected in the repo while a commit still does its EOL
>> validation.
>
> But the same would be true if the repo format would be always LF:
> when "native" matches (which would then be on Unix), the extension
> would *still* have to do nothing but validation.

Right, but I am just thinking about how we specify in .hgeols what the
repository is expected to be as this extension might work out nicely
for other projects who prefer CLRF as their repo-native line ending.

>
>> I still think we need to have a server-side block which
>> rejects commits that messes up the line-endings so people can fix
>> them.
>
> Certainly.
>
>> Shouldn't mess up 'blame' as the messed up line-endings should
>> only be from their edits.
>
> It could be that they had a number of commits that eventually lead
> to the version that they push; this will also push the intermediate
> versions. So when you then do a blame, it will tell you that the
> revision was logged as "fix whitespace", rather than "resolve issue
> #9743".
>

Yep.

> You are mostly right that the committer name would be the same
> (except when the committer was pushing some changes pulled from
> the actual contributor), however, I still see these whitespace-only
> changes as a complication.

It's unfortunate, but I see it as a rare occurrence as it would only
happen if someone got sloppy. And it should typically get caught
client-side before the commit ever occurs, minimizing the
whitespace-only commits even more.

-Brett

From martin at v.loewis.de  Sun Sep  6 00:35:26 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 06 Sep 2009 00:35:26 +0200
Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help
 needed)
In-Reply-To: <bbaeab100909051525v726d6fc1he7fc2dd5b0a5c599@mail.gmail.com>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de>
	<79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com>
	<4AA2731A.9060001@v.loewis.de>
	<bbaeab100909051320l533340c4y2157bfd00e40566c@mail.gmail.com>
	<4AA2E0D3.3050206@v.loewis.de>
	<bbaeab100909051525v726d6fc1he7fc2dd5b0a5c599@mail.gmail.com>
Message-ID: <4AA2E7AE.6020308@v.loewis.de>

> Right, but I am just thinking about how we specify in .hgeols what the
> repository is expected to be as this extension might work out nicely
> for other projects who prefer CLRF as their repo-native line ending.

This is what I refer to as YAGNI. Subversion has LF as the internal
storage, and, IIRC, so does CVS. I don't think there is any precedence
for wanting something else - and frankly, I can't see how repository
storage would matter.

>> You are mostly right that the committer name would be the same
>> (except when the committer was pushing some changes pulled from
>> the actual contributor), however, I still see these whitespace-only
>> changes as a complication.
> 
> It's unfortunate, but I see it as a rare occurrence as it would only
> happen if someone got sloppy. And it should typically get caught
> client-side before the commit ever occurs, minimizing the
> whitespace-only commits even more.

It would, of course, be possible to ban them altogether, at the expense
of users having to replay changes.

Regards,
Martin

From brett at python.org  Sun Sep  6 00:39:50 2009
From: brett at python.org (Brett Cannon)
Date: Sat, 5 Sep 2009 15:39:50 -0700
Subject: [Python-Dev] Problems with hex-conversion functions
In-Reply-To: <8449046c0909051426g2eaa3f94s4c5842c25af17abf@mail.gmail.com>
References: <8449046c0909051426g2eaa3f94s4c5842c25af17abf@mail.gmail.com>
Message-ID: <bbaeab100909051539o5ea1f5b9t5f73505f8b3b16a7@mail.gmail.com>

On Sat, Sep 5, 2009 at 14:26, Ender Wiggin<wiggin15 at gmail.com> wrote:
> Hello everyone.
>
> I see several problems with the two hex-conversion function pairs that
> Python offers:
> 1. binascii.hexlify and binascii.unhexlify
> 2. bytes.fromhex and bytes.hex
>
> Problem #1:
> bytes.hex is not implemented, although it was specified in PEP 358.

Probably an oversight.

> This means there is no symmetrical function to accompany bytes.fromhex.
>
> Problem #2:
> Both pairs perform the same function, although The Zen Of Python suggests
> that
> "There should be one-- and preferably only one --obvious way to do it."
> I do not understand why PEP 358 specified the bytes function pair although
> it mentioned the binascii pair...
>

It's nicer to have this kind of functionality on the built-ins than in
the standard library. "Practicality beats purity".

> Problem #3:
> bytes.fromhex may receive spaces in the input string, although
> binascii.unhexlify may not.
> I see no good reason for these two functions to have different features.
>

Well, one allows for sloppy input while the other does not. Usually
accepting sloppy input but giving strict input is better.

> Problem #4:
> binascii.unhexlify may receive both input types: strings or bytes, whereas
> bytes.fromhex raises an exception when given a bytes parameter.
> Again there is no reason for these functions to be different.

Well, giving bytes back into bytes seems somewhat silly. That's an
error in mixing your strings and bytes.

>
> Problem #5:
> binascii.hexlify returns a bytes type - although ideally, converting to hex
> should
> always return string types and converting from hex should always return
> bytes.
> IMO there is no meaning of bytes as an output of hexlify, since the output
> is a
> representation of other bytes.
> This is also the suggested behavior of bytes.hex in PEP 358
>
> Problems #4 and #5 call for a decision about the input and output of the
> functions being discussed:
>
> Option A : Strict input and output
> unhexlify (and bytes.fromhex) may only receives string and may only return
> bytes
> hexlify (and bytes.hex) may only receives bytes and may only return strings
>
> Option B : Robust input and strict output
> unhexlify (and bytes.fromhex) may receive bytes and strings and may only
> return bytes
> hexlify (and bytes.hex) may receive bytes or strings and may only return
> strings
>
> Of course we may also consider a third option, which will allow the return
> type of
> all functions to be robust (perhaps specified in a keyword argument), but as
> I wrote in
> the description of problem #5, I see no sense in that.
>
> Note that PEP 3137 describes: "... the more strict definitions of encoding
> and decoding in
> Python 3000: encoding always takes a Unicode string and returns a bytes
> sequence, and decoding
> always takes a bytes sequence and returns a Unicode string." - suggesting
> option A.
>
> To repeat problems #4 and #5, the current behavior does not match any
> option:
> * The return type of binascii.hexlify should be string, and this is not the
> current behavior.
> As for the input:
> * Option A is not the current behavior because binascii.unhexlify may
> receive both input types.
> * Option B is not the current behavior because bytes.fromhex does not allow
> bytes as input.
>
> To fix these issues, three changes should be applied:
> 1. Deprecate bytes.fromhex. This fixes the following problems:
> ?? #4 (go with option B and remove the function that does not allow bytes
> input)
> ?? #2 (the binascii functions will be the only way to "do it")
> ?? #1 (bytes.hex should not be implemented)
> 2. In order to keep the functionality that bytes.fromhex has over unhexlify,
> ?? the latter function should be able to handle spaces in its input (fix #3)
> 3. binascii.hexlify should return string as its return type (fix #5)

Or we fix bytes.fromhex(), add bytes.hex() and deprecate binascii.(un)hexlify().

-Brett

From nyamatongwe at gmail.com  Sun Sep  6 00:56:55 2009
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Sun, 6 Sep 2009 08:56:55 +1000
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <ea2499da0909051002p20dc2270vd74fbf516644f5db@mail.gmail.com>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <h7tser$tib$1@ger.gmane.org>
	<4AA2910A.4070504@v.loewis.de>
	<ea2499da0909051002p20dc2270vd74fbf516644f5db@mail.gmail.com>
Message-ID: <50862ebd0909051556nf68ca7cm6e5d1bc83f40826a@mail.gmail.com>

Dirkjan Ochtman:

> I know a lot of projects use Mercurial on Windows as well, I'm not
> aware of any big problems with it.

   If you have a Windows-only project with CRLF files using Mercurial
then there is no line end problem as Mercurial preserves the CRLFs for
you. Line end problems occur on mixed projects where both Windows and
Unix tools are used.

   Neil

From nyamatongwe at gmail.com  Sun Sep  6 01:01:49 2009
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Sun, 6 Sep 2009 09:01:49 +1000
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <h7tser$tib$1@ger.gmane.org>
	<4AA2910A.4070504@v.loewis.de>
	<79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com>
Message-ID: <50862ebd0909051601p3a52cb4cv83b2cb121ce6424c@mail.gmail.com>

Paul Moore:

> 1. Given that the "problematic" tools (notepad and Visual Studio) are
> Windows tools, we seem to be back to the idea that this extension is
> only needed by Windows developers. As I understood the consensus to be
> that the extension should be for all users, I suspect I've missed
> something.

   Some of the problems come from users on Unix checking in files with
CRLF line ends that they have received using some other mechanism such
as sharing a disk between Windows and Linux. I was going to point to a
bad revision in a bzr housed project I work on but launchpad isn't
working currently. What happened was that an OS X user committed a set
of changes but with all the files having a different line ending to
the repository. The result is that it is no longer easy to track
changes before that revision. It also makes a check out larger.

   It would help in such cases for the commit command on Unix to
either automatically change any CRLF line ends to LF for text files
(but not files with an explicitly specified line end) or to display a
warning.

   Neil

From mg at lazybytes.net  Sun Sep  6 01:57:13 2009
From: mg at lazybytes.net (Martin Geisler)
Date: Sun, 06 Sep 2009 01:57:13 +0200
Subject: [Python-Dev] hgeol extension
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de>
	<79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com>
	<4AA2731A.9060001@v.loewis.de>
Message-ID: <87y6otndwm.fsf@hbox.dyndns.org>

"Martin v. L?wis" <martin at v.loewis.de> writes:

>> Can anyone (re-) post the specification of the proposed extension, to
>> the level that it is currently defined?
>
> For reference, here are the original specification, mine and Martin
> Geisler's:
>
> http://mail.python.org/pipermail/python-dev/2009-August/090984.html
> http://mail.python.org/pipermail/python-dev/2009-August/091453.html
>
> Here is my attempt at summarizing it:
>
> - name of versioned configuration file (in root of tree): .hgeol
> - names of conversion modes: native, LF, CRLF
> In the configuration file, there is a section [patterns] which
> maps file name patterns to conversion modes, e.g.
>
> [patterns]
> **.txt = native
> **.py = native
> **.dsp = CRLF
> **.bat = CRLF
> Tools/bgen/README = native
> Lib/email/test/data/msg_26.txt = CRLF
>
> - Martin Geisler also proposes that there is a section
> [repository]
> native = <conversionmode>
> I personally feel YAGNI; it should only support LF (adding such
> a feature later may be considered)

I don't think it's a good idea to store everything in LF in the
repository. Unlike Subversion, you cannot expect all interactions to
take place through the "eol-filter" we're implementing. Letting people
checkout a useful unfiltered clone would be possible if we know the
repository native format and convert back to that.

Anyway, it's a minor detail. More importantly, I've posted a simple,
rough extension that does this here:

  http://markmail.org/message/yj4so736t4cfdulv

I figured it would be better to discuss the design and implementation on
mercurial-devel since there are more Mercurial hackers there. I've CC'ed
a bunch of people from this thread to "seed" the discussion -- the rest
of you on python-devel are hereby invited to join :-)

  http://selenic.com/mailman/listinfo/mercurial-devel

-- 
Martin Geisler

VIFF (Virtual Ideal Functionality Framework) brings easy and efficient
SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090906/e105fda3/attachment.pgp>

From ncoghlan at gmail.com  Sun Sep  6 05:51:05 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 06 Sep 2009 13:51:05 +1000
Subject: [Python-Dev] Problems with hex-conversion functions
In-Reply-To: <bbaeab100909051539o5ea1f5b9t5f73505f8b3b16a7@mail.gmail.com>
References: <8449046c0909051426g2eaa3f94s4c5842c25af17abf@mail.gmail.com>
	<bbaeab100909051539o5ea1f5b9t5f73505f8b3b16a7@mail.gmail.com>
Message-ID: <4AA331A9.70209@gmail.com>

Brett Cannon wrote:
>> To fix these issues, three changes should be applied:
>> 1. Deprecate bytes.fromhex. This fixes the following problems:
>>    #4 (go with option B and remove the function that does not allow bytes
>> input)
>>    #2 (the binascii functions will be the only way to "do it")
>>    #1 (bytes.hex should not be implemented)
>> 2. In order to keep the functionality that bytes.fromhex has over unhexlify,
>>    the latter function should be able to handle spaces in its input (fix #3)
>> 3. binascii.hexlify should return string as its return type (fix #5)
> 
> Or we fix bytes.fromhex(), add bytes.hex() and deprecate binascii.(un)hexlify().

binascii is the legacy approach here, so if anything was to go, those
functions would be it. I'm not sure getting rid of them is worth the
hassle though (especially in 2.x).

Regarding bytes.hex(), it may be better to modify the builtin hex()
function to accept bytes as an input type.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From stephen at xemacs.org  Sun Sep  6 09:41:24 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 06 Sep 2009 16:41:24 +0900
Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help
 needed)
In-Reply-To: <4AA2E7AE.6020308@v.loewis.de>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de>
	<79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com>
	<4AA2731A.9060001@v.loewis.de>
	<bbaeab100909051320l533340c4y2157bfd00e40566c@mail.gmail.com>
	<4AA2E0D3.3050206@v.loewis.de>
	<bbaeab100909051525v726d6fc1he7fc2dd5b0a5c599@mail.gmail.com>
	<4AA2E7AE.6020308@v.loewis.de>
Message-ID: <873a70y0yj.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:

 > This is what I refer to as YAGNI. Subversion has LF as the internal
 > storage, and, IIRC, so does CVS. I don't think there is any precedence
 > for wanting something else - and frankly, I can't see how repository
 > storage would matter.

Well, internally you could use U+2028 LINE SEPARATOR, which would
screw up *everybody* if they don't use the converter, since there are
probably very few editors that understand U+2028.  I've heard that
this is what Samba did when converting to Unicode: intead of using
UTF-8 they used UTF-16 so that English would be at least as buggy as
any other language.

Maybe there's somebody who was participating in Samba at that time who
knows?

From stephen at xemacs.org  Sun Sep  6 09:56:08 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 06 Sep 2009 16:56:08 +0900
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <4AA2835C.6000002@ochtman.nl>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost>
	<87ws4dcu9d.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4AA2835C.6000002@ochtman.nl>
Message-ID: <871vmky09z.fsf@uwakimon.sk.tsukuba.ac.jp>

Dirkjan Ochtman writes:
 > On 05/09/2009 16:59, Stephen J. Turnbull wrote:
 > > git has a nice filter-branch command, which would allow you to
 > > automatically repair the problem (it works basically by checking out
 > > each changeset and rerecording it with the appropriate commands).  I
 > > know bzr is growing something similar, so presumably it is or will
 > > soon be available in hg.
 > 
 > That means you change hashes on the server side, without human feedback. 
 > Let's try not to subvert the immutability design that Mercurial tries to 
 > encourage.

No, I mean the server refuses to accept it and the submitter can fix
it easily (with mq or histedit as Martin G points out), then resubmit.

In any case Mercurial's notion of immutability is unsustainable in
practice, as the plethora of extensions which mutate history
testifies.

From stephen at xemacs.org  Sun Sep  6 10:26:05 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 06 Sep 2009 17:26:05 +0900
Subject: [Python-Dev] Mercurial migration: help needed
In-Reply-To: <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com>
References: <4A8A6256.1030009@v.loewis.de>
	<ea2499da0908180120qcc1c8d7ta0d586653e3840a5@mail.gmail.com>
	<4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org>
	<4AA1B118.3030404@gmail.com>
	<loom.20090905T134046-455@post.gmane.org>
	<4AA2656F.4040805@v.loewis.de> <h7tser$tib$1@ger.gmane.org>
	<4AA2910A.4070504@v.loewis.de>
	<79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com>
Message-ID: <87zl98wkbm.fsf@uwakimon.sk.tsukuba.ac.jp>

Paul Moore writes:

 > The result is that user workspaces *may* (quite probably, will)
 > contain files with a mixture of line endings if care is not taken.

Yes.  Under your "fixed-EOL-files-are-binary" scheme, this is
guaranteed for Unix systems.

 > As regards (1), I assume that for "text" files, a consistent EOL
 > convention (assumed LF) should be used in the repository.  It's not
 > clear to me what should be held in the repo for the files requiring
 > specific line endings - my instinct is that they should be treated
 > as "binary" and stored in the repo with the mandated line endings,
 > and checked out unchanged.

Why?  Files that require specific line endings are in general used in
platform-specific ways.  So checking them out with the platform's
normal line ending should work fine.

 > My view is that how I store text files in my workspace is entirely up
 > to me (and the capabilities of my tools). So, how files get checked
 > out should not be centrally mandated.

Your tools will be able to work with the native EOL convention, or you
wouldn't be able to stand using them.  In general the extension should
default to checking out with the native convention.  If you really
want to change that, you can; there's nothing the server can do to
mandate what's in your workspaces.  The "mandate" here is simply the
default extension that Python and/or Mercurial will distribute to help
developers avoid having their pushes aborted for incorrect EOLs.

 > So, the issues:
 > 
 > 1. Given that the "problematic" tools (notepad and Visual Studio) are
 > Windows tools, we seem to be back to the idea that this extension is
 > only needed by Windows developers. As I understood the consensus to be
 > that the extension should be for all users, I suspect I've missed
 > something.

What you've missed is that developers *of* the Windows port are not
necessarily developers *on* Windows.  If we treat these as text files
and check them out in the native convention everywhere, then it
doesn't matter if you edit them on Unix or Windows, when you check
them out and build on Windows it Just Works[tm].  I have never heard
of a Unix cross-IDE port of Visual Studio....

 > 2. Allowing text files to be checked out in whatever form the user
 > prefers seems complicated.

It's not a question of "allow".  AIUI, you won't be allowed to push a
commit with broken line endings to the public repo.  This is too much
of a burden to impose given the wayward behavior of existing tools, so
an extension will be distributed that does the checking (and any
needed conversion) for you.  If you don't like that extension, you can
change it; it shouldn't be too difficult.  Eg:

 > That works, but would irritate me as I work on Windows, but prefer
 > strongly to use LF line endings (yes, I realise that makes me an
 > oddball...) I'd put up with it if it was the consensus to do this,
 > of course.

You don't need to.  In that case I would guess that you are at very
low risk if you disable the checkout side of the extension.

 > 3. Is there a realistic possibility that a user could edit one of the
 > CRLF-requiring files with a tool that converts it to LF?

Yes.  It happened occasionally in XEmacs's CVS repository, and caused
great consternation among Windows testers.

 > If so, is there a need to trap that programmatically (as opposed to
 > simply accepting that this equates to the individual accidentally
 > breaking the build, no worse or better than checking in a C file
 > with a syntax error)?

It's worse.  Until the problem is fixed, the people who need a
different EOL convention are working with a file with breakage on
every line.  Furthermore, that breakage may be quite silent, eg, if
you use XEmacs to edit on Windows but Visual Studio to build.  With
Emacs it's easy enough to change -- if you recognize the breakage.


From frantzcj at gmail.com  Mon Sep  7 18:09:36 2009
From: frantzcj at gmail.com (Chris Frantz)
Date: Mon, 7 Sep 2009 11:09:36 -0500
Subject: [Python-Dev] Controlling the cipher list for SSL connections
Message-ID: <aeddc9570909070909v3c9bf3b0of7ebb4b62857aa00@mail.gmail.com>

Greetings,

I would like to be able to set the cipher list when creating an SSL
connection.? It appears that the current SSL module doesn't provide
this functionality.

The attached patch (against trunk) adds this ability to SSLSocket.

Thank you,
--Chris

PS: Please reply directly to me, as I'm not subscribed to this list.

Index: Python-2.7/Lib/ssl.py
===================================================================
--- Python-2.7/Lib/ssl.py??? (revision 74703)
+++ Python-2.7/Lib/ssl.py??? (working copy)
@@ -88,7 +88,7 @@
????????????????? server_side=False, cert_reqs=CERT_NONE,
????????????????? ssl_version=PROTOCOL_SSLv23, ca_certs=None,
????????????????? do_handshake_on_connect=True,
-???????????????? suppress_ragged_eofs=True):
+???????????????? suppress_ragged_eofs=True, cipher_list=None):
???????? socket.__init__(self, _sock=sock._sock)
???????? # the initializer for socket trashes the methods (tsk, tsk), so...
???????? self.send = lambda data, flags=0: SSLSocket.send(self, data, flags)
@@ -110,7 +110,8 @@
???????????? # yes, create the SSL object
???????????? self._sslobj = _ssl.sslwrap(self._sock, server_side,
???????????????????????????????????????? keyfile, certfile,
-??????????????????????????????????????? cert_reqs, ssl_version, ca_certs)
+??????????????????????????????????????? cert_reqs, ssl_version,
+??????????????????????????????????????? ca_certs, cipher_list)
???????????? if do_handshake_on_connect:
???????????????? timeout = self.gettimeout()
???????????????? try:
Index: Python-2.7/Modules/_ssl.c
===================================================================
--- Python-2.7/Modules/_ssl.c??? (revision 74703)
+++ Python-2.7/Modules/_ssl.c??? (working copy)
@@ -261,7 +261,8 @@
???? ?????? enum py_ssl_server_or_client socket_type,
???? ?????? enum py_ssl_cert_requirements certreq,
???? ?????? enum py_ssl_version proto_version,
-??? ?????? char *cacerts_file)
+??? ?????? char *cacerts_file,
+??? ?????? char *cipher_list)
?{
???? PySSLObject *self;
???? char *errstr = NULL;
@@ -366,6 +367,9 @@
???? SSL_CTX_set_verify(self->ctx, verification_mode,
???? ??? ??? ?? NULL); /* set verify lvl */

+??? if (cipher_list)
+??? ??? SSL_CTX_set_cipher_list(self->ctx, cipher_list);
+
???? PySSL_BEGIN_ALLOW_THREADS
???? self->ssl = SSL_new(self->ctx); /* New ssl struct */
???? PySSL_END_ALLOW_THREADS
@@ -407,14 +411,17 @@
???? char *key_file = NULL;
???? char *cert_file = NULL;
???? char *cacerts_file = NULL;
+??? char *cipher_list = NULL;

-??? if (!PyArg_ParseTuple(args, "O!i|zziiz:sslwrap",
+
+??? if (!PyArg_ParseTuple(args, "O!i|zziizz:sslwrap",
???? ??? ??? ????? PySocketModule.Sock_Type,
???? ??? ??? ????? &Sock,
???? ??? ??? ????? &server_side,
???? ??? ??? ????? &key_file, &cert_file,
???? ??? ??? ????? &verification_mode, &protocol,
-??? ??? ??? ????? &cacerts_file))
+??? ??? ??? ????? &cacerts_file,
+??? ??? ??? ????? &cipher_list))
???? ??? return NULL;

???? /*
@@ -427,12 +434,12 @@

???? return (PyObject *) newPySSLObject(Sock, key_file, cert_file,
???? ??? ??? ??? ??? ?? server_side, verification_mode,
-??? ??? ??? ??? ??? ?? protocol, cacerts_file);
+??? ??? ??? ??? ??? ?? protocol, cacerts_file, cipher_list);
?}

?PyDoc_STRVAR(ssl_doc,
?"sslwrap(socket, server_side, [keyfile, certfile, certs_mode, protocol,\n"
-"????????????????????????????? cacertsfile]) -> sslobject");
+"????????????????????????????? cacertsfile, cipherlist]) -> sslobject");

?/* SSL object methods */

From fuzzyman at voidspace.org.uk  Mon Sep  7 18:15:21 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 07 Sep 2009 17:15:21 +0100
Subject: [Python-Dev] Controlling the cipher list for SSL connections
In-Reply-To: <aeddc9570909070909v3c9bf3b0of7ebb4b62857aa00@mail.gmail.com>
References: <aeddc9570909070909v3c9bf3b0of7ebb4b62857aa00@mail.gmail.com>
Message-ID: <4AA53199.5090409@voidspace.org.uk>

Hello Chris,

Can you post your patch to the Python bug tracker please - 
http://bugs.python.org

Patches posted to this list tend to get lost...

Thanks

Michael

Chris Frantz wrote:
> Greetings,
>
> I would like to be able to set the cipher list when creating an SSL
> connection.  It appears that the current SSL module doesn't provide
> this functionality.
>
> The attached patch (against trunk) adds this ability to SSLSocket.
>
> Thank you,
> --Chris
>
> PS: Please reply directly to me, as I'm not subscribed to this list.
>
> Index: Python-2.7/Lib/ssl.py
> ===================================================================
> --- Python-2.7/Lib/ssl.py    (revision 74703)
> +++ Python-2.7/Lib/ssl.py    (working copy)
> @@ -88,7 +88,7 @@
>                   server_side=False, cert_reqs=CERT_NONE,
>                   ssl_version=PROTOCOL_SSLv23, ca_certs=None,
>                   do_handshake_on_connect=True,
> -                 suppress_ragged_eofs=True):
> +                 suppress_ragged_eofs=True, cipher_list=None):
>          socket.__init__(self, _sock=sock._sock)
>          # the initializer for socket trashes the methods (tsk, tsk), so...
>          self.send = lambda data, flags=0: SSLSocket.send(self, data, flags)
> @@ -110,7 +110,8 @@
>              # yes, create the SSL object
>              self._sslobj = _ssl.sslwrap(self._sock, server_side,
>                                          keyfile, certfile,
> -                                        cert_reqs, ssl_version, ca_certs)
> +                                        cert_reqs, ssl_version,
> +                                        ca_certs, cipher_list)
>              if do_handshake_on_connect:
>                  timeout = self.gettimeout()
>                  try:
> Index: Python-2.7/Modules/_ssl.c
> ===================================================================
> --- Python-2.7/Modules/_ssl.c    (revision 74703)
> +++ Python-2.7/Modules/_ssl.c    (working copy)
> @@ -261,7 +261,8 @@
>             enum py_ssl_server_or_client socket_type,
>             enum py_ssl_cert_requirements certreq,
>             enum py_ssl_version proto_version,
> -           char *cacerts_file)
> +           char *cacerts_file,
> +           char *cipher_list)
>  {
>      PySSLObject *self;
>      char *errstr = NULL;
> @@ -366,6 +367,9 @@
>      SSL_CTX_set_verify(self->ctx, verification_mode,
>                 NULL); /* set verify lvl */
>
> +    if (cipher_list)
> +        SSL_CTX_set_cipher_list(self->ctx, cipher_list);
> +
>      PySSL_BEGIN_ALLOW_THREADS
>      self->ssl = SSL_new(self->ctx); /* New ssl struct */
>      PySSL_END_ALLOW_THREADS
> @@ -407,14 +411,17 @@
>      char *key_file = NULL;
>      char *cert_file = NULL;
>      char *cacerts_file = NULL;
> +    char *cipher_list = NULL;
>
> -    if (!PyArg_ParseTuple(args, "O!i|zziiz:sslwrap",
> +
> +    if (!PyArg_ParseTuple(args, "O!i|zziizz:sslwrap",
>                    PySocketModule.Sock_Type,
>                    &Sock,
>                    &server_side,
>                    &key_file, &cert_file,
>                    &verification_mode, &protocol,
> -                  &cacerts_file))
> +                  &cacerts_file,
> +                  &cipher_list))
>          return NULL;
>
>      /*
> @@ -427,12 +434,12 @@
>
>      return (PyObject *) newPySSLObject(Sock, key_file, cert_file,
>                         server_side, verification_mode,
> -                       protocol, cacerts_file);
> +                       protocol, cacerts_file, cipher_list);
>  }
>
>  PyDoc_STRVAR(ssl_doc,
>  "sslwrap(socket, server_side, [keyfile, certfile, certs_mode, protocol,\n"
> -"                              cacertsfile]) -> sslobject");
> +"                              cacertsfile, cipherlist]) -> sslobject");
>
>  /* SSL object methods */
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From frantzcj at gmail.com  Mon Sep  7 18:32:31 2009
From: frantzcj at gmail.com (Chris Frantz)
Date: Mon, 7 Sep 2009 11:32:31 -0500
Subject: [Python-Dev] Controlling the cipher list for SSL connections
In-Reply-To: <4AA53199.5090409@voidspace.org.uk>
References: <aeddc9570909070909v3c9bf3b0of7ebb4b62857aa00@mail.gmail.com>
	<4AA53199.5090409@voidspace.org.uk>
Message-ID: <aeddc9570909070932t39933602g7605a10c234bb0cc@mail.gmail.com>

Done.

Attached to Issue 3597, which is a similar request to mine.

Best Regards,
--Chris

From eric.pruitt at gmail.com  Mon Sep  7 20:13:35 2009
From: eric.pruitt at gmail.com (Eric Pruitt)
Date: Mon, 7 Sep 2009 14:13:35 -0400
Subject: [Python-Dev] PEP 3145
Message-ID: <171e8a410909071113r7d06399t8029f4b795e5b0aa@mail.gmail.com>

Hello all,

I have been working on adding asynchronous I/O to the Python
subprocess module as part of my Google Summer of Code project. Now
that I have finished documenting and pruning the code, I present PEP
3145 for its inclusion into the Python core code. Any and all feedback
on the PEP (http://www.python.org/dev/peps/pep-3145/) is appreciated.

Eric

From eric at trueblade.com  Mon Sep  7 23:46:42 2009
From: eric at trueblade.com (Eric Smith)
Date: Mon, 07 Sep 2009 17:46:42 -0400
Subject: [Python-Dev] Numeric alignment issue with PEP 3101
Message-ID: <4AA57F42.3050201@trueblade.com>

The default string formatting alignment for all types, according to PEP 
3101, is left aligned. Issue 6857 (http://bugs.python.org/issue6857) 
points out that for numeric types (int, float, and decimal, at least), 
the actual implemented default alignment is right aligned.

Mark Dickinson and I agree that for numeric types, right alignment makes 
much more sense as a default. And that's what %-formatting and 
str.format() both do.

I think the PEP should be modified to say that right alignment is the 
default for numeric types. Also the documentation at 
http://docs.python.org/library/string.html#formatstrings should have the 
same modification.

Does anyone disagree?

Eric.

From greg.ewing at canterbury.ac.nz  Tue Sep  8 00:05:33 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 08 Sep 2009 10:05:33 +1200
Subject: [Python-Dev] Numeric alignment issue with PEP 3101
In-Reply-To: <4AA57F42.3050201@trueblade.com>
References: <4AA57F42.3050201@trueblade.com>
Message-ID: <4AA583AD.1000104@canterbury.ac.nz>

Eric Smith wrote:

> Mark Dickinson and I agree that for numeric types, right alignment makes 
> much more sense as a default. And that's what %-formatting and 
> str.format() both do.

Is the new formatting supposed to behave the same way
as %-formatting for the same format codes? Because the
default for %-formatting is actually right alignment
for *all* types, including strings.

-- 
Greg

From eric at trueblade.com  Tue Sep  8 00:10:04 2009
From: eric at trueblade.com (Eric Smith)
Date: Mon, 07 Sep 2009 18:10:04 -0400
Subject: [Python-Dev] Numeric alignment issue with PEP 3101
In-Reply-To: <4AA583AD.1000104@canterbury.ac.nz>
References: <4AA57F42.3050201@trueblade.com>
	<4AA583AD.1000104@canterbury.ac.nz>
Message-ID: <4AA584BC.2020001@trueblade.com>

Greg Ewing wrote:
> Is the new formatting supposed to behave the same way
> as %-formatting for the same format codes? Because the
> default for %-formatting is actually right alignment
> for *all* types, including strings.

Hmm, I never noticed that. At this point, I think changing the 
formatting for any types would break code, so we should just change the 
documentation to reflect how currently works.

Eric.


From python at rcn.com  Tue Sep  8 02:16:18 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 7 Sep 2009 17:16:18 -0700
Subject: [Python-Dev] Numeric alignment issue with PEP 3101
In-Reply-To: <4AA57F42.3050201@trueblade.com>
References: <4AA57F42.3050201@trueblade.com>
Message-ID: <6930F234-6E8E-4E78-AE7F-FFE97AF72D5C@rcn.com>

I concur.  Numbers are naturally right aligned.


On Sep 7, 2009, at 2:46 PM, Eric Smith wrote:

> The default string formatting alignment for all types, according to  
> PEP 3101, is left aligned. Issue 6857 (http://bugs.python.org/issue6857 
> ) points out that for numeric types (int, float, and decimal, at  
> least), the actual implemented default alignment is right aligned.
>
> Mark Dickinson and I agree that for numeric types, right alignment  
> makes much more sense as a default. And that's what %-formatting and  
> str.format() both do.
>
> I think the PEP should be modified to say that right alignment is  
> the default for numeric types. Also the documentation at http://docs.python.org/library/string.html#formatstrings 
>  should have the same modification.
>
> Does anyone disagree?
>
> Eric.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com


From dickinsm at gmail.com  Tue Sep  8 09:11:21 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 8 Sep 2009 08:11:21 +0100
Subject: [Python-Dev] Numeric alignment issue with PEP 3101
In-Reply-To: <4AA584BC.2020001@trueblade.com>
References: <4AA57F42.3050201@trueblade.com>
	<4AA583AD.1000104@canterbury.ac.nz> <4AA584BC.2020001@trueblade.com>
Message-ID: <5c6f2a5d0909080011gd558970r321832de90497c2b@mail.gmail.com>

On Mon, Sep 7, 2009 at 11:10 PM, Eric Smith<eric at trueblade.com> wrote:
> Hmm, I never noticed that. At this point, I think changing the formatting
> for any types would break code, so we should just change the documentation
> to reflect how currently works.

I think the alignment for Decimal *does* need to be changed, though.  It
currently left-aligns by default (my fault:  I just blindly followed PEP 3101
without thinking too hard about it).  I'd like to fix this for 3.2 and 2.7; I'm
not sure whether it's too disruptive to fix it in 3.1 and 2.6.

Mark

From eric at trueblade.com  Tue Sep  8 13:56:53 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 08 Sep 2009 07:56:53 -0400
Subject: [Python-Dev] Numeric alignment issue with PEP 3101
In-Reply-To: <5c6f2a5d0909080011gd558970r321832de90497c2b@mail.gmail.com>
References: <4AA57F42.3050201@trueblade.com>	<4AA583AD.1000104@canterbury.ac.nz>
	<4AA584BC.2020001@trueblade.com>
	<5c6f2a5d0909080011gd558970r321832de90497c2b@mail.gmail.com>
Message-ID: <4AA64685.1010803@trueblade.com>

Mark Dickinson wrote:
> On Mon, Sep 7, 2009 at 11:10 PM, Eric Smith<eric at trueblade.com> wrote:
>> Hmm, I never noticed that. At this point, I think changing the formatting
>> for any types would break code, so we should just change the documentation
>> to reflect how currently works.
> 
> I think the alignment for Decimal *does* need to be changed, though.  It
> currently left-aligns by default (my fault:  I just blindly followed PEP 3101
> without thinking too hard about it).  I'd like to fix this for 3.2 and 2.7; I'm
> not sure whether it's too disruptive to fix it in 3.1 and 2.6.

Right, not changing "any types" was too strong. I think that the numeric 
types should all default to right alignment. That means Decimal should 
change. I agree this should only be in 3.2 and 2.7. In addition, PEP 
3101 and the documentation should be change to reflect the default right 
alignment for numeric types.

Shame on you, Mark, for actually reading and implementing the specs! ;) 
You see that I didn't bother with that. At the time, I think I was 
trying to get str.format() to exactly agree with %-formatting.

From naveen.garg at gmail.com  Tue Sep  8 07:42:17 2009
From: naveen.garg at gmail.com (tinku99)
Date: Mon, 7 Sep 2009 22:42:17 -0700 (PDT)
Subject: [Python-Dev] compiling python2.5 on linux under wine
In-Reply-To: <ced5f0f60901090251x3206f668le54ea1d2e9cc97cf@mail.gmail.com>
References: <ced5f0f60901031322k283451c7i766ac2164489ad50@mail.gmail.com>
	<fb73205e0901080442i22046e46pb99a0539c23edda8@mail.gmail.com>
	<5b8d13220901080511x2fe96845l8ad86e70908f0a3e@mail.gmail.com>
	<ced5f0f60901080602k1bf11ae4rb93a21ebbd261fdf@mail.gmail.com>
	<49666AF7.9010804@v.loewis.de>
	<ced5f0f60901090251x3206f668le54ea1d2e9cc97cf@mail.gmail.com>
Message-ID: <25340242.post@talk.nabble.com>


aren't you better off just cross compiling under mingw? 
You don't even really need to do that...
you can just dynamically load the brebuilt windows python libraries in your
wine apps...
wine is for running windows apps on linux, not for cross compiling... 
wine itself uses the mingw cross compiler when necessary from my
understanding. 
winelib is for creating elf binaries, not windows binaries...
-- 
View this message in context: http://www.nabble.com/compiling-python2.5-on-linux-under-wine-tp21270106p25340242.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From chris at simplistix.co.uk  Tue Sep  8 14:18:56 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 08 Sep 2009 13:18:56 +0100
Subject: [Python-Dev] [Distutils] Static metadata using setup.cfg
In-Reply-To: <13d2dc789ffc897481edc43e88432a31@preisshare.net>
References: <94bdd2610908170034q544bca48g7619589bd3764@mail.gmail.com>
	<aa030340d3d05b66b14aa22eabade5b7@preisshare.net>
	<4A8A606D.6090306@taupro.com>
	<0b8f78d03f5dfdeba987ef233b2ae8eb@preisshare.net>
	<4A8A8FE9.6070905@trueblade.com>
	<94bdd2610908180932te51f49byc9f3938c29a0c100@mail.gmail.com>
	<94bdd2610908250248o46349e76uea66245f5ebd1173@mail.gmail.com>
	<4A9BBA2B.6010903@simplistix.co.uk>
	<94bdd2610908310637r17af992al469498e90544c5e0@mail.gmail.com>
	<4AA4F94C.9080702@simplistix.co.uk>
	<23CAC1AA-9399-405A-A7E6-64F4C052BEC1@mac.com>
	<4AA6136A.7080904@simplistix.co.uk>
	<13d2dc789ffc897481edc43e88432a31@preisshare.net>
Message-ID: <4AA64BB0.7090309@simplistix.co.uk>

David Lyon wrote:
> On Tue, 08 Sep 2009 09:18:50 +0100, Chris Withers <chris at simplistix.co.uk>
> wrote:
>> <mini rant>
>> If Python had a packaging system *and* used it for the standard library, 
>> then things like this wouldn't be a problem...
>> The setup.cfg could just say "requires sqlite greater than version 
>> x.y.z", and if it was in the standard library, it would be used unless a 
>> newer version was needed. 
> 
> +1
> 
> Actually, this was already discussed on this mailing list.

Yeah, I know, but I had memories of it be poo-poo'ed on Python-Dev...
(CC'ing so they can tell me I'm wrong ;-) )

> I suggested that a "requires" section could easily do this, something
> along the lines of:
> 
> [Requires]
> stdlib=sqlite>=1.5

Tarek, How are requirements spelled for packages in your current setup.cfg?

I really don't see why anything should be different for standard library 
packages (ie: the stdlib= prefix in David's example). Python 
distributions should just declare all the versions they come with in the 
same way that whatever-is-being-built by Tarek can introspect in the 
same way as any other package...

> So the concept of having an if/else test for this is superfluous.

Right.

>> It would also mean it would be possible to 
>> release bug fix versions of the standard library packages without having 
>> to roll a whole python release.
> 
> +1

...which in turn would mean that the standard library is no longer a 
place where packages go to die...

>> Better yet, since "python" should be a package as far as the packaging 
>> system is concerned, library versions can just say what versions of 
>> python they work with.
> 
> +1 - good idea

...and for me, the "python" package should be just another package in 
the distributions dance, called "python" ;-)

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From ziade.tarek at gmail.com  Tue Sep  8 14:29:12 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 8 Sep 2009 14:29:12 +0200
Subject: [Python-Dev] [Distutils] Static metadata using setup.cfg
In-Reply-To: <4AA64BB0.7090309@simplistix.co.uk>
References: <94bdd2610908170034q544bca48g7619589bd3764@mail.gmail.com>
	<94bdd2610908180932te51f49byc9f3938c29a0c100@mail.gmail.com>
	<94bdd2610908250248o46349e76uea66245f5ebd1173@mail.gmail.com>
	<4A9BBA2B.6010903@simplistix.co.uk>
	<94bdd2610908310637r17af992al469498e90544c5e0@mail.gmail.com>
	<4AA4F94C.9080702@simplistix.co.uk>
	<23CAC1AA-9399-405A-A7E6-64F4C052BEC1@mac.com>
	<4AA6136A.7080904@simplistix.co.uk>
	<13d2dc789ffc897481edc43e88432a31@preisshare.net>
	<4AA64BB0.7090309@simplistix.co.uk>
Message-ID: <94bdd2610909080529w4b3fc34cy43be7643a8e06391@mail.gmail.com>

2009/9/8 Chris Withers <chris at simplistix.co.uk>:
>> I suggested that a "requires" section could easily do this, something
>> along the lines of:
>>
>> [Requires]
>> stdlib=sqlite>=1.5
>
> Tarek, How are requirements spelled for packages in your current setup.cfg?

Sorry, that's another problem we are dealing with here e.g. How *one*
requirement is defined.
The .cfg file is just a format that holds values, like you would
express them in command line or in
arguments.

The "How are requirements spelled" part is the changes we will add in PEP 345.

The templating part is just here to make these requirements (or anything else)
vary, depending on a few environment values, so it can be read without
installing
the distribution, with a vanilla Python.

Please stop cross-posting in other mailing-lists it makes the
discussions confusing.

there are two discussions, to be talked in distutils-SIG:

1/ static metadata using a setup.cfg.in (the current one)
2/ changes that will occur in PEP 345, to include requirement definitions

Regards
Tarek

-- 
Tarek Ziad? | http://ziade.org |???????????

From ncoghlan at gmail.com  Tue Sep  8 15:00:30 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 08 Sep 2009 23:00:30 +1000
Subject: [Python-Dev] [Distutils] Static metadata using setup.cfg
In-Reply-To: <4AA64BB0.7090309@simplistix.co.uk>
References: <94bdd2610908170034q544bca48g7619589bd3764@mail.gmail.com>	<aa030340d3d05b66b14aa22eabade5b7@preisshare.net>	<4A8A606D.6090306@taupro.com>	<0b8f78d03f5dfdeba987ef233b2ae8eb@preisshare.net>	<4A8A8FE9.6070905@trueblade.com>	<94bdd2610908180932te51f49byc9f3938c29a0c100@mail.gmail.com>	<94bdd2610908250248o46349e76uea66245f5ebd1173@mail.gmail.com>	<4A9BBA2B.6010903@simplistix.co.uk>	<94bdd2610908310637r17af992al469498e90544c5e0@mail.gmail.com>	<4AA4F94C.9080702@simplistix.co.uk>	<23CAC1AA-9399-405A-A7E6-64F4C052BEC1@mac.com>	<4AA6136A.7080904@simplistix.co.uk>	<13d2dc789ffc897481edc43e88432a31@preisshare.net>
	<4AA64BB0.7090309@simplistix.co.uk>
Message-ID: <4AA6556E.8070902@gmail.com>

Chris Withers wrote:
> David Lyon wrote:
>> On Tue, 08 Sep 2009 09:18:50 +0100, Chris Withers
>> <chris at simplistix.co.uk>
>> wrote:
>>> <mini rant>
>>> If Python had a packaging system *and* used it for the standard
>>> library, then things like this wouldn't be a problem...
>>> The setup.cfg could just say "requires sqlite greater than version
>>> x.y.z", and if it was in the standard library, it would be used
>>> unless a newer version was needed. 
>>
>> +1
>>
>> Actually, this was already discussed on this mailing list.
> 
> Yeah, I know, but I had memories of it be poo-poo'ed on Python-Dev...
> (CC'ing so they can tell me I'm wrong ;-) )

My recollection of the python-dev discussions was that the outcome was
"there isn't even a distutils-sig consensus yet, so it's a little early
for python-dev to be sticking its oar in". Also, I don't think providing
new metadata *describing* the standard library was particularly
controversial - it was more radical ideas of actually breaking up the
core distro into separate components that met resistance. Third party
installers having the power to easily "replace" standard library modules
for more than a single application was also a heavily debated feature
(since that way lies unintended coupling and application/library
installations that break previously installed applications).

It's probably also worth mentioning that on many of these topics there
is a recurring communications gap that usually becomes painful for the
distutils-sig side as core devs re-ask questions that were hashed out on
the distutils list long before. So I would suggest being prepared to
summarise and provide links to such discussions when this all becomes
settled enough to bring back to the whole of python-dev. The back and
forth of an archived mailing list discussion can sometimes be more
persuasive than the relatively sanitised summaries that end up in a PEP :)

That said, Tarek's response suggests that this still isn't ready for a
wider python-dev discussion, so I've set followup-to accordingly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From eric at trueblade.com  Tue Sep  8 22:41:56 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 8 Sep 2009 16:41:56 -0400 (EDT)
Subject: [Python-Dev] Numeric alignment issue with PEP 3101
In-Reply-To: <4AA64685.1010803@trueblade.com>
References: <4AA57F42.3050201@trueblade.com>
	<4AA583AD.1000104@canterbury.ac.nz> <4AA584BC.2020001@trueblade.com>
	<5c6f2a5d0909080011gd558970r321832de90497c2b@mail.gmail.com>
	<4AA64685.1010803@trueblade.com>
Message-ID: <50109.63.251.87.214.1252442516.squirrel@mail.trueblade.com>

> Mark Dickinson wrote:
>> I think the alignment for Decimal *does* need to be changed, though.  It
>> currently left-aligns by default (my fault:  I just blindly followed PEP
>> 3101
>> without thinking too hard about it).  I'd like to fix this for 3.2 and
>> 2.7; I'm
>> not sure whether it's too disruptive to fix it in 3.1 and 2.6.

I notice that complex has the same problem as the just-fixed Decimal. I'll
make those changes.

From peter at hda3.com  Wed Sep  9 01:36:41 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 8 Sep 2009 16:36:41 -0700
Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the
	Python Standard Library
In-Reply-To: <8517e9350908270652w4846dfdal8278ce71e3d238b4@mail.gmail.com>
References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com>
	<8517e9350908270652w4846dfdal8278ce71e3d238b4@mail.gmail.com>
Message-ID: <8517e9350909081636n29ee5082x51fce2721b1e2791@mail.gmail.com>

Howdy,

So I've made most of the suggested changes here and put up a release
of ipaddr. Do folks here consider this pep is final (enough) for
submission or is there more work to be done?

Cheers,
/peter

On Thu, Aug 27, 2009 at 6:52 AM, Peter Moody<peter at hda3.com> wrote:
> Howdy folks,
>
> the reference code has been updated per your comments; specifically,
> there's no more IP/IPv4/IPv6 factory functions, it's all IPAddress()
> and IPNetwork constructors.
>
> I've submitted a patch to the PEP with updated examples and a lengthy
> description of the class inheritance and the benefits from that design
> choice. hopefully that should go live soon.
>
> If there are any more suggestions on the PEP or the code, please let me know.
>
> Cheers,
> /peter
>
> On Tue, Aug 18, 2009 at 1:00 PM, Peter Moody<peter at hda3.com> wrote:
>> Howdy folks,
>>
>> I have a first draft of a PEP for including an IP address manipulation
>> library in the python stdlib. It seems like there are a lot of really
>> smart folks with some, ahem, strong ideas about what an IP address
>> module should and shouldn't be so I wanted to solicit your input on
>> this pep.
>>
>> the pep can be found here:
>>
>> ?http://www.python.org/dev/peps/pep-3144/
>>
>> the code can be found here:
>>
>> ?http://ipaddr-py.googlecode.com/svn/branches/2.0.x/
>>
>> Please let me know if you have any comments (some already coming :)
>>
>> Cheers,
>> /peter
>>
>

From benjamin at python.org  Wed Sep  9 04:56:35 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 8 Sep 2009 21:56:35 -0500
Subject: [Python-Dev] PEP 3145
In-Reply-To: <171e8a410909071113r7d06399t8029f4b795e5b0aa@mail.gmail.com>
References: <171e8a410909071113r7d06399t8029f4b795e5b0aa@mail.gmail.com>
Message-ID: <1afaf6160909081956x262ed97dlc2ba1df99e7bc262@mail.gmail.com>

2009/9/7 Eric Pruitt <eric.pruitt at gmail.com>:
> Hello all,
>
> I have been working on adding asynchronous I/O to the Python
> subprocess module as part of my Google Summer of Code project. Now
> that I have finished documenting and pruning the code, I present PEP
> 3145 for its inclusion into the Python core code. Any and all feedback
> on the PEP (http://www.python.org/dev/peps/pep-3145/) is appreciated.

Hi Eric,
One of the reasons you're not getting many response is that you've not
pasted the contents of the PEP in this message. That makes it really
easy for people to comment on various sections.

BTW, it seems like you were trying to use reST formatting with the
text PEP layout. Double backquotes only mean something in reST.


-- 
Regards,
Benjamin

From eric.pruitt at gmail.com  Wed Sep  9 05:52:45 2009
From: eric.pruitt at gmail.com (Eric Pruitt)
Date: Tue, 8 Sep 2009 23:52:45 -0400
Subject: [Python-Dev] PEP 3145 (With Contents)
Message-ID: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com>

Alright, I will re-submit with the contents pasted. I never use double
backquotes as I think them rather ugly; that is the work of an editor
or some automated program in the chain. Plus, it also messed up my
line formatting and now I have lines with one word on them... Anyway,
the contents of PEP 3145:

PEP: 3145
Title: Asynchronous I/O For subprocess.Popen
Author: (James) Eric Pruitt, Charles R. McCreary, Josiah Carlson
Type: Standards Track
Content-Type: text/plain
Created: 04-Aug-2009
Python-Version: 3.2

Abstract:

    In its present form, the subprocess.Popen implementation is prone to
    dead-locking and blocking of the parent Python script while waiting on data
    from the child process.

Motivation:

    A search for "python asynchronous subprocess" will turn up numerous
    accounts of people wanting to execute a child process and communicate with
    it from time to time reading only the data that is available instead of
    blocking to wait for the program to produce data [1] [2] [3].  The current
    behavior of the subprocess module is that when a user sends or receives
    data via the stdin, stderr and stdout file objects, dead locks are common
    and documented [4] [5].  While communicate can be used to alleviate some of
    the buffering issues, it will still cause the parent process to block while
    attempting to read data when none is available to be read from the child
    process.

Rationale:

    There is a documented need for asynchronous, non-blocking functionality in
    subprocess.Popen [6] [7] [2] [3].  Inclusion of the code would improve the
    utility of the Python standard library that can be used on Unix based and
    Windows builds of Python.  Practically every I/O object in Python has a
    file-like wrapper of some sort.  Sockets already act as such and for
    strings there is StringIO.  Popen can be made to act like a file by simply
    using the methods attached the the subprocess.Popen.stderr, stdout and
    stdin file-like objects.  But when using the read and write methods of
    those options, you do not have the benefit of asynchronous I/O.  In the
    proposed solution the wrapper wraps the asynchronous methods to mimic a
    file object.

Reference Implementation:

    I have been maintaining a Google Code repository that contains all of my
    changes including tests and documentation [9] as well as blog detailing
    the problems I have come across in the development process [10].

    I have been working on implementing non-blocking asynchronous I/O in the
    subprocess.Popen module as well as a wrapper class for subprocess.Popen
    that makes it so that an executed process can take the place of a file by
    duplicating all of the methods and attributes that file objects have.

    There are two base functions that have been added to the subprocess.Popen
    class: Popen.send and Popen._recv, each with two separate implementations,
    one for Windows and one for Unix based systems.  The Windows
    implementation uses ctypes to access the functions needed to control pipes
    in the kernel 32 DLL in an asynchronous manner.  On Unix based systems,
    the Python interface for file control serves the same purpose.  The
    different implementations of Popen.send and Popen._recv have identical
    arguments to make code that uses these functions work across multiple
    platforms.

    When calling the Popen._recv function, it requires the pipe name be
    passed as an argument so there exists the Popen.recv function that passes
    selects stdout as the pipe for Popen._recv by default.  Popen.recv_err
    selects stderr as the pipe by default. "Popen.recv" and "Popen.recv_err"
    are much easier to read and understand than "Popen._recv('stdout' ..." and
    "Popen._recv('stderr' ..." respectively.

    Since the Popen._recv function does not wait on data to be produced
    before returning a value, it may return empty bytes.  Popen.asyncread
    handles this issue by returning all data read over a given time
    interval.

    The ProcessIOWrapper class uses the asyncread and asyncwrite functions to
    allow a process to act like a file so that there are no blocking issues
    that can arise from using the stdout and stdin file objects produced from
    a subprocess.Popen call.


References:

    [1] [ python-Feature Requests-1191964 ] asynchronous Subprocess
        http://mail.python.org/pipermail/python-bugs-list/2006-December/
          036524.html

    [2] Daily Life in an Ivory Basement : /feb-07/problems-with-subprocess
        http://ivory.idyll.org/blog/feb-07/problems-with-subprocess

    [3] How can I run an external command asynchronously from Python? - Stack
        Overflow
        http://stackoverflow.com/questions/636561/how-can-i-run-an-external-
          command-asynchronously-from-python

    [4] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation
        http://docs.python.org/library/subprocess.html#subprocess.Popen.wait

    [5] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation
        http://docs.python.org/library/subprocess.html#subprocess.Popen.kill

    [6] Issue 1191964: asynchronous Subprocess - Python tracker
        http://bugs.python.org/issue1191964

    [7] Module to allow Asynchronous subprocess use on Windows and Posix
        platforms - ActiveState Code
        http://code.activestate.com/recipes/440554/

    [8] subprocess.rst - subprocdev - Project Hosting on Google Code
        http://code.google.com/p/subprocdev/source/browse/doc/subprocess.rst?spec=svn2c925e935cad0166d5da85e37c742d8e7f609de5&r=2c925e935cad0166d5da85e37c742d8e7f609de5#437

    [9] subprocdev - Project Hosting on Google Code
        http://code.google.com/p/subprocdev

    [10] Python Subprocess Dev
         http://subdev.blogspot.com/

Copyright:

    This P.E.P. is licensed under the Open Publication License;
    http://www.opencontent.org/openpub/.



On Tue, Sep 8, 2009 at 22:56, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/9/7 Eric Pruitt <eric.pruitt at gmail.com>:
>> Hello all,
>>
>> I have been working on adding asynchronous I/O to the Python
>> subprocess module as part of my Google Summer of Code project. Now
>> that I have finished documenting and pruning the code, I present PEP
>> 3145 for its inclusion into the Python core code. Any and all feedback
>> on the PEP (http://www.python.org/dev/peps/pep-3145/) is appreciated.
>
> Hi Eric,
> One of the reasons you're not getting many response is that you've not
> pasted the contents of the PEP in this message. That makes it really
> easy for people to comment on various sections.
>
> BTW, it seems like you were trying to use reST formatting with the
> text PEP layout. Double backquotes only mean something in reST.
>
>
> --
> Regards,
> Benjamin
>

From barry at python.org  Wed Sep  9 15:01:35 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 9 Sep 2009 09:01:35 -0400
Subject: [Python-Dev] Python 2.6.3
Message-ID: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>

I had previously wanted to release Python 2.6.3 over the summer, but  
for various personal reasons, the summer was just too insane.  I'd  
like to reschedule a 2.6.3 release, shooting for final release on 25- 
September.

We should probably do a release candidate, so I'd like to make that on  
23-September.

Does anybody have objections to that schedule?  If not, I'll try to  
spend some time over the next few days looking at outstanding bugs,  
and marking release blockers, etc.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090909/9a17efc9/attachment.pgp>

From ncoghlan at gmail.com  Wed Sep  9 15:29:47 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 09 Sep 2009 23:29:47 +1000
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
Message-ID: <4AA7ADCB.6080101@gmail.com>

Barry Warsaw wrote:
> I had previously wanted to release Python 2.6.3 over the summer, but for
> various personal reasons, the summer was just too insane.  I'd like to
> reschedule a 2.6.3 release, shooting for final release on 25-September.
> 
> We should probably do a release candidate, so I'd like to make that on
> 23-September.
> 
> Does anybody have objections to that schedule?  If not, I'll try to
> spend some time over the next few days looking at outstanding bugs, and
> marking release blockers, etc.

2 days seems a little short (particularly allowing 24 hours or so for
the Windows and Mac installers to be produced). Haven't we historically
left a week between the RC and actual release for maintenance releases?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Wed Sep  9 15:52:22 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 9 Sep 2009 09:52:22 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <4AA7ADCB.6080101@gmail.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
Message-ID: <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>

On Sep 9, 2009, at 9:29 AM, Nick Coghlan wrote:

> Barry Warsaw wrote:
>> I had previously wanted to release Python 2.6.3 over the summer,  
>> but for
>> various personal reasons, the summer was just too insane.  I'd like  
>> to
>> reschedule a 2.6.3 release, shooting for final release on 25- 
>> September.
>>
>> We should probably do a release candidate, so I'd like to make that  
>> on
>> 23-September.
>>
>> Does anybody have objections to that schedule?  If not, I'll try to
>> spend some time over the next few days looking at outstanding bugs,  
>> and
>> marking release blockers, etc.
>
> 2 days seems a little short (particularly allowing 24 hours or so for
> the Windows and Mac installers to be produced). Haven't we  
> historically
> left a week between the RC and actual release for maintenance  
> releases?

Actually, I've rarely done rc's for point releases.  JFDI :)

I still want to release by the 25th, but I'd be willing to move the rc  
to Monday the 21st.  We're really just trying to avoid a brown bag  
moment, so that should give us enough time to double check the releases.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090909/f7aad7c6/attachment.pgp>

From brett at python.org  Wed Sep  9 18:14:19 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 9 Sep 2009 09:14:19 -0700
Subject: [Python-Dev] PEP 3145
In-Reply-To: <1afaf6160909081956x262ed97dlc2ba1df99e7bc262@mail.gmail.com>
References: <171e8a410909071113r7d06399t8029f4b795e5b0aa@mail.gmail.com> 
	<1afaf6160909081956x262ed97dlc2ba1df99e7bc262@mail.gmail.com>
Message-ID: <bbaeab100909090914n50d7511fwc3d0d8b8b811e554@mail.gmail.com>

On Tue, Sep 8, 2009 at 19:56, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/9/7 Eric Pruitt <eric.pruitt at gmail.com>:
>> Hello all,
>>
>> I have been working on adding asynchronous I/O to the Python
>> subprocess module as part of my Google Summer of Code project. Now
>> that I have finished documenting and pruning the code, I present PEP
>> 3145 for its inclusion into the Python core code. Any and all feedback
>> on the PEP (http://www.python.org/dev/peps/pep-3145/) is appreciated.
>
> Hi Eric,
> One of the reasons you're not getting many response is that you've not
> pasted the contents of the PEP in this message. That makes it really
> easy for people to comment on various sections.
>
> BTW, it seems like you were trying to use reST formatting with the
> text PEP layout. Double backquotes only mean something in reST.

That reST markup is my fault, not Eric. So used to reST PEPs that I
just instinctively added it when I check the PEP in.

-Brett

From thomas at python.org  Wed Sep  9 18:07:01 2009
From: thomas at python.org (Thomas Wouters)
Date: Wed, 9 Sep 2009 18:07:01 +0200
Subject: [Python-Dev] pthreads, fork, import, and execvp
In-Reply-To: <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com>
References: <d2115f4f0605090639h72570784k1f4ef0802b5a8d51@mail.gmail.com>
	<ee2a432c0605172343h30b1f469ydb5dff9435ca371f@mail.gmail.com>
	<446C2211.5060408@v.loewis.de> <446C6FC9.5060205@gmail.com>
	<446CB69E.4090609@v.loewis.de>
	<9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com>
	<3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com>
	<9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com>
	<9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com>
	<52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com>
Message-ID: <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com>

On Sat, Jul 25, 2009 at 19:25, Gregory P. Smith <greg at krypto.org> wrote:

> On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters<thomas at python.org> wrote:
> >
> > So attached (and at http://codereview.appspot.com/96125/show ) is a
> > preliminary fix, correcting the problem with os.fork(), os.forkpty() and
> > os.fork1(). This doesn't expose a general API for C code to use, for two
> > reasons: it's not easy, and I need this fix more than I need the API
> change
> > :-) (I actually need this fix myself for Python 2.4, but it applies
> fairly
> > cleanly.)
>
> This looks good to me.
>

Anyone else want to take a look at this before I check it in? I updated the
patch (in Rietveld) to contain some documentation about the hazards of
mixing fork and threads, which is the best we can do at the moment, at least
without seriously overhauling the threading APIs (which, granted, is not
that bad an idea, considering the mess they're in.) I've now thoroughly
tested the patch, and for most platforms it's strictly better. On AIX it
*may* behave differently (possibly 'incorrectly' for specific cases) if
something other than os.fork() calls the C fork() and calls
PyOS_AfterFork(), since on AIX it used to nuke the thread lock. *I* think
the new behaviour (not nuking the lock) is the correct thing to do, but
since most places that release the import lock don't bother to check if the
lock was even held, the old behaviour may have been succesfully masking the
bug on AIX systems.

Perhaps for the backport to 2.6 (which I think is in order, and also in
accordance with the guidelines) I should leave the AIX workaround in? Anyone
think it should not be removed from 3.x/2.7?


> Your idea of making this an API called a 'fork lock' or something
> sounds good (though I think it needs a better name.  PyBeforeFork &
> PyAfterFork?).  The subprocess module, for example, disables garbage
> collection before forking and restores it afterwards to avoid
> http://bugs.python.org/issue1336.  That type of thing could also be
> done in such a function.
>

Unfortunately it's rather hard to make those functions work correctly with
the current API -- we can't provide functions you can just use as arguments
to pthread_atfork because the global interpreter lock is not re-entrant and
we have no way of testing whether the current thread holds the GIL. I also
get the creepy-crawlies when I look at the various thread_* implementations
and see the horribly unsafe things they do (and also, for instance, the
PendingCall stuff in ceval.c :S) Unfortunately there's no good way to fix
these things without breaking API compatibility, let alone ABI
compatibility.


> Related to the above subprocess fork + gc bug.. It'd be nice for
> CPython to have code that does the fork+misc twiddling+exec all in one
> C call without needing to execute Python code in the child process
> prior to the exec().  Most of the inner body of subprocess's
> _execute_child() method could be done that way.  (with the obvious
> exception of the preexec_fn)
>
> >
> > To fix the mutex-across-fork problem correctly we should really acquire
> > three locks (the import lock, the GIL and the TLS lock, in that order.)
> The
> > import lock is re-entrant, and the TLS lock is tightly confined to the
> > thread-local-storage lookup functions, but the GIL is neither re-entrant
> nor
> > inspectable. That means we can't use pthread_atfork (we can't tell
> whether
> > we have the GIL already or not, right before the fork), nor can we
> provide a
> > convenient API for extension modules to use, really. The best we can do
> is
> > provide three functions, pthread_atfork-style: a 'prepare' function, an
> > 'after-fork-in-child' function, and an 'after-fork-in-parent' function.
> The
> > 'prepare' function would start by releasing the GIL, then acquire the
> import
> > lock, the GIL and the TLS lock in that order. It would also record
> > *somewhere* the thread_ident of the current thread. The 'in-parent'
> function
> > would simply release the TLS lock and the import lock, and the 'in-child'
> > would release those locks, call the threading._at_fork() function, and
> fix
> > up the TLS data, using the recorded thread ident to do lookups. The
> > 'in-child' function would replace the current PyOS_AfterFork() function
> > (which blindly reinitializes the GIL and the TLS lock, and calls
> > threading._at_fork().)
> >
> > Alternatively we could do what we do now, which is to ignore the fact
> that
> > thread idents may change by fork() and thus that thread-local data may
> > disappear, in which case the 'in-child' function could do a little less
> > work. I'm suitably scared and disgusted of the TLS implementation, the
> > threading implementations we support (including the pthread one) and the
> way
> > we blindly type-pun a pthread_t to a long, that I'm not sure I want to
> try
> > and build something correct and reliable on top of it.
> >
> > --
> > Thomas Wouters <thomas at python.org>
> >
> > Hi! I'm a .signature virus! copy me into your .signature file to help me
> > spread!
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
> >
> >
>



-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090909/20a224e5/attachment-0001.htm>

From nad at acm.org  Wed Sep  9 19:29:04 2009
From: nad at acm.org (Ned Deily)
Date: Wed, 09 Sep 2009 10:29:04 -0700
Subject: [Python-Dev] Python 2.6.3
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
Message-ID: <nad-2A73EB.10290309092009@ger.gmane.org>

In article <11A6545D-7204-4F61-B55B-1CC77CB5645E at python.org>,
 Barry Warsaw <barry at python.org> wrote:
> I still want to release by the 25th, but I'd be willing to move the rc  
> to Monday the 21st.  We're really just trying to avoid a brown bag  
> moment, so that should give us enough time to double check the releases.

The recent release of OS X 10.6 (Snow Leopard) has triggered a fair 
amount of 2.6 bug tracker activity, since 10.6 now includes 2.6 (2.6.1) 
and a 64-bit version at that.   A number of patches have either just 
been checked-in over the past couple of weeks or are getting some 
exposure before check-in.  Given the timing and the (appropriate) 
infrequency of 2.6.x releases, I think it would be unfortunate to push 
2.6.3 out the door without ensuring that it works well on 10.6.  
Therefore, I propose that 2.6.3 should have 10.6 compatibility as a 
"release goal".

Without trying to put Ronald on the spot (too much!), it would be a good 
idea to get his assessment where things stand wrt 2.6 on 10.6 before 
setting a final release date.

-- 
 Ned Deily,
 nad at acm.org


From greg at krypto.org  Wed Sep  9 20:19:42 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Wed, 9 Sep 2009 11:19:42 -0700
Subject: [Python-Dev] pthreads, fork, import, and execvp
In-Reply-To: <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com>
References: <d2115f4f0605090639h72570784k1f4ef0802b5a8d51@mail.gmail.com> 
	<446C2211.5060408@v.loewis.de> <446C6FC9.5060205@gmail.com> 
	<446CB69E.4090609@v.loewis.de>
	<9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com> 
	<3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com> 
	<9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com> 
	<9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com> 
	<52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> 
	<9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com>
Message-ID: <52dc1c820909091119x57d7aee6vda213659a2402dea@mail.gmail.com>

On Wed, Sep 9, 2009 at 9:07 AM, Thomas Wouters<thomas at python.org> wrote:
>
>
> On Sat, Jul 25, 2009 at 19:25, Gregory P. Smith <greg at krypto.org> wrote:
>>
>> On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters<thomas at python.org> wrote:
>> >
>> > So attached (and at http://codereview.appspot.com/96125/show ) is a
>> > preliminary fix, correcting the problem with os.fork(), os.forkpty() and
>> > os.fork1(). This doesn't expose a general API for C code to use, for two
>> > reasons: it's not easy, and I need this fix more than I need the API
>> > change
>> > :-) (I actually need this fix myself for Python 2.4, but it applies
>> > fairly
>> > cleanly.)
>>
>> This looks good to me.
>
> Anyone else want to take a look at this before I check it in? I updated the
> patch (in Rietveld) to contain some documentation about the hazards of
> mixing fork and threads, which is the best we can do at the moment, at least
> without seriously overhauling the threading APIs (which, granted, is not
> that bad an idea, considering the mess they're in.) I've now thoroughly
> tested the patch, and for most platforms it's strictly better. On AIX it
> *may* behave differently (possibly 'incorrectly' for specific cases) if
> something other than os.fork() calls the C fork() and calls
> PyOS_AfterFork(), since on AIX it used to nuke the thread lock. *I* think
> the new behaviour (not nuking the lock) is the correct thing to do, but
> since most places that release the import lock don't bother to check if the
> lock was even held, the old behaviour may have been succesfully masking the
> bug on AIX systems.
> Perhaps for the backport to 2.6 (which I think is in order, and also in
> accordance with the guidelines) I should leave the AIX workaround in? Anyone
> think it should not be removed from 3.x/2.7?
>
>>
>> Your idea of making this an API called a 'fork lock' or something
>> sounds good (though I think it needs a better name. ?PyBeforeFork &
>> PyAfterFork?). ?The subprocess module, for example, disables garbage
>> collection before forking and restores it afterwards to avoid
>> http://bugs.python.org/issue1336. ?That type of thing could also be
>> done in such a function.
>
> Unfortunately it's rather hard to make those functions work correctly with
> the current API -- we can't provide functions you can just use as arguments
> to pthread_atfork because the global interpreter lock is not re-entrant and
> we have no way of testing whether the current thread holds the GIL. I also
> get the creepy-crawlies when I look at the various thread_* implementations
> and see the horribly unsafe things they do (and also, for instance, the
> PendingCall stuff in ceval.c :S) Unfortunately there's no good way to fix
> these things without breaking API compatibility, let alone ABI
> compatibility.

Take a look at http://code.google.com/p/python-atfork/ which I created
to address the general fork+threading with locks held causing
deadlocks issue with many standard library modules.  Currently it only
patches the logging module but I intend to support others.  I want to
get an atfork mechanism into 2.7/3.2 along with every lock in the
standard library making proper use of it.

See also http://bugs.python.org/issue6721

I make no attempt to deal with C-level locks, only those acquired from
python.  It doesn't use pthread_atfork but instead models its behavior
after that and wraps os.fork and os.forkpty so that they call the
registered atfork methods as appropriate.

>
>>
>> Related to the above subprocess fork + gc bug.. It'd be nice for
>> CPython to have code that does the fork+misc twiddling+exec all in one
>> C call without needing to execute Python code in the child process
>> prior to the exec(). ?Most of the inner body of subprocess's
>> _execute_child() method could be done that way. ?(with the obvious
>> exception of the preexec_fn)
>>
>> >
>> > To fix the mutex-across-fork problem correctly we should really acquire
>> > three locks (the import lock, the GIL and the TLS lock, in that order.)
>> > The
>> > import lock is re-entrant, and the TLS lock is tightly confined to the
>> > thread-local-storage lookup functions, but the GIL is neither re-entrant
>> > nor
>> > inspectable. That means we can't use pthread_atfork (we can't tell
>> > whether
>> > we have the GIL already or not, right before the fork), nor can we
>> > provide a
>> > convenient API for extension modules to use, really. The best we can do
>> > is
>> > provide three functions, pthread_atfork-style: a 'prepare' function, an
>> > 'after-fork-in-child' function, and an 'after-fork-in-parent' function.
>> > The
>> > 'prepare' function would start by releasing the GIL, then acquire the
>> > import
>> > lock, the GIL and the TLS lock in that order. It would also record
>> > *somewhere* the thread_ident of the current thread. The 'in-parent'
>> > function
>> > would simply release the TLS lock and the import lock, and the
>> > 'in-child'
>> > would release those locks, call the threading._at_fork() function, and
>> > fix
>> > up the TLS data, using the recorded thread ident to do lookups. The
>> > 'in-child' function would replace the current PyOS_AfterFork() function
>> > (which blindly reinitializes the GIL and the TLS lock, and calls
>> > threading._at_fork().)
>> >
>> > Alternatively we could do what we do now, which is to ignore the fact
>> > that
>> > thread idents may change by fork() and thus that thread-local data may
>> > disappear, in which case the 'in-child' function could do a little less
>> > work. I'm suitably scared and disgusted of the TLS implementation, the
>> > threading implementations we support (including the pthread one) and the
>> > way
>> > we blindly type-pun a pthread_t to a long, that I'm not sure I want to
>> > try
>> > and build something correct and reliable on top of it.
>> >
>> > --
>> > Thomas Wouters <thomas at python.org>
>> >
>> > Hi! I'm a .signature virus! copy me into your .signature file to help me
>> > spread!
>> >
>> > _______________________________________________
>> > Python-Dev mailing list
>> > Python-Dev at python.org
>> > http://mail.python.org/mailman/listinfo/python-dev
>> > Unsubscribe:
>> > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>> >
>> >
>
>
>
> --
> Thomas Wouters <thomas at python.org>
>
> Hi! I'm a .signature virus! copy me into your .signature file to help me
> spread!
>

From greg at krypto.org  Wed Sep  9 20:38:53 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Wed, 9 Sep 2009 11:38:53 -0700
Subject: [Python-Dev] pthreads, fork, import, and execvp
In-Reply-To: <9e804ac0909091136x57272ab2h67e4bdaf458c0b0a@mail.gmail.com>
References: <d2115f4f0605090639h72570784k1f4ef0802b5a8d51@mail.gmail.com> 
	<446CB69E.4090609@v.loewis.de>
	<9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com> 
	<3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com> 
	<9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com> 
	<9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com> 
	<52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> 
	<9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com> 
	<52dc1c820909091119x57d7aee6vda213659a2402dea@mail.gmail.com> 
	<9e804ac0909091136x57272ab2h67e4bdaf458c0b0a@mail.gmail.com>
Message-ID: <52dc1c820909091138q42d66f42s8712ac2c4eebaa9d@mail.gmail.com>

On Wed, Sep 9, 2009 at 11:36 AM, Thomas Wouters<thomas at python.org> wrote:
>
> On Wed, Sep 9, 2009 at 20:19, Gregory P. Smith <greg at krypto.org> wrote:
>>
>> Take a look at http://code.google.com/p/python-atfork/ which I created
>> to address the general fork+threading with locks held causing
>> deadlocks issue with many standard library modules. ?Currently it only
>> patches the logging module but I intend to support others. ?I want to
>> get an atfork mechanism into 2.7/3.2 along with every lock in the
>> standard library making proper use of it.
>>
>> See also http://bugs.python.org/issue6721
>>
>> I make no attempt to deal with C-level locks, only those acquired from
>> python. ?It doesn't use pthread_atfork but instead models its behavior
>> after that and wraps os.fork and os.forkpty so that they call the
>> registered atfork methods as appropriate.
>
> Well, sure, the *Python code* side of the problem is fixable, thanks to
> Python being so awesome ;-P I'm strictly thinking of applications embedding
> Python (or even extending it and calling into code that doesn't consider
> Python.) Your python-atfork project looks like a terribly good idea, but it
> won't fix the embedding/extending issues (nor is it intended to, right?)

yeah I decided to only fix what could obviously be fixed and was
causing me pain at the moment. ;)

From thomas at python.org  Wed Sep  9 20:36:02 2009
From: thomas at python.org (Thomas Wouters)
Date: Wed, 9 Sep 2009 20:36:02 +0200
Subject: [Python-Dev] pthreads, fork, import, and execvp
In-Reply-To: <52dc1c820909091119x57d7aee6vda213659a2402dea@mail.gmail.com>
References: <d2115f4f0605090639h72570784k1f4ef0802b5a8d51@mail.gmail.com>
	<446C6FC9.5060205@gmail.com> <446CB69E.4090609@v.loewis.de>
	<9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com>
	<3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com>
	<9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com>
	<9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com>
	<52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com>
	<9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com>
	<52dc1c820909091119x57d7aee6vda213659a2402dea@mail.gmail.com>
Message-ID: <9e804ac0909091136x57272ab2h67e4bdaf458c0b0a@mail.gmail.com>

On Wed, Sep 9, 2009 at 20:19, Gregory P. Smith <greg at krypto.org> wrote:

> On Wed, Sep 9, 2009 at 9:07 AM, Thomas Wouters<thomas at python.org> wrote:
> >
> >
> > On Sat, Jul 25, 2009 at 19:25, Gregory P. Smith <greg at krypto.org> wrote:
> >>
> >> On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters<thomas at python.org>
> wrote:
> >> >
> >> > So attached (and at http://codereview.appspot.com/96125/show ) is a
> >> > preliminary fix, correcting the problem with os.fork(), os.forkpty()
> and
> >> > os.fork1(). This doesn't expose a general API for C code to use, for
> two
> >> > reasons: it's not easy, and I need this fix more than I need the API
> >> > change
> >> > :-) (I actually need this fix myself for Python 2.4, but it applies
> >> > fairly
> >> > cleanly.)
> >>
> >> This looks good to me.
> >
> > Anyone else want to take a look at this before I check it in? I updated
> the
> > patch (in Rietveld) to contain some documentation about the hazards of
> > mixing fork and threads, which is the best we can do at the moment, at
> least
> > without seriously overhauling the threading APIs (which, granted, is not
> > that bad an idea, considering the mess they're in.) I've now thoroughly
> > tested the patch, and for most platforms it's strictly better. On AIX it
> > *may* behave differently (possibly 'incorrectly' for specific cases) if
> > something other than os.fork() calls the C fork() and calls
> > PyOS_AfterFork(), since on AIX it used to nuke the thread lock. *I* think
> > the new behaviour (not nuking the lock) is the correct thing to do, but
> > since most places that release the import lock don't bother to check if
> the
> > lock was even held, the old behaviour may have been succesfully masking
> the
> > bug on AIX systems.
> > Perhaps for the backport to 2.6 (which I think is in order, and also in
> > accordance with the guidelines) I should leave the AIX workaround in?
> Anyone
> > think it should not be removed from 3.x/2.7?
> >
> >>
> >> Your idea of making this an API called a 'fork lock' or something
> >> sounds good (though I think it needs a better name.  PyBeforeFork &
> >> PyAfterFork?).  The subprocess module, for example, disables garbage
> >> collection before forking and restores it afterwards to avoid
> >> http://bugs.python.org/issue1336.  That type of thing could also be
> >> done in such a function.
> >
> > Unfortunately it's rather hard to make those functions work correctly
> with
> > the current API -- we can't provide functions you can just use as
> arguments
> > to pthread_atfork because the global interpreter lock is not re-entrant
> and
> > we have no way of testing whether the current thread holds the GIL. I
> also
> > get the creepy-crawlies when I look at the various thread_*
> implementations
> > and see the horribly unsafe things they do (and also, for instance, the
> > PendingCall stuff in ceval.c :S) Unfortunately there's no good way to fix
> > these things without breaking API compatibility, let alone ABI
> > compatibility.
>
> Take a look at http://code.google.com/p/python-atfork/ which I created
> to address the general fork+threading with locks held causing
> deadlocks issue with many standard library modules.  Currently it only
> patches the logging module but I intend to support others.  I want to
> get an atfork mechanism into 2.7/3.2 along with every lock in the
> standard library making proper use of it.
>
> See also http://bugs.python.org/issue6721
>
> I make no attempt to deal with C-level locks, only those acquired from
> python.  It doesn't use pthread_atfork but instead models its behavior
> after that and wraps os.fork and os.forkpty so that they call the
> registered atfork methods as appropriate.


Well, sure, the *Python code* side of the problem is fixable, thanks to
Python being so awesome ;-P I'm strictly thinking of applications embedding
Python (or even extending it and calling into code that doesn't consider
Python.) Your python-atfork project looks like a terribly good idea, but it
won't fix the embedding/extending issues (nor is it intended to, right?)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090909/218a0ac2/attachment-0001.htm>

From barry at python.org  Wed Sep  9 21:06:28 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 9 Sep 2009 15:06:28 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <nad-2A73EB.10290309092009@ger.gmane.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
Message-ID: <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>

On Sep 9, 2009, at 1:29 PM, Ned Deily wrote:

> In article <11A6545D-7204-4F61-B55B-1CC77CB5645E at python.org>,
> Barry Warsaw <barry at python.org> wrote:
>> I still want to release by the 25th, but I'd be willing to move the  
>> rc
>> to Monday the 21st.  We're really just trying to avoid a brown bag
>> moment, so that should give us enough time to double check the  
>> releases.
>
> The recent release of OS X 10.6 (Snow Leopard) has triggered a fair
> amount of 2.6 bug tracker activity, since 10.6 now includes 2.6  
> (2.6.1)
> and a 64-bit version at that.   A number of patches have either just
> been checked-in over the past couple of weeks or are getting some
> exposure before check-in.  Given the timing and the (appropriate)
> infrequency of 2.6.x releases, I think it would be unfortunate to push
> 2.6.3 out the door without ensuring that it works well on 10.6.
> Therefore, I propose that 2.6.3 should have 10.6 compatibility as a
> "release goal".
>
> Without trying to put Ronald on the spot (too much!), it would be a  
> good
> idea to get his assessment where things stand wrt 2.6 on 10.6 before
> setting a final release date.

I'm hoping that Python won't have any issues building and running on  
10.6, but I don't have it yet so I can't personally test it out.

How would you quantify "works well"?  Do you have any thoughts on  
tests you'd run other than the standard test suite?  If 2.6.3 is shown  
to pass its test suite on 10.5.x, is that good enough?  Are the  
specific bug fixes necessary for 10.6?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090909/2efed518/attachment.pgp>

From ronaldoussoren at mac.com  Wed Sep  9 20:19:37 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 09 Sep 2009 20:19:37 +0200
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <nad-2A73EB.10290309092009@ger.gmane.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
Message-ID: <C11F8211-8938-4C68-8221-C3C73E8E517B@mac.com>


On 9 Sep, 2009, at 19:29, Ned Deily wrote:
>
>
> Without trying to put Ronald on the spot (too much!), it would be a  
> good
> idea to get his assessment where things stand wrt 2.6 on 10.6 before
> setting a final release date.

MacOS X 10.6 support should be stable now, except for a critical issue
with IDLE: opening a new window hangs IDLE (issue 6864).

That said, I haven't scanned the issue tracker for more 10.6 related  
issues.

BTW. I'm fine with a sept. 25th release, that should give us enough time
to shake out issues with OSX 10.6 support.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090909/470c0564/attachment.bin>

From ncoghlan at gmail.com  Wed Sep  9 23:56:44 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 10 Sep 2009 07:56:44 +1000
Subject: [Python-Dev] pthreads, fork, import, and execvp
In-Reply-To: <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com>
References: <d2115f4f0605090639h72570784k1f4ef0802b5a8d51@mail.gmail.com>	
	<ee2a432c0605172343h30b1f469ydb5dff9435ca371f@mail.gmail.com>	
	<446C2211.5060408@v.loewis.de> <446C6FC9.5060205@gmail.com>	
	<446CB69E.4090609@v.loewis.de>	
	<9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com>	
	<3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com>	
	<9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com>	
	<9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com>	
	<52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com>
	<9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com>
Message-ID: <4AA8249C.9010707@gmail.com>

Thomas Wouters wrote:
>     Your idea of making this an API called a 'fork lock' or something
>     sounds good (though I think it needs a better name.  PyBeforeFork &
>     PyAfterFork?).  The subprocess module, for example, disables garbage
>     collection before forking and restores it afterwards to avoid
>     http://bugs.python.org/issue1336.  That type of thing could also be
>     done in such a function.
> 
> 
> Unfortunately it's rather hard to make those functions work correctly
> with the current API -- we can't provide functions you can just use as
> arguments to pthread_atfork because the global interpreter lock is not
> re-entrant and we have no way of testing whether the current thread
> holds the GIL.

I thought "make sure I have the GIL, either by already having it or
waiting for it if I don't yet have it" was the entire point of the
PyGILState_Ensure() API? [1]

Cheers,
Nick.

[1]
http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock





-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From zvezdan at zope.com  Thu Sep 10 14:47:40 2009
From: zvezdan at zope.com (Zvezdan Petkovic)
Date: Thu, 10 Sep 2009 08:47:40 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <C11F8211-8938-4C68-8221-C3C73E8E517B@mac.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<C11F8211-8938-4C68-8221-C3C73E8E517B@mac.com>
Message-ID: <D47379B3-3BE4-49C4-8C84-224273DE4242@zope.com>


On Sep 9, 2009, at 2:19 PM, Ronald Oussoren wrote:
> MacOS X 10.6 support should be stable now, except for a critical issue
> with IDLE: opening a new window hangs IDLE (issue 6864).
>
> That said, I haven't scanned the issue tracker for more 10.6 related  
> issues.

I just opened issue 6877 and provided a patch that I use for a long  
time already.  It would be nice to have it in the release though since  
other people could benefit from it too.

It resolves a BusError crash when Python readline module is built with  
a native Mac OS X editline (emulates readline).  The patch fixes the  
issue and enables use of readline module on Mac OS X.

Best regards,

	Zvezdan


From aahz at pythoncraft.com  Thu Sep 10 17:11:55 2009
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 10 Sep 2009 08:11:55 -0700
Subject: [Python-Dev] FWD: Front Runner Program
Message-ID: <20090910151155.GA13116@panix.com>

I'm still no-mail on python-dev, forwarding as FYI

----- Forwarded message from Eric Albrecht <eric.albrecht at Nichecubed.com> -----

From: Eric Albrecht <eric.albrecht at Nichecubed.com>
To: "webmaster at python.org" <webmaster at python.org>
Date: Thu, 10 Sep 2009 10:48:11 -0400
Subject: Front Runner Program 

Regarding: Windows 7 Compatibility for Python Application.

I am trying to contact your company regarding the Microsoft Windows 7 Compatibility Program for the application above. I have not been able to get in touch with the person responsible for this application in your company and this is why I am reaching out to you through the Support Team. This application has been identified as one of the applications Microsoft would like to see supported on Windows 7 and I have been tasked by Microsoft to help answer your questions about Windows 7 application compatibility and help you get your application through the Windows 7 "Green Light" compatibility process.

If your application already supports Windows Vista, chances are it will already be compatible with Windows 7 without the need for any code changes.  By pledging support for Windows 7 you're application will automatically be listed in the Windows Application Compatibility seen currently by more than 1 million users per month. The registration is extremely simple and just asks a few key questions.

Here is the link to Microsoft's ISV Application Compatibility page: www.isvappcompat.com<http://www.isvappcompat.com/> . When you have a moment, I would encourage you to visit the site and complete the process to pledge support for your application on Windows 7 by October 22nd 2009 when Windows 7 is officially released.

In addition if you are able to pledge compatibility you'll receive access to a special Windows 7 Partner Marketing Kit that includes a press release with a Microsoft quote, plus customizable marketing templates including; email templates, postcards, web banners, business letter, and copy blocks, all to identify to your customers, or potential customers that your solutions are compatible with Windows 7.

If you provide me with a phone number where to get in touch with you, I will call you to answer any questions you may have.

Once you register on the ISV Application Compatibility site, I would appreciate it if email me to let me know that you have completed so that I can make a note of it for Microsoft. If you register the application under a different partner or application name please let me know in order to track changes. If there is a new version of the application and there are no plans to support Windows 7 on the older version please register the older version as "No planned Support" on the site as well as the new version with desired Win7 compatibility date.

Should you have any questions about this email feel free to call me or send an email to my supervisor at v-mafl at microsoft.com<mailto:v-mafl at microsoft.com>.

Best regards,
Eric Albrecht
800-508-4291 EXT: 309
eric.albrecht at nichecubed.com<mailto:eric.albrecht at nichecubed.com>


----- End forwarded message -----

-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"To me vi is Zen.  To use vi is to practice zen.  Every command is a
koan.  Profound to the user, unintelligible to the uninitiated.  You
discover truth everytime you use it."  --reddy at lion.austin.ibm.com

From ctb at msu.edu  Thu Sep 10 17:20:35 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Thu, 10 Sep 2009 08:20:35 -0700
Subject: [Python-Dev] FWD: Front Runner Program
In-Reply-To: <20090910151155.GA13116@panix.com>
References: <20090910151155.GA13116@panix.com>
Message-ID: <20090910152035.GC26180@idyll.org>

On Thu, Sep 10, 2009 at 08:11:55AM -0700, Aahz wrote:
-> I'm still no-mail on python-dev, forwarding as FYI
->
-> ----- Forwarded message from Eric Albrecht <eric.albrecht at Nichecubed.com> -----
-> 
-> From: Eric Albrecht <eric.albrecht at Nichecubed.com>
-> To: "webmaster at python.org" <webmaster at python.org>
-> Date: Thu, 10 Sep 2009 10:48:11 -0400
-> Subject: Front Runner Program 
-> 
-> Regarding: Windows 7 Compatibility for Python Application.

Thanks, Aahz!

I don't see a Windows 7 buildbot up here:

	http://www.python.org/dev/buildbot/all/

but I confess that I'm bad at reading these pages.  Has anyone tried
compiling either trunk or py3k on Win 7?  Would this be useful?

My recently acquired* MSDN account has led me to getting XP up and
running in a VM, and I would be happy to try other Windows OSes of
interest.

--titus

* acquired courtesy of Snakebite/Trent Nelson.
--
C. Titus Brown, ctb at msu.edu

From nad at acm.org  Thu Sep 10 18:23:08 2009
From: nad at acm.org (Ned Deily)
Date: Thu, 10 Sep 2009 09:23:08 -0700
Subject: [Python-Dev] Python 2.6.3
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
Message-ID: <nad-C736AE.09230710092009@ger.gmane.org>

In article <9D506035-7C2D-4929-A134-E88EEB7B7D9E at python.org>,
 Barry Warsaw <barry at python.org> wrote:

> On Sep 9, 2009, at 1:29 PM, Ned Deily wrote:
> 
> > In article <11A6545D-7204-4F61-B55B-1CC77CB5645E at python.org>,
> > Barry Warsaw <barry at python.org> wrote:
> >> I still want to release by the 25th, but I'd be willing to move the  
> >> rc
> >> to Monday the 21st.  We're really just trying to avoid a brown bag
> >> moment, so that should give us enough time to double check the  
> >> releases.
> >
> > The recent release of OS X 10.6 (Snow Leopard) has triggered a fair
> > amount of 2.6 bug tracker activity, since 10.6 now includes 2.6  
> > (2.6.1)
> > and a 64-bit version at that.   A number of patches have either just
> > been checked-in over the past couple of weeks or are getting some
> > exposure before check-in.  Given the timing and the (appropriate)
> > infrequency of 2.6.x releases, I think it would be unfortunate to push
> > 2.6.3 out the door without ensuring that it works well on 10.6.
> > Therefore, I propose that 2.6.3 should have 10.6 compatibility as a
> > "release goal".
> >
> > Without trying to put Ronald on the spot (too much!), it would be a  
> > good
> > idea to get his assessment where things stand wrt 2.6 on 10.6 before
> > setting a final release date.
> 
> I'm hoping that Python won't have any issues building and running on  
> 10.6, but I don't have it yet so I can't personally test it out.
> 
> How would you quantify "works well"?  Do you have any thoughts on  
> tests you'd run other than the standard test suite?  If 2.6.3 is shown  
> to pass its test suite on 10.5.x, is that good enough?  Are the  
> specific bug fixes necessary for 10.6?

Running the standard test suite on 10.6 and seeing no regressions 
compared to the same suite on 10.5.x seems a reasonable necessary 
requirement.  We have the resources to do that.  Beyond that, as Ronald 
suggests, I think it important to go through the open issues in the next 
couple of days and identify and flag any potential release-blockers 
(besides the IDLE problem already mentioned).

One other open issue is 64-bit support in the python.org OS X installer.  
There have been discussions and requests in the past and, with Apple 
providing 64-bit out of the box in 10.6, it seems like it's time to 
provide something on python.org as well.   One option: continue to 
provide a 32-bit only installer for ppc and i386 for 10.3.9 and beyond 
and add a second installer image with 3-way (ppc, i386, x86_64 but no 
ppc64) 32/64 for 10.5 and beyond.   Ronald, is that your current 
thinking?

-- 
 Ned Deily,
 nad at acm.org


From martin at v.loewis.de  Thu Sep 10 18:57:09 2009
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 10 Sep 2009 18:57:09 +0200
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
Message-ID: <4AA92FE5.5060004@v.loewis.de>

Barry Warsaw schrieb:
> I had previously wanted to release Python 2.6.3 over the summer, but for
> various personal reasons, the summer was just too insane.  I'd like to
> reschedule a 2.6.3 release, shooting for final release on 25-September.

I'm travelling that week (as well as the time until then), so I would
prefer to make it a week later.

Regards,
Martin

From martin at v.loewis.de  Thu Sep 10 19:03:01 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 10 Sep 2009 19:03:01 +0200
Subject: [Python-Dev] FWD: Front Runner Program
In-Reply-To: <20090910152035.GC26180@idyll.org>
References: <20090910151155.GA13116@panix.com>
	<20090910152035.GC26180@idyll.org>
Message-ID: <4AA93145.2050602@v.loewis.de>


> My recently acquired* MSDN account has led me to getting XP up and
> running in a VM, and I would be happy to try other Windows OSes of
> interest.

Without checking the specific program, I think chances are that there
are higher requirements for being listed in some Microsoft list than
"it works", in particular wrt. the installation procedure.

Of course, testing that it works is a good idea independent of any logo
certification.

Regards,
Martin


From janssen at parc.com  Thu Sep 10 19:09:50 2009
From: janssen at parc.com (Bill Janssen)
Date: Thu, 10 Sep 2009 10:09:50 PDT
Subject: [Python-Dev] Controlling the cipher list for SSL connections
In-Reply-To: <aeddc9570909070932t39933602g7605a10c234bb0cc@mail.gmail.com>
References: <aeddc9570909070909v3c9bf3b0of7ebb4b62857aa00@mail.gmail.com>
	<4AA53199.5090409@voidspace.org.uk>
	<aeddc9570909070932t39933602g7605a10c234bb0cc@mail.gmail.com>
Message-ID: <699.1252602590@parc.com>

Thanks, Chris.  Can you explain why you want to set the cipher list
explicitly?  IMO, it's usually better to select a security scheme (TLS1,
or SSLv3, etc.), and let the implementation pick the cipher list.

Bill

Chris Frantz <frantzcj at gmail.com> wrote:

> Done.
> 
> Attached to Issue 3597, which is a similar request to mine.
> 
> Best Regards,
> --Chris
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com

From janssen at parc.com  Thu Sep 10 20:14:51 2009
From: janssen at parc.com (Bill Janssen)
Date: Thu, 10 Sep 2009 11:14:51 PDT
Subject: [Python-Dev] Controlling the cipher list for SSL connections
In-Reply-To: <aeddc9570909101104s667c98f8va135010e5111aa26@mail.gmail.com>
References: <aeddc9570909070909v3c9bf3b0of7ebb4b62857aa00@mail.gmail.com>
	<4AA53199.5090409@voidspace.org.uk>
	<aeddc9570909070932t39933602g7605a10c234bb0cc@mail.gmail.com>
	<699.1252602590@parc.com>
	<aeddc9570909101104s667c98f8va135010e5111aa26@mail.gmail.com>
Message-ID: <4250.1252606491@parc.com>

Chris,

OK, seems reasonable.  Thanks.  In the near term, can you do this with
M2Crypto or PyOpenSSL?

When I started this update in 2007, we were trying to keep the API
simple to avoid confusing people and avoid competition with the two
full-fledged toolkits out there.  But I don't see any real reason not to
extend the API a bit.

Bill

Chris Frantz <frantzcj at gmail.com> wrote:

> Bill,
> 
> I agree that it's usually better to let the SSL implementation pick
> the ciphers.
> 
> I have a certain device that I'd like to talk to that is running on an
> underpowered embedded CPU.   When I let OpenSSL pick the ciphers, it
> chooses something like EDH-RSA-AES-SHA and takes about 3.5 seconds to
> finish  the handshake.  If I can restrict the cipher list to
> RSA-RC4-SHA I can reduce the handshake time to less than a second and
> improve the throughput of any bulk data transfer over the connection.
> 
> --Chris
> 
> 
> 
> On Thu, Sep 10, 2009 at 12:09 PM, Bill Janssen<janssen at parc.com> wrote:
> > Thanks, Chris. ?Can you explain why you want to set the cipher list
> > explicitly? ?IMO, it's usually better to select a security scheme (TLS1,
> > or SSLv3, etc.), and let the implementation pick the cipher list.
> >
> > Bill
> >
> > Chris Frantz <frantzcj at gmail.com> wrote:
> >
> >> Done.
> >>
> >> Attached to Issue 3597, which is a similar request to mine.
> >>
> >> Best Regards,
> >> --Chris
> >> _______________________________________________
> >> Python-Dev mailing list
> >> Python-Dev at python.org
> >> http://mail.python.org/mailman/listinfo/python-dev
> >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com
> >

From htoivonen at spikesource.com  Thu Sep 10 20:32:16 2009
From: htoivonen at spikesource.com (Heikki Toivonen)
Date: Thu, 10 Sep 2009 11:32:16 -0700
Subject: [Python-Dev] Controlling the cipher list for SSL connections
In-Reply-To: <4250.1252606491@parc.com>
References: <aeddc9570909070909v3c9bf3b0of7ebb4b62857aa00@mail.gmail.com>	<4AA53199.5090409@voidspace.org.uk>	<aeddc9570909070932t39933602g7605a10c234bb0cc@mail.gmail.com>	<699.1252602590@parc.com>	<aeddc9570909101104s667c98f8va135010e5111aa26@mail.gmail.com>
	<4250.1252606491@parc.com>
Message-ID: <4AA94630.4040904@spikesource.com>

Bill Janssen wrote:
> OK, seems reasonable.  Thanks.  In the near term, can you do this with
> M2Crypto or PyOpenSSL?
> 
> When I started this update in 2007, we were trying to keep the API
> simple to avoid confusing people and avoid competition with the two
> full-fledged toolkits out there.  But I don't see any real reason not to
> extend the API a bit.

Speaking as the M2Crypto maintainer, I don't mind the stdlib competing
with M2Crypto/getting better at SSL. In fact, I would actually like to
see the stdlib SSL implementation getting good enough so that people
would not need M2Crypto for SSL (except maybe in special circumstances).
There is much M2Crypto does besides SSL so this wouldn't even obsolete it.

One of the main things IMO missing from stdlib SSL implementation is
hostname checking by default (with override option), but I know you and
I have different opinions on this. I would be happy to provide patches
against the stdlib SSL implementation for some things M2Crypto does that
the stdlib SSL module is missing if we could agree on the
features/design first. Simple is good, but I'd like the defaults to be
secure and commonly overridden things to be overrideable.

-- 
  Heikki Toivonen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090910/d6367287/attachment.pgp>

From frantzcj at gmail.com  Thu Sep 10 20:26:09 2009
From: frantzcj at gmail.com (Chris Frantz)
Date: Thu, 10 Sep 2009 13:26:09 -0500
Subject: [Python-Dev] Controlling the cipher list for SSL connections
In-Reply-To: <4250.1252606491@parc.com>
References: <aeddc9570909070909v3c9bf3b0of7ebb4b62857aa00@mail.gmail.com>
	<4AA53199.5090409@voidspace.org.uk>
	<aeddc9570909070932t39933602g7605a10c234bb0cc@mail.gmail.com>
	<699.1252602590@parc.com>
	<aeddc9570909101104s667c98f8va135010e5111aa26@mail.gmail.com>
	<4250.1252606491@parc.com>
Message-ID: <aeddc9570909101126k7e155dfbh9e11e234dda97a9f@mail.gmail.com>

Bill,

For now, using pyOpenSSL is acceptable.  I just discovered that the
web.py framework wants pyOpenSSL.  Since my project is also using
web.py, I'll need pyOpenSSL anyway.

Thank you,
--Chris


On Thu, Sep 10, 2009 at 1:14 PM, Bill Janssen<janssen at parc.com> wrote:
> Chris,
>
> OK, seems reasonable. ?Thanks. ?In the near term, can you do this with
> M2Crypto or PyOpenSSL?
>
> When I started this update in 2007, we were trying to keep the API
> simple to avoid confusing people and avoid competition with the two
> full-fledged toolkits out there. ?But I don't see any real reason not to
> extend the API a bit.
>
> Bill
>
> Chris Frantz <frantzcj at gmail.com> wrote:
>
>> Bill,
>>
>> I agree that it's usually better to let the SSL implementation pick
>> the ciphers.
>>
>> I have a certain device that I'd like to talk to that is running on an
>> underpowered embedded CPU. ? When I let OpenSSL pick the ciphers, it
>> chooses something like EDH-RSA-AES-SHA and takes about 3.5 seconds to
>> finish ?the handshake. ?If I can restrict the cipher list to
>> RSA-RC4-SHA I can reduce the handshake time to less than a second and
>> improve the throughput of any bulk data transfer over the connection.
>>
>> --Chris
>>
>>
>>
>> On Thu, Sep 10, 2009 at 12:09 PM, Bill Janssen<janssen at parc.com> wrote:
>> > Thanks, Chris. ?Can you explain why you want to set the cipher list
>> > explicitly? ?IMO, it's usually better to select a security scheme (TLS1,
>> > or SSLv3, etc.), and let the implementation pick the cipher list.
>> >
>> > Bill
>> >
>> > Chris Frantz <frantzcj at gmail.com> wrote:
>> >
>> >> Done.
>> >>
>> >> Attached to Issue 3597, which is a similar request to mine.
>> >>
>> >> Best Regards,
>> >> --Chris
>> >> _______________________________________________
>> >> Python-Dev mailing list
>> >> Python-Dev at python.org
>> >> http://mail.python.org/mailman/listinfo/python-dev
>> >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com
>> >
>

From frantzcj at gmail.com  Thu Sep 10 20:04:22 2009
From: frantzcj at gmail.com (Chris Frantz)
Date: Thu, 10 Sep 2009 13:04:22 -0500
Subject: [Python-Dev] Controlling the cipher list for SSL connections
In-Reply-To: <699.1252602590@parc.com>
References: <aeddc9570909070909v3c9bf3b0of7ebb4b62857aa00@mail.gmail.com>
	<4AA53199.5090409@voidspace.org.uk>
	<aeddc9570909070932t39933602g7605a10c234bb0cc@mail.gmail.com>
	<699.1252602590@parc.com>
Message-ID: <aeddc9570909101104s667c98f8va135010e5111aa26@mail.gmail.com>

Bill,

I agree that it's usually better to let the SSL implementation pick
the ciphers.

I have a certain device that I'd like to talk to that is running on an
underpowered embedded CPU.   When I let OpenSSL pick the ciphers, it
chooses something like EDH-RSA-AES-SHA and takes about 3.5 seconds to
finish  the handshake.  If I can restrict the cipher list to
RSA-RC4-SHA I can reduce the handshake time to less than a second and
improve the throughput of any bulk data transfer over the connection.

--Chris



On Thu, Sep 10, 2009 at 12:09 PM, Bill Janssen<janssen at parc.com> wrote:
> Thanks, Chris. ?Can you explain why you want to set the cipher list
> explicitly? ?IMO, it's usually better to select a security scheme (TLS1,
> or SSLv3, etc.), and let the implementation pick the cipher list.
>
> Bill
>
> Chris Frantz <frantzcj at gmail.com> wrote:
>
>> Done.
>>
>> Attached to Issue 3597, which is a similar request to mine.
>>
>> Best Regards,
>> --Chris
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com
>

From janssen at parc.com  Thu Sep 10 21:01:12 2009
From: janssen at parc.com (Bill Janssen)
Date: Thu, 10 Sep 2009 12:01:12 PDT
Subject: [Python-Dev] evolving the SSL module API
In-Reply-To: <4AA94630.4040904@spikesource.com>
Message-ID: <5335.1252609272@parc.com>

Heikki, I'm OK with this, too.  would you like to propose an extended
API for the SSL module?  That would give us a starting point to talk
about.

This should probably be a PEP, just for the sake of writing things down.

As you say, the hostname checking feature seems to me possibly
appropriate for some application protocols, though it's made the use of
HTTPS as a transport-level protocol unnecessarily confusing and buggy.
I don't see putting that into the SSL module as a default, but perhaps a
utility function in that module, to check a server-side cert against a
hostname, is a good idea.

Bill


Heikki Toivonen <htoivonen at spikesource.com> wrote:

> Bill Janssen wrote:
> > OK, seems reasonable.  Thanks.  In the near term, can you do this with
> > M2Crypto or PyOpenSSL?
> > 
> > When I started this update in 2007, we were trying to keep the API
> > simple to avoid confusing people and avoid competition with the two
> > full-fledged toolkits out there.  But I don't see any real reason not to
> > extend the API a bit.
> 
> Speaking as the M2Crypto maintainer, I don't mind the stdlib competing
> with M2Crypto/getting better at SSL. In fact, I would actually like to
> see the stdlib SSL implementation getting good enough so that people
> would not need M2Crypto for SSL (except maybe in special circumstances).
> There is much M2Crypto does besides SSL so this wouldn't even obsolete it.
> 
> One of the main things IMO missing from stdlib SSL implementation is
> hostname checking by default (with override option), but I know you and
> I have different opinions on this. I would be happy to provide patches
> against the stdlib SSL implementation for some things M2Crypto does that
> the stdlib SSL module is missing if we could agree on the
> features/design first. Simple is good, but I'd like the defaults to be
> secure and commonly overridden things to be overrideable.
> 
> -- 
>   Heikki Toivonen
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com


From jnoller at gmail.com  Thu Sep 10 21:17:33 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 10 Sep 2009 15:17:33 -0400
Subject: [Python-Dev] evolving the SSL module API
In-Reply-To: <5335.1252609272@parc.com>
References: <4AA94630.4040904@spikesource.com> <5335.1252609272@parc.com>
Message-ID: <4222a8490909101217v4959e340m4986593863fe196@mail.gmail.com>

There's also the patch to httplib that Devin Cook has been working on
for SSL enhancements, some of which do name checking. He's got most of
a patch completed.

On Thu, Sep 10, 2009 at 3:01 PM, Bill Janssen <janssen at parc.com> wrote:
> Heikki, I'm OK with this, too. ?would you like to propose an extended
> API for the SSL module? ?That would give us a starting point to talk
> about.
>
> This should probably be a PEP, just for the sake of writing things down.
>
> As you say, the hostname checking feature seems to me possibly
> appropriate for some application protocols, though it's made the use of
> HTTPS as a transport-level protocol unnecessarily confusing and buggy.
> I don't see putting that into the SSL module as a default, but perhaps a
> utility function in that module, to check a server-side cert against a
> hostname, is a good idea.
>
> Bill
>
>
> Heikki Toivonen <htoivonen at spikesource.com> wrote:
>
>> Bill Janssen wrote:
>> > OK, seems reasonable. ?Thanks. ?In the near term, can you do this with
>> > M2Crypto or PyOpenSSL?
>> >
>> > When I started this update in 2007, we were trying to keep the API
>> > simple to avoid confusing people and avoid competition with the two
>> > full-fledged toolkits out there. ?But I don't see any real reason not to
>> > extend the API a bit.
>>
>> Speaking as the M2Crypto maintainer, I don't mind the stdlib competing
>> with M2Crypto/getting better at SSL. In fact, I would actually like to
>> see the stdlib SSL implementation getting good enough so that people
>> would not need M2Crypto for SSL (except maybe in special circumstances).
>> There is much M2Crypto does besides SSL so this wouldn't even obsolete it.
>>
>> One of the main things IMO missing from stdlib SSL implementation is
>> hostname checking by default (with override option), but I know you and
>> I have different opinions on this. I would be happy to provide patches
>> against the stdlib SSL implementation for some things M2Crypto does that
>> the stdlib SSL module is missing if we could agree on the
>> features/design first. Simple is good, but I'd like the defaults to be
>> secure and commonly overridden things to be overrideable.
>>
>> --
>> ? Heikki Toivonen
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com
>

From rowen at uw.edu  Thu Sep 10 21:55:56 2009
From: rowen at uw.edu (Russell E. Owen)
Date: Thu, 10 Sep 2009 12:55:56 -0700
Subject: [Python-Dev] Python 2.6.3
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<C11F8211-8938-4C68-8221-C3C73E8E517B@mac.com>
Message-ID: <rowen-F055DF.12555610092009@news.gmane.org>

In article <C11F8211-8938-4C68-8221-C3C73E8E517B at mac.com>,
 Ronald Oussoren <ronaldoussoren at mac.com> wrote:

> On 9 Sep, 2009, at 19:29, Ned Deily wrote:
> >
> >
> > Without trying to put Ronald on the spot (too much!), it would be a  
> > good
> > idea to get his assessment where things stand wrt 2.6 on 10.6 before
> > setting a final release date.
> 
> MacOS X 10.6 support should be stable now, except for a critical issue
> with IDLE: opening a new window hangs IDLE (issue 6864).
> 
> That said, I haven't scanned the issue tracker for more 10.6 related  
> issues.
> 
> BTW. I'm fine with a sept. 25th release, that should give us enough time
> to shake out issues with OSX 10.6 support.
> 
> Ronald---------------------------------------------------------------------

Some or all earlier Mac binaries of Python 2.6 were not compatible with 
3rd party Aqua Tcl/Tk (e.g. ActiveState's versions) -- at least on MacOS 
X 10.5. I hope that will be fixed with the current release.

-- Russell


From p.f.moore at gmail.com  Thu Sep 10 22:52:16 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 10 Sep 2009 21:52:16 +0100
Subject: [Python-Dev] FWD: Front Runner Program
In-Reply-To: <20090910152035.GC26180@idyll.org>
References: <20090910151155.GA13116@panix.com>
	<20090910152035.GC26180@idyll.org>
Message-ID: <79990c6b0909101352i20f697b1pab8987643cba0fd8@mail.gmail.com>

2009/9/10 C. Titus Brown <ctb at msu.edu>:
> I don't see a Windows 7 buildbot up here:
>
> ? ? ? ?http://www.python.org/dev/buildbot/all/
>
> but I confess that I'm bad at reading these pages. ?Has anyone tried
> compiling either trunk or py3k on Win 7? ?Would this be useful?

I plan to do something like this but PC breakage has delayed this.

Paul.

From devin.c.cook at gmail.com  Thu Sep 10 23:00:58 2009
From: devin.c.cook at gmail.com (Devin Cook)
Date: Thu, 10 Sep 2009 17:00:58 -0400
Subject: [Python-Dev] evolving the SSL module API
In-Reply-To: <4222a8490909101217v4959e340m4986593863fe196@mail.gmail.com>
References: <4AA94630.4040904@spikesource.com> <5335.1252609272@parc.com>
	<4222a8490909101217v4959e340m4986593863fe196@mail.gmail.com>
Message-ID: <8847f980909101400y683ac7d6vad673838be832544@mail.gmail.com>

Yes, my patch implements hostname checking in httplib (although I
haven't had time to do much testing). I also made the documentation
changes, but have not yet created any test cases since there really
aren't any HTTPS test cases in the test_httplib.py file (which is
probably another issue that needs attention).

We had talked a month or two back about including hostname checking in
the ssl module, but the consensus seemed to be that it doesn't belong
there.

Personally, I would like to see it make it into the ssl module, as
that would mean all the modules that use the ssl module (httplib,
etc.) wouldn't have to write their own (and it isn't very
straightforward... lots of different RFCs involved). Just my 2 cents.

-Devin

On Thu, Sep 10, 2009 at 3:17 PM, Jesse Noller <jnoller at gmail.com> wrote:
> There's also the patch to httplib that Devin Cook has been working on
> for SSL enhancements, some of which do name checking. He's got most of
> a patch completed.
>
> On Thu, Sep 10, 2009 at 3:01 PM, Bill Janssen <janssen at parc.com> wrote:
>> Heikki, I'm OK with this, too. ?would you like to propose an extended
>> API for the SSL module? ?That would give us a starting point to talk
>> about.
>>
>> This should probably be a PEP, just for the sake of writing things down.
>>
>> As you say, the hostname checking feature seems to me possibly
>> appropriate for some application protocols, though it's made the use of
>> HTTPS as a transport-level protocol unnecessarily confusing and buggy.
>> I don't see putting that into the SSL module as a default, but perhaps a
>> utility function in that module, to check a server-side cert against a
>> hostname, is a good idea.
>>
>> Bill
>>
>>

From cs at zip.com.au  Fri Sep 11 08:06:58 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 11 Sep 2009 16:06:58 +1000
Subject: [Python-Dev] pthreads, fork, import, and execvp
In-Reply-To: <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com>
References: <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com>
Message-ID: <20090911060658.GA19749@cskk.homeip.net>

On 25Jul2009 10:25, Gregory P. Smith <greg at krypto.org> wrote:
| On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters<thomas at python.org> wrote:
| > So attached (and at http://codereview.appspot.com/96125/show ) is a
| > preliminary fix, correcting the problem with os.fork(), os.forkpty() and
| > os.fork1(). This doesn't expose a general API for C code to use, for two
| > reasons: it's not easy, and I need this fix more than I need the API change
| > :-) (I actually need this fix myself for Python 2.4, but it applies fairly
| > cleanly.)
| 
| This looks good to me.
[...]

Where's this stand with respect to the upcoming Python 2.6.3?
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Life is pleasant. Death is peaceful. It's the transition that's troublesome.
- Isaac Asimov

From mueller.os at web.De  Fri Sep 11 09:51:39 2009
From: mueller.os at web.De (R. =?ISO-8859-1?Q?M=FCller?=)
Date: Fri, 11 Sep 2009 09:51:39 +0200
Subject: [Python-Dev] M.L.S.(Multiple Language Support) in Core-Compiler
Message-ID: <1252655499.3392.1.camel@Klapprechner>

How to adapt the existing M.L.S. from Compilermessages to
Core-Compiler-Words.

Where to begin, for implemantation of this feature?




From thomas at python.org  Fri Sep 11 13:09:55 2009
From: thomas at python.org (Thomas Wouters)
Date: Fri, 11 Sep 2009 13:09:55 +0200
Subject: [Python-Dev] pthreads, fork, import, and execvp
In-Reply-To: <20090911060658.GA19749@cskk.homeip.net>
References: <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com>
	<20090911060658.GA19749@cskk.homeip.net>
Message-ID: <9e804ac0909110409o5bc8b018icc40766b2aacf3e3@mail.gmail.com>

On Fri, Sep 11, 2009 at 08:06, Cameron Simpson <cs at zip.com.au> wrote:

> On 25Jul2009 10:25, Gregory P. Smith <greg at krypto.org> wrote:
> | On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters<thomas at python.org>
> wrote:
> | > So attached (and at http://codereview.appspot.com/96125/show ) is a
> | > preliminary fix, correcting the problem with os.fork(), os.forkpty()
> and
> | > os.fork1(). This doesn't expose a general API for C code to use, for
> two
> | > reasons: it's not easy, and I need this fix more than I need the API
> change
> | > :-) (I actually need this fix myself for Python 2.4, but it applies
> fairly
> | > cleanly.)
> |
> | This looks good to me.
> [...]
>
> Where's this stand with respect to the upcoming Python 2.6.3?


Unless anyone speaks up, I'll submit the fix (without the change in
semantics on AIX) to release26-maint early next week, so it would be in
2.6.3.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090911/c45a4984/attachment.htm>

From barry at python.org  Fri Sep 11 17:29:22 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 11 Sep 2009 11:29:22 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <4AA92FE5.5060004@v.loewis.de>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA92FE5.5060004@v.loewis.de>
Message-ID: <E6F7D80F-82EC-453F-9293-1555FCEB1BFC@python.org>

On Sep 10, 2009, at 12:57 PM, Martin v. L?wis wrote:

> Barry Warsaw schrieb:
>> I had previously wanted to release Python 2.6.3 over the summer,  
>> but for
>> various personal reasons, the summer was just too insane.  I'd like  
>> to
>> reschedule a 2.6.3 release, shooting for final release on 25- 
>> September.
>
> I'm travelling that week (as well as the time until then), so I would
> prefer to make it a week later.

Cool, thanks Martin.  Let's make it:

2.6.3rc1 on 29-sep
2.6.3 final on 02-oct

That gives us an extra day between rc and final.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090911/84292594/attachment.pgp>

From status at bugs.python.org  Fri Sep 11 18:07:22 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 11 Sep 2009 18:07:22 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090911160722.1820578637@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (09/04/09 - 09/11/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2394 open (+32) / 16312 closed (+15) / 18706 total (+47)

Open issues with patches:   950

Average duration of open issues: 660 days.
Median duration of open issues: 416 days.

Open Issues Breakdown
   open  2361 (+32)
pending    32 ( +0)

Issues Created Or Reopened (48)
_______________________________

should we include argparse                                       09/10/09
       http://bugs.python.org/issue6247    reopened michael.foord                 
                                                                               

httplib's _read_chunked extremely slow for lots of chunks        09/04/09
CLOSED http://bugs.python.org/issue6838    created  cjw296                        
       easy                                                                    

zipfile can't extract file                                       09/04/09
       http://bugs.python.org/issue6839    created  NewerCookie                   
                                                                               

genericpath: _splitext returns '.ext', '' instead of '', '.ext'  09/04/09
CLOSED http://bugs.python.org/issue6840    created  optimix                       
       patch                                                                   

A typo in Doc/library/stdtypes.rst                               09/05/09
CLOSED http://bugs.python.org/issue6841    created  cheeselee                     
       patch                                                                   

program run in interpreter, fails at command line                09/05/09
CLOSED http://bugs.python.org/issue6842    created  jvanpraag                     
                                                                               

No documentation for the module argument to warnings.filterwarni 09/05/09
CLOSED http://bugs.python.org/issue6843    reopened exarkun                       
                                                                               

BaseException DeprecationError raises inappropriately            09/05/09
       http://bugs.python.org/issue6844    created  aisaac                        
                                                                               

Restart support in binary upload for ftplib                      09/05/09
       http://bugs.python.org/issue6845    created  pablomouzo                    
       patch                                                                   

bytearray.pop() returns negative ints                            09/05/09
CLOSED http://bugs.python.org/issue6846    created  hagen                         
       patch                                                                   

Exception strings for bytearray say "bytes"                      09/05/09
CLOSED http://bugs.python.org/issue6847    created  hagen                         
       patch                                                                   

curses module build failure (ncurses 5.7)                        09/06/09
CLOSED http://bugs.python.org/issue6848    created  marketdickinson               
                                                                               

Tutorial changes                                                 09/06/09
       http://bugs.python.org/issue6849    created  michael.foord                 
       easy                                                                    

decimal.py: format_dict                                          09/06/09
CLOSED http://bugs.python.org/issue6850    created  skrah                         
                                                                               

urllib.urlopen crashes in a thread on Snow Leopard               09/07/09
       http://bugs.python.org/issue6851    created  jweber                        
                                                                               

Unicode IO not working in cgi applet                             09/07/09
CLOSED http://bugs.python.org/issue6852    created  loveminix                     
                                                                               

system proxy not used for https (on windows)                     09/07/09
       http://bugs.python.org/issue6853    created  ldeller                       
       patch                                                                   

UnicodeDecodeError when retrieving binary data from cgi.FieldSto 09/07/09
       http://bugs.python.org/issue6854    created  loveminix                     
                                                                               

ihooks support for relative imports                              09/07/09
       http://bugs.python.org/issue6855    created  nascheme                      
                                                                               

allow setting uid and gid when creating tar files                09/07/09
       http://bugs.python.org/issue6856    created  tarek                         
       patch                                                                   

float().__format__() default alignment                           09/07/09
       http://bugs.python.org/issue6857    created  skrah                         
                                                                               

This is a python file, apply syntax highlighting                 09/07/09
       http://bugs.python.org/issue6858    created  gert                          
                                                                               

stdint (eg. uint64_t) for ctypes                                 09/07/09
CLOSED http://bugs.python.org/issue6859    created  haypo                         
                                                                               

Inconsistent naming of custom command in setup.py help output    09/08/09
       http://bugs.python.org/issue6860    created  herzbube                      
                                                                               

bytearray.__new__ doesn't subclass                               09/08/09
CLOSED http://bugs.python.org/issue6861    created  kaizhu                        
                                                                               

exec(), locals() and local variable access                       09/08/09
CLOSED http://bugs.python.org/issue6862    created  ooev                          
                                                                               

Wrong linker command if CXX set to "ccache g++"                  09/08/09
       http://bugs.python.org/issue6863    created  perttikellomaki               
       patch                                                                   

IDLE 2.6.1 locks up on Mac OS 10.6                               09/08/09
       http://bugs.python.org/issue6864    created  dpogg1                        
                                                                               

Refcount error in pwd module                                     09/08/09
CLOSED http://bugs.python.org/issue6865    created  caglar10ur                    
       patch                                                                   

TestLoader.loadTestsFromName does not use suiteClass attribute t 09/08/09
       http://bugs.python.org/issue6866    created  MarkRoddy                     
       patch                                                                   

return value of epoll.register                                   09/09/09
       http://bugs.python.org/issue6867    created  shaovie                       
                                                                               

Check errno of epoll_ctrl                                        09/09/09
       http://bugs.python.org/issue6868    created  shaovie                       
                                                                               

Embedded python crashed on 4th run, if "ctypes" is used          09/09/09
       http://bugs.python.org/issue6869    created  warlock                       
                                                                               

sybase module for python3.1                                      09/09/09
CLOSED http://bugs.python.org/issue6870    created  pravinpchavan                 
                                                                               

decimal.py: more format issues                                   09/09/09
       http://bugs.python.org/issue6871    created  skrah                         
                                                                               

Support system readline on OS X 10.6                             09/09/09
       http://bugs.python.org/issue6872    created  marketdickinson               
       patch                                                                   

posix_lchown: possible overflow of uid, gid                      09/09/09
       http://bugs.python.org/issue6873    created  boya                          
       patch                                                                   

sequence method  .count() and .index() shoud be in immutable seq 09/10/09
       http://bugs.python.org/issue6874    created  s7v7nislands                  
                                                                               

add os.close() suggestion to mkstemp documentation               09/10/09
       http://bugs.python.org/issue6875    created  vincele                       
       patch                                                                   

readline documentation example error                             09/10/09
       http://bugs.python.org/issue6876    created  schwarz                       
                                                                               

enable compilation of readline module on Mac OS X 10.5 and 10.6  09/10/09
       http://bugs.python.org/issue6877    created  zvezdan                       
       patch                                                                   

outdated docstring in tkinter.Canvas.coords                      09/10/09
       http://bugs.python.org/issue6878    created  SilentGhost                   
                                                                               

misstatement in example explanation using raise                  09/11/09
       http://bugs.python.org/issue6879    created  bluebloodpole                 
                                                                               

class needs forward reference                                    09/11/09
       http://bugs.python.org/issue6880    created  bluebloodpole                 
                                                                               

incorrect signature in doc for PyByteArray_Resize                09/11/09
       http://bugs.python.org/issue6881    created  egreen                        
                                                                               

uuid creates zombies                                             09/11/09
       http://bugs.python.org/issue6882    created  dhg                           
                                                                               

OptionParser.allow_interspersed_args is undocumented             09/11/09
       http://bugs.python.org/issue6883    created  skip.montanaro                
                                                                               

Impossible to include file in sdist that starts with 'build' on  09/11/09
       http://bugs.python.org/issue6884    created  cjw296                        
                                                                               



Issues Now Closed (32)
______________________

IDLE - configDialog - new layout for key config                   659 days
       http://bugs.python.org/issue1457    amaury.forgeotdarc            
       patch                                                                   

Cmd module doesn't support readline completion on OSX Leopard     492 days
       http://bugs.python.org/issue2740    ronaldoussoren                
                                                                               

Cannot build _multiprocessing, math, mmap and readline of Python  320 days
       http://bugs.python.org/issue4204    zvezdan                       
       patch                                                                   

Mac DMG install missing version.plist required by bundlebuilder.  236 days
       http://bugs.python.org/issue4937    ronaldoussoren                
                                                                               

PEP 372:  OrderedDict                                             191 days
       http://bugs.python.org/issue5397    matthieu.labbe                
       patch                                                                   

Darwin framework libpython3.0.a is not a normal static library    172 days
       http://bugs.python.org/issue5514    ronaldoussoren                
                                                                               

[HP-UX] ld: Unrecognized argument: +s -L<dir>                     100 days
       http://bugs.python.org/issue6163    tarek                         
       patch                                                                   

c_char_p return value returns string, not bytes                    92 days
       http://bugs.python.org/issue6239    haypo                         
       patch                                                                   

Native (and default) tarfile support for setup.py sdist	in distu   82 days
       http://bugs.python.org/issue6296    tarek                         
                                                                               

Build Applet.app                                                   45 days
       http://bugs.python.org/issue6552    ronaldoussoren                
                                                                               

decimal.py: minor issues && usability                              11 days
       http://bugs.python.org/issue6795    marketdickinson               
                                                                               

build fails on Snow Leopard                                         8 days
       http://bugs.python.org/issue6802    blb                           
       patch                                                                   

Mark the compiler package as deprecated                             1 days
       http://bugs.python.org/issue6837    georg.brandl                  
                                                                               

httplib's _read_chunked extremely slow for lots of chunks           0 days
       http://bugs.python.org/issue6838    cjw296                        
       easy                                                                    

genericpath: _splitext returns '.ext', '' instead of '', '.ext'     0 days
       http://bugs.python.org/issue6840    benjamin.peterson             
       patch                                                                   

A typo in Doc/library/stdtypes.rst                                  0 days
       http://bugs.python.org/issue6841    georg.brandl                  
       patch                                                                   

program run in interpreter, fails at command line                   0 days
       http://bugs.python.org/issue6842    marketdickinson               
                                                                               

No documentation for the module argument to warnings.filterwarni    0 days
       http://bugs.python.org/issue6843    georg.brandl                  
                                                                               

bytearray.pop() returns negative ints                               1 days
       http://bugs.python.org/issue6846    marketdickinson               
       patch                                                                   

Exception strings for bytearray say "bytes"                         1 days
       http://bugs.python.org/issue6847    marketdickinson               
       patch                                                                   

curses module build failure (ncurses 5.7)                           0 days
       http://bugs.python.org/issue6848    marketdickinson               
                                                                               

decimal.py: format_dict                                             1 days
       http://bugs.python.org/issue6850    marketdickinson               
                                                                               

Unicode IO not working in cgi applet                                1 days
       http://bugs.python.org/issue6852    marketdickinson               
                                                                               

stdint (eg. uint64_t) for ctypes                                    0 days
       http://bugs.python.org/issue6859    haypo                         
                                                                               

bytearray.__new__ doesn't subclass                                  0 days
       http://bugs.python.org/issue6861    benjamin.peterson             
                                                                               

exec(), locals() and local variable access                          0 days
       http://bugs.python.org/issue6862    amaury.forgeotdarc            
                                                                               

Refcount error in pwd module                                        1 days
       http://bugs.python.org/issue6865    benjamin.peterson             
       patch                                                                   

sybase module for python3.1                                         0 days
       http://bugs.python.org/issue6870    loewis                        
                                                                               

Packaging without disturbing an existing installation            2314 days
       http://bugs.python.org/issue734115  ronaldoussoren                
       patch                                                                   

os.path.splitext don't handle unix hidden file correctly         1674 days
       http://bugs.python.org/issue1115886 optimix                       
                                                                               

size_t warnings on OSX 10.3                                      1246 days
       http://bugs.python.org/issue1467201 ronaldoussoren                
                                                                               

Universal MacPython 2.5.1 installation fails                      813 days
       http://bugs.python.org/issue1738250 ronaldoussoren                
                                                                               



Top Issues Most Discussed (10)
______________________________

 14 BaseException DeprecationError raises inappropriately              6 days
open    http://bugs.python.org/issue6844   

 12 Integer & Long types:  Performance improvement of 1.6x to 2x fo   26 days
open    http://bugs.python.org/issue6713   

  9 build fails on Snow Leopard                                        8 days
closed  http://bugs.python.org/issue6802   

  9 [HP-UX] ld: Unrecognized argument: +s -L<dir>                    100 days
closed  http://bugs.python.org/issue6163   

  7 Support system readline on OS X 10.6                               2 days
open    http://bugs.python.org/issue6872   

  6 posix_lchown: possible overflow of uid, gid                        2 days
open    http://bugs.python.org/issue6873   

  6 zipfile can't extract file                                         7 days
open    http://bugs.python.org/issue6839   

  6 Mark the compiler package as deprecated                            1 days
closed  http://bugs.python.org/issue6837   

  6 Add os.link() and os.symlink() and os.path.islink()	support for 1061 days
open    http://bugs.python.org/issue1578269

  5 decimal.py: more format issues                                     2 days
open    http://bugs.python.org/issue6871   




From brett at python.org  Fri Sep 11 20:59:43 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 11 Sep 2009 11:59:43 -0700
Subject: [Python-Dev] M.L.S.(Multiple Language Support) in Core-Compiler
In-Reply-To: <1252655499.3392.1.camel@Klapprechner>
References: <1252655499.3392.1.camel@Klapprechner>
Message-ID: <bbaeab100909111159g68ad401frad53912e2e3059ef@mail.gmail.com>

On Fri, Sep 11, 2009 at 00:51, R. <mueller.os at web.de> wrote:
> How to adapt the existing M.L.S. from Compilermessages to
> Core-Compiler-Words.
>
> Where to begin, for implemantation of this feature?

Um, what existing multi-language support? CPython doesn't have any such thing.

-Brett

From solipsis at pitrou.net  Fri Sep 11 21:03:40 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 11 Sep 2009 19:03:40 +0000 (UTC)
Subject: [Python-Dev] M.L.S.(Multiple Language Support) in Core-Compiler
References: <1252655499.3392.1.camel@Klapprechner>
	<bbaeab100909111159g68ad401frad53912e2e3059ef@mail.gmail.com>
Message-ID: <loom.20090911T210309-160@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:
> 
> On Fri, Sep 11, 2009 at 00:51, R. <mueller.os <at> web.de> wrote:
> > How to adapt the existing M.L.S. from Compilermessages to
> > Core-Compiler-Words.
> >
> > Where to begin, for implemantation of this feature?
> 
> Um, what existing multi-language support? CPython doesn't have any such thing.

Well, I suppose the message was aimed at another mailing-list...




From kalite.lab at feka.tk  Fri Sep 11 13:44:02 2009
From: kalite.lab at feka.tk (Kalite LAB.)
Date: Fri, 11 Sep 2009 14:44:02 +0300
Subject: [Python-Dev] About Python
Message-ID: <mailman.337.1252748775.2835.python-dev@python.org>

Dear  authorized, 
I have a question about Python programming language can help you glad you
do. User as a decimal point "." Instead of character "," we can use that
character? For example, 10.7 instead of 10,7
  
Thanks in advance ...
 
Best Regards...
Feyyaz OZEN
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090911/dd460020/attachment.htm>

From fuzzyman at voidspace.org.uk  Sat Sep 12 11:51:43 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 12 Sep 2009 10:51:43 +0100
Subject: [Python-Dev] About Python
In-Reply-To: <200909120946.n8C9krOj012318@mail2.webfaction.com>
References: <200909120946.n8C9krOj012318@mail2.webfaction.com>
Message-ID: <4AAB6F2F.5020405@voidspace.org.uk>

Kalite LAB. wrote:
> Dear  authorized, 
> I have a question about Python programming language can help you glad 
> you do. User as a decimal point "." Instead of character "," we can 
> use that character? For example, 10.7 instead of 10,7

This mailing list is for the development of Python, not development with 
Python. For general questions about how to do things the python-list 
(comp.lang.python.newsgroup) and python-tutor-list are appropriate 
places to ask.

That aside I *think* that Python is always culture invariant in its 
syntax (you always use the '.' point for floating point literals in 
source code) and *also* for how the float type converts strings to 
numbers (float('10.7') and not float('10,7')).

I assume there is a way of turning strings to floats honoring the 
locale, but off the top of my head I don't know what it is.

All the best,

Michael Foord

>  
> Thanks in advance ...
>  
> Best Regards...
> Feyyaz OZEN
>  
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From jacky830 at gmail.com  Sat Sep 12 13:04:18 2009
From: jacky830 at gmail.com (Jacky Lam)
Date: Sat, 12 Sep 2009 19:04:18 +0800
Subject: [Python-Dev] Anyone knows how to compile pywin32 for 64bit
	operating system
Message-ID: <d9dcd20c0909120404r7ebccc54j132e400e1fc2dbb5@mail.gmail.com>

I would like to compile a 64 bit version of pywin.
I am currently using MingGW for that, but I fail to do so.
Does anyone know how to compile pywin32 for 64bit operating system?
In addition, any examples for that?

Thanks.

Jacky

From fuzzyman at voidspace.org.uk  Sat Sep 12 13:19:09 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sat, 12 Sep 2009 12:19:09 +0100
Subject: [Python-Dev] Anyone knows how to compile pywin32 for
 64bit	operating system
In-Reply-To: <d9dcd20c0909120404r7ebccc54j132e400e1fc2dbb5@mail.gmail.com>
References: <d9dcd20c0909120404r7ebccc54j132e400e1fc2dbb5@mail.gmail.com>
Message-ID: <4AAB83AD.6040103@voidspace.org.uk>

Jacky Lam wrote:
> I would like to compile a 64 bit version of pywin.
> I am currently using MingGW for that, but I fail to do so.
> Does anyone know how to compile pywin32 for 64bit operating system?
> In addition, any examples for that?
>
>   
Hello Jacky,

This list is for the development of Python and pywin32 is not a standard 
part of Python itself.

pywin32 has its own mailing list where you are more likely to get an 
answer to your question:

    http://mail.python.org/mailman/listinfo/python-win32

All the best,

Michael Foord


> Thanks.
>
> Jacky
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From tjgolden at gmail.com  Sat Sep 12 15:08:33 2009
From: tjgolden at gmail.com (Tim Golden)
Date: Sat, 12 Sep 2009 14:08:33 +0100
Subject: [Python-Dev] Anyone knows how to compile pywin32 for
 64bit	operating system
In-Reply-To: <4AAB83AD.6040103@voidspace.org.uk>
References: <d9dcd20c0909120404r7ebccc54j132e400e1fc2dbb5@mail.gmail.com>
	<4AAB83AD.6040103@voidspace.org.uk>
Message-ID: <4AAB9D51.9000808@gmail.com>

Michael Foord wrote:
> pywin32 has its own mailing list where you are more likely to get an 
> answer to your question:
> 
>    http://mail.python.org/mailman/listinfo/python-win32

And, just for accuracy's sake: that list isn't a list for
pywin32 stuff as such -- although it's obviously a good
place to ask. It's for Python development under Windows
in general. pywin32 has its own trackers shown here:

  http://sourceforge.net/projects/pywin32/support

Nonetheless, I certainly second the recommendation to post
to python-win32.

TJG

From dsdale24 at gmail.com  Sat Sep 12 15:57:58 2009
From: dsdale24 at gmail.com (Darren Dale)
Date: Sat, 12 Sep 2009 09:57:58 -0400
Subject: [Python-Dev] clarification on PEP 3124 status
Message-ID: <a08e5f80909120657h417ec1f7pc607582be4ec9e6c@mail.gmail.com>

Could somebody please clarify the status of PEP 3124? At
http://ftp.python.org/dev/peps/ , it is listed as "under
consideration", but http://ftp.python.org/dev/peps/pep-3124/ says it
has been deferred.

I was reading through the discussion on the python-3000 mailing list
archive, and at one point somebody asked for other examples where
generic functions are used in the community. The numpy project has a
basic generic function mechanism for numpy's ufuncs (regular functions
that operate on arrays), where subclasses of numpy.ndarray can define
__array_prepare__ (this method will be added in numpy-1.4) and
__array_wrap__ methods, which are sort of analogous to @before and
@after in PEP 3124 (ndarray subclasses define an __array_priority__
attribute to decide how to dispatch). The numpy approach is not a
general solution and is not as flexible as what is described in the
PEP, but it can be used by functions that operate on subclasses that
implement matrices, masked arrays, arrays with physical units.

I would be very interested in seeing a framework for generic functions
in the numpy standard library. I think would be more simple and
flexible than what we currently have. Is there still
interest/motivation for supporting generic functions in the standard
library?

Darren

From dsdale24 at gmail.com  Sat Sep 12 16:50:33 2009
From: dsdale24 at gmail.com (Darren Dale)
Date: Sat, 12 Sep 2009 10:50:33 -0400
Subject: [Python-Dev] clarification on PEP 3124 status
In-Reply-To: <a08e5f80909120657h417ec1f7pc607582be4ec9e6c@mail.gmail.com>
References: <a08e5f80909120657h417ec1f7pc607582be4ec9e6c@mail.gmail.com>
Message-ID: <a08e5f80909120750j35c6251chb89dc8df07901d37@mail.gmail.com>

On Sat, Sep 12, 2009 at 9:57 AM, Darren Dale <dsdale24 at gmail.com> wrote:
> I would be very interested in seeing a framework for generic functions
> in the numpy standard library.

Sorry, I meant to say "python standard library"

From martin at v.loewis.de  Sun Sep 13 08:29:42 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Sep 2009 08:29:42 +0200
Subject: [Python-Dev] clarification on PEP 3124 status
In-Reply-To: <a08e5f80909120657h417ec1f7pc607582be4ec9e6c@mail.gmail.com>
References: <a08e5f80909120657h417ec1f7pc607582be4ec9e6c@mail.gmail.com>
Message-ID: <4AAC9156.8070904@v.loewis.de>

> Could somebody please clarify the status of PEP 3124? At
> http://ftp.python.org/dev/peps/ , it is listed as "under
> consideration", but http://ftp.python.org/dev/peps/pep-3124/ says it
> has been deferred.

This isn't really contradictory. "under consideration" means "in
progress": it has neither been accepted or rejected.

If Phillip doesn't respond here, you may want to ask him directly.
My impression is that it is deferred because nobody is pursuing it
actively (including Phillip Eby). It's common for a PEP to be in that
state for several years, "deferred" then is an indication that readers
shouldn't expect a resolution in short term.

That said: my personal feeling is that this PEP is way too large, and
should be broken into seperate pieces of functionality that can be
considered independently. There is a lot of stuff in it that isn't
strictly necessary to provide the feature listed in the rationale.

Regards,
Martin


From dsdale24 at gmail.com  Sun Sep 13 16:14:32 2009
From: dsdale24 at gmail.com (Darren Dale)
Date: Sun, 13 Sep 2009 10:14:32 -0400
Subject: [Python-Dev] clarification on PEP 3124 status
In-Reply-To: <4AAC9156.8070904@v.loewis.de>
References: <a08e5f80909120657h417ec1f7pc607582be4ec9e6c@mail.gmail.com>
	<4AAC9156.8070904@v.loewis.de>
Message-ID: <a08e5f80909130714u5ea2097bqbd22902bdb93e3d1@mail.gmail.com>

Hi Martin,

On Sun, Sep 13, 2009 at 2:29 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Could somebody please clarify the status of PEP 3124? At
>> http://ftp.python.org/dev/peps/ , it is listed as "under
>> consideration", but http://ftp.python.org/dev/peps/pep-3124/ says it
>> has been deferred.
>
> This isn't really contradictory. "under consideration" means "in
> progress": it has neither been accepted or rejected.
>
> If Phillip doesn't respond here, you may want to ask him directly.
> My impression is that it is deferred because nobody is pursuing it
> actively (including Phillip Eby). It's common for a PEP to be in that
> state for several years, "deferred" then is an indication that readers
> shouldn't expect a resolution in short term.

That is why I asked, I wondered if it is being actively considered and
pursued, or if it had been deferred or worse abandoned.

> That said: my personal feeling is that this PEP is way too large, and
> should be broken into seperate pieces of functionality that can be
> considered independently. There is a lot of stuff in it that isn't
> strictly necessary to provide the feature listed in the rationale.

It would be nice to have a suitable foundation upon which more
elaborate third party dispatchers could build. The potential generic
functions have in a project like numpy are pretty exciting.

Darren

From p.f.moore at gmail.com  Sun Sep 13 16:54:20 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 13 Sep 2009 15:54:20 +0100
Subject: [Python-Dev] clarification on PEP 3124 status
In-Reply-To: <a08e5f80909130714u5ea2097bqbd22902bdb93e3d1@mail.gmail.com>
References: <a08e5f80909120657h417ec1f7pc607582be4ec9e6c@mail.gmail.com>
	<4AAC9156.8070904@v.loewis.de>
	<a08e5f80909130714u5ea2097bqbd22902bdb93e3d1@mail.gmail.com>
Message-ID: <79990c6b0909130754r76364b4es8daeec6592684f6e@mail.gmail.com>

2009/9/13 Darren Dale <dsdale24 at gmail.com>:
>> If Phillip doesn't respond here, you may want to ask him directly.
>> My impression is that it is deferred because nobody is pursuing it
>> actively (including Phillip Eby). It's common for a PEP to be in that
>> state for several years, "deferred" then is an indication that readers
>> shouldn't expect a resolution in short term.
>
> That is why I asked, I wondered if it is being actively considered and
> pursued, or if it had been deferred or worse abandoned.
>
>> That said: my personal feeling is that this PEP is way too large, and
>> should be broken into seperate pieces of functionality that can be
>> considered independently. There is a lot of stuff in it that isn't
>> strictly necessary to provide the feature listed in the rationale.
>
> It would be nice to have a suitable foundation upon which more
> elaborate third party dispatchers could build. The potential generic
> functions have in a project like numpy are pretty exciting.

You may also be interested in http://bugs.python.org/issue5135 which
is a (much) simpler attempt to introduce generic functions into the
standard library.

Generally, these things get stalled because the core developers don't
have sufficient interest in the topic to do anything directly, and the
arguments in favour aren't compelling enough to make a difference.
Maybe the benefits numpy would get would help the case.

Paul.

From dsdale24 at gmail.com  Sun Sep 13 18:27:01 2009
From: dsdale24 at gmail.com (Darren Dale)
Date: Sun, 13 Sep 2009 12:27:01 -0400
Subject: [Python-Dev] clarification on PEP 3124 status
In-Reply-To: <79990c6b0909130754r76364b4es8daeec6592684f6e@mail.gmail.com>
References: <a08e5f80909120657h417ec1f7pc607582be4ec9e6c@mail.gmail.com>
	<4AAC9156.8070904@v.loewis.de>
	<a08e5f80909130714u5ea2097bqbd22902bdb93e3d1@mail.gmail.com>
	<79990c6b0909130754r76364b4es8daeec6592684f6e@mail.gmail.com>
Message-ID: <a08e5f80909130927t19802cc4o28fe7f7ec38c855a@mail.gmail.com>

Hi Paul,

On Sun, Sep 13, 2009 at 10:54 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/9/13 Darren Dale <dsdale24 at gmail.com>:
>>> If Phillip doesn't respond here, you may want to ask him directly.
>>> My impression is that it is deferred because nobody is pursuing it
>>> actively (including Phillip Eby). It's common for a PEP to be in that
>>> state for several years, "deferred" then is an indication that readers
>>> shouldn't expect a resolution in short term.
>>
>> That is why I asked, I wondered if it is being actively considered and
>> pursued, or if it had been deferred or worse abandoned.
>>
>>> That said: my personal feeling is that this PEP is way too large, and
>>> should be broken into seperate pieces of functionality that can be
>>> considered independently. There is a lot of stuff in it that isn't
>>> strictly necessary to provide the feature listed in the rationale.
>>
>> It would be nice to have a suitable foundation upon which more
>> elaborate third party dispatchers could build. The potential generic
>> functions have in a project like numpy are pretty exciting.
>
> You may also be interested in http://bugs.python.org/issue5135 which
> is a (much) simpler attempt to introduce generic functions into the
> standard library.

Thanks for the pointer. I actually read through the discussion there
yesterday. I don't think simplegeneric would be especially useful to
numpy. For example, multiplying a numpy.array([1,2,3]) with a
quantities.Quantity([1,2,3], 'm/s') should produce a new Quantity
regardless of the order in which they are provided to
numpy.multiply(). Numpy can handle this particular example now, but
the mechanisms are a bit convoluted.

> Generally, these things get stalled because the core developers don't
> have sufficient interest in the topic to do anything directly, and the
> arguments in favour aren't compelling enough to make a difference.
> Maybe the benefits numpy would get would help the case.

I am a relatively new contributor to the numpy project, contributing
bug fixes and features (most of which have been related to -- or could
benefit from -- generic functions) to better support subclasses like
Quantity. Numpy has different kinds of arrays (ndarrays, array
scalars, masked arrays, matrices) and supports many different data
types (int8, float32, complex64, etc). The ability to dispatch based
on the object type (or combinations thereof), or on combinations of
data types, or perhaps on the units of quantities, seem like good
examples where predicative dispatch would be useful.

I am primarily trying to get up to speed to help with the effort to
transition numpy to python-3. Perhaps generic functions could help
make the numpy source code more accessible and maintainable, so that
maybe someday there would even be interest in including numpy or some
subset thereof in the standard library. Anyway, it is helpful to me to
see where generic functions stand and how they might develop in the
standard library as we work on numpy support for python 3.

Regards,
Darren

From pje at telecommunity.com  Sun Sep 13 20:28:58 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 13 Sep 2009 14:28:58 -0400
Subject: [Python-Dev] clarification on PEP 3124 status
In-Reply-To: <4AAC9156.8070904@v.loewis.de>
References: <a08e5f80909120657h417ec1f7pc607582be4ec9e6c@mail.gmail.com>
	<4AAC9156.8070904@v.loewis.de>
Message-ID: <20090913182855.F36BD3A403D@sparrow.telecommunity.com>

At 08:29 AM 9/13/2009 +0200, Martin v. L?wis wrote:
> > Could somebody please clarify the status of PEP 3124? At
> > http://ftp.python.org/dev/peps/ , it is listed as "under
> > consideration", but http://ftp.python.org/dev/peps/pep-3124/ says it
> > has been deferred.
>
>This isn't really contradictory. "under consideration" means "in
>progress": it has neither been accepted or rejected.
>
>If Phillip doesn't respond here, you may want to ask him directly.
>My impression is that it is deferred because nobody is pursuing it
>actively (including Phillip Eby). It's common for a PEP to be in that
>state for several years, "deferred" then is an indication that readers
>shouldn't expect a resolution in short term.
>
>That said: my personal feeling is that this PEP is way too large, and
>should be broken into seperate pieces of functionality that can be
>considered independently. There is a lot of stuff in it that isn't
>strictly necessary to provide the feature listed in the rationale.

It's deferred because the PEP needs a rewrite, and it isn't high on 
my priorities at the moment.  It's also unlikely the rewrite will 
happen before PEAK-Rules reaches a non-alpha release status.  (See 
http://ftp.python.org/dev/peps/pep-3124/#implementation-notes .)


From aahz at pythoncraft.com  Mon Sep 14 16:16:27 2009
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 14 Sep 2009 07:16:27 -0700
Subject: [Python-Dev] FWD: Windows 7 Compatibility
Message-ID: <20090914141627.GA29347@panix.com>

Still no-mail on python-dev, forwarding this because it has a direct
e-mail contact for Microsoft at the bottom.

----- Forwarded message from Joanna Cobb <joanna.cobb at nichecubed.com> -----

> From: Joanna Cobb <joanna.cobb at nichecubed.com>
> To: "webmaster at python.org" <webmaster at python.org>
> Date: Mon, 14 Sep 2009 10:05:34 -0400
> Subject: Windows 7 Compatibility
> 
> Regarding:  Windows 7 Compatibility for Python Software Foundation - Application:  Python
> 
> I am trying to contact your company regarding the Microsoft Windows 7 Compatibility Program for Python.  I have not been able to get in touch with the person responsible for this application in your company and this is why I am reaching out to you through the Support Team. This application has been identified as one of the applications Microsoft would like to see supported on Windows 7 and I have been tasked by Microsoft to help answer your questions about Windows 7 application compatibility and help you get your application through the Windows 7 "Green Light" compatibility process.
> 
> If your application already supports Windows Vista, chances are it will already be compatible with Windows 7 without the need for any code changes.  By pledging support for Windows 7 you're application will automatically be listed in the Windows Application Compatibility seen currently by more than 1 million users per month. The registration is extremely simple and just asks a few key questions.
> 
> Here is the link to Microsoft's ISV Application Compatibility page: http://partner.microsoft.com/isvappcompat.  When you have a moment, I would encourage you to visit the site and complete the process to pledge support for your application on Windows 7 by October 22nd 2009 when Windows 7 is officially released.
> 
> In addition if you are able to pledge compatibility you'll receive access to a special Windows 7 Partner Marketing Kit  that includes a press release with a Microsoft quote, plus customizable marketing templates including; email templates, postcards, web banners, business letter, and copy blocks, all to identify to your customers, or potential customers that your solutions are compatible with Windows 7.
> 
> If you provide me with a phone number where to get in touch with you, I will call you to answer any questions you may have.
> 
> Once you register on the ISV Application Compatibility site, I would appreciate it if you would email me to let me know that you have completed so that I can make a note of it for Microsoft. If you register the application under a different partner or application name please let me know in order to track changes. If there is a new version of the application and there are no plans to support Windows 7 on the older version please register the older version as "No planned Support" on the site as well as the new version with desired Win7 compatibility date.
> 
> Should you have any questions about this email feel free to call me or send an email to my supervisor at  v-mafl at microsoft.com<mailto:v-mafl at microsoft.com>.
> 
> Best regards,
> 
> Joanna Cobb
> Business Development Representative
> N(3)
> niche cubed
> Office:  800.508.4291 Ext. 316
> joanna.cobb at nichecubed.com<mailto:first.last at nichecubed.com>
> www.nichecubed.com<http://www.nichecubed.com>
> 
> Confidentiality note: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named in the e-mail.  If the reader of the e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading it is strictly prohibited.  If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.  Thank you.
> 

----- End forwarded message -----

-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It's 106 miles to Chicago.  We have a full tank of gas, a half-pack of
cigarettes, it's dark, and we're wearing sunglasses."  "Hit it."

From peter at hda3.com  Mon Sep 14 18:44:12 2009
From: peter at hda3.com (Peter Moody)
Date: Mon, 14 Sep 2009 09:44:12 -0700
Subject: [Python-Dev] PEP 3144 review.
Message-ID: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>

Folks, Guido,

I believe PEP 3144 is ready for your review.  When you get a chance,
can you take a look/make a pronouncement?

Cheers,
/peter

From guido at python.org  Mon Sep 14 18:54:41 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Sep 2009 09:54:41 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
Message-ID: <ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>

What's the opinion of the other interested party or parties? I don't
want a repeat of the events last time, where we had to pull it at the
last time because there hadn't been enough discussion.

On Mon, Sep 14, 2009 at 9:44 AM, Peter Moody <peter at hda3.com> wrote:
> Folks, Guido,
>
> I believe PEP 3144 is ready for your review. ?When you get a chance,
> can you take a look/make a pronouncement?
>
> Cheers,
> /peter
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From andrewm at object-craft.com.au  Tue Sep 15 04:11:30 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Tue, 15 Sep 2009 12:11:30 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
Message-ID: <20090915021130.4E57A600115@longblack.object-craft.com.au>

>I believe PEP 3144 is ready for your review.  When you get a chance,
>can you take a look/make a pronouncement?

In my experience it is common to leave out the masked octets when
referring to an IPv4 network (the octets are assumed to be zero), so I
don't agree with this behaviour from the reference implementation:

    >>> ipaddr.IPv4Network('10/8')
    IPv4Network('0.0.0.10/8')
    >>> ipaddr.IPv4Network('192.168/16')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/src/py/ipaddr/ipaddr.py", line 1246, in __init__
        raise IPv4IpValidationError(addr[0])
    ipaddr.IPv4IpValidationError: '192.168' is not a valid IPv4 address

I also couldn't see an easy way to get from a network address to the
containing network. For example:

    >>> ipaddr.IPv4Network('192.168.1.1/16')
    IPv4Network('192.168.1.1/16')

This is close:

    >>> ipaddr.IPv4Network('192.168.1.1/16').network
    IPv4Address('192.168.0.0')

What I want is a method that returns:

    IPv4Network('192.168.0.0/16')

I appreciate these requests are somewhat contradictory (one calls
for masked octets to be insignificant, the other calls for them to be
significant), but they are both valid use cases in my experience.

Apologies if these have already been covered in prior discussion -
I've tried to keep up, but I haven't been able to give it the attention
it deserves.

I also note that many methods in the reference implementation are not
discussed in the PEP. While I don't consider this a problem for the PEP,
anyone reviewing the module for inclusion in the standard lib needs to 
consider them.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From peter at hda3.com  Tue Sep 15 08:02:26 2009
From: peter at hda3.com (Peter Moody)
Date: Mon, 14 Sep 2009 23:02:26 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090915021130.4E57A600115@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
Message-ID: <8517e9350909142302qcb2e13dyf857874649d3b4c6@mail.gmail.com>

On Mon, Sep 14, 2009 at 7:11 PM, Andrew McNamara
<andrewm at object-craft.com.au> wrote:
>>I believe PEP 3144 is ready for your review. ?When you get a chance,
>>can you take a look/make a pronouncement?
>
> In my experience it is common to leave out the masked octets when
> referring to an IPv4 network (the octets are assumed to be zero), so I
> don't agree with this behaviour from the reference implementation:

huh, this appears to be a bug. filing an issue and I'll have this
fixed before anything is submitted
(http://code.google.com/p/ipaddr-py/issues/detail?id=35)

> ? ?>>> ipaddr.IPv4Network('10/8')
> ? ?IPv4Network('0.0.0.10/8')
> ? ?>>> ipaddr.IPv4Network('192.168/16')
> ? ?Traceback (most recent call last):
> ? ? ?File "<stdin>", line 1, in <module>
> ? ? ?File "/usr/src/py/ipaddr/ipaddr.py", line 1246, in __init__
> ? ? ? ?raise IPv4IpValidationError(addr[0])
> ? ?ipaddr.IPv4IpValidationError: '192.168' is not a valid IPv4 address
>
> I also couldn't see an easy way to get from a network address to the
> containing network. For example:
>
> ? ?>>> ipaddr.IPv4Network('192.168.1.1/16')
> ? ?IPv4Network('192.168.1.1/16')
>
> This is close:
>
> ? ?>>> ipaddr.IPv4Network('192.168.1.1/16').network
> ? ?IPv4Address('192.168.0.0')
>
> What I want is a method that returns:
>
> ? ?IPv4Network('192.168.0.0/16')

I can see about adding this. you can currently do:

>>> a = ipaddr.IPv4Network('192.168.1.1/16')
>>> '%s/%s' % (a.network, a.prefixlen)
192.168.0.0/16

(I do something very similar to this in address_exclude)

> I appreciate these requests are somewhat contradictory (one calls
> for masked octets to be insignificant, the other calls for them to be
> significant), but they are both valid use cases in my experience.
>
> Apologies if these have already been covered in prior discussion -
> I've tried to keep up, but I haven't been able to give it the attention
> it deserves.

no need to apologize, all comments welcome.

> I also note that many methods in the reference implementation are not
> discussed in the PEP. While I don't consider this a problem for the PEP,
> anyone reviewing the module for inclusion in the standard lib needs to
> consider them.

yeah, I didn't exactly want the PEP to be the pydoc of ipaddr, but I
did want to explain the features and show how I thought they were
important to whatever might be accepted by python. I can certainly go
into much more detail in the PEP if that's deemed important.

Cheers,
/peter

> --
> Andrew McNamara, Senior Developer, Object Craft
> http://www.object-craft.com.au/
>

From lie.1296 at gmail.com  Tue Sep 15 08:15:48 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Tue, 15 Sep 2009 16:15:48 +1000
Subject: [Python-Dev] Numeric alignment issue with PEP 3101
In-Reply-To: <6930F234-6E8E-4E78-AE7F-FFE97AF72D5C@rcn.com>
References: <4AA57F42.3050201@trueblade.com>
	<6930F234-6E8E-4E78-AE7F-FFE97AF72D5C@rcn.com>
Message-ID: <h8nbev$iqn$1@ger.gmane.org>

Raymond Hettinger wrote:
> I concur.  Numbers are naturally right aligned.
> 
> 

Isn't numbers are "naturally right aligned" because of the Big Endian 
notations that most mathematicians currently use. Had we been using 
Little Endian notation, numbers would be naturally left-aligned, 
wouldn't they?

And I sort of disagree with saying it's naturally left- or right- 
aligned; as numbers are naturally (decimal) dot-aligned (if you use dot 
to separate the whole and fractional part of your number).


From python at rcn.com  Tue Sep 15 08:20:35 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 14 Sep 2009 23:20:35 -0700
Subject: [Python-Dev] Numeric alignment issue with PEP 3101
References: <4AA57F42.3050201@trueblade.com><6930F234-6E8E-4E78-AE7F-FFE97AF72D5C@rcn.com>
	<h8nbev$iqn$1@ger.gmane.org>
Message-ID: <24D0205DCEB74A7CB21EF07D2F0B3B65@RaymondLaptop1>


> And I sort of disagree with saying it's naturally left- or right- 
> aligned; as numbers are naturally (decimal) dot-aligned (if you use dot 
> to separate the whole and fractional part of your number).


How about, "naturally aligned by place value".


Raymond

From ronaldoussoren at mac.com  Tue Sep 15 15:53:14 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 15 Sep 2009 15:53:14 +0200
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <nad-C736AE.09230710092009@ger.gmane.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-C736AE.09230710092009@ger.gmane.org>
Message-ID: <F10025B7-8AEA-4FD2-88C4-231F8BD77458@mac.com>


On 10 Sep, 2009, at 18:23, Ned Deily wrote:

> In article <9D506035-7C2D-4929-A134-E88EEB7B7D9E at python.org>,
> Barry Warsaw <barry at python.org> wrote:
>
>> On Sep 9, 2009, at 1:29 PM, Ned Deily wrote:
>>
>>> In article <11A6545D-7204-4F61-B55B-1CC77CB5645E at python.org>,
>>> Barry Warsaw <barry at python.org> wrote:
>>>> I still want to release by the 25th, but I'd be willing to move the
>>>> rc
>>>> to Monday the 21st.  We're really just trying to avoid a brown bag
>>>> moment, so that should give us enough time to double check the
>>>> releases.
>>>
>>> The recent release of OS X 10.6 (Snow Leopard) has triggered a fair
>>> amount of 2.6 bug tracker activity, since 10.6 now includes 2.6
>>> (2.6.1)
>>> and a 64-bit version at that.   A number of patches have either just
>>> been checked-in over the past couple of weeks or are getting some
>>> exposure before check-in.  Given the timing and the (appropriate)
>>> infrequency of 2.6.x releases, I think it would be unfortunate to  
>>> push
>>> 2.6.3 out the door without ensuring that it works well on 10.6.
>>> Therefore, I propose that 2.6.3 should have 10.6 compatibility as a
>>> "release goal".
>>>
>>> Without trying to put Ronald on the spot (too much!), it would be a
>>> good
>>> idea to get his assessment where things stand wrt 2.6 on 10.6 before
>>> setting a final release date.
>>
>> I'm hoping that Python won't have any issues building and running on
>> 10.6, but I don't have it yet so I can't personally test it out.
>>
>> How would you quantify "works well"?  Do you have any thoughts on
>> tests you'd run other than the standard test suite?  If 2.6.3 is  
>> shown
>> to pass its test suite on 10.5.x, is that good enough?  Are the
>> specific bug fixes necessary for 10.6?
>
> Running the standard test suite on 10.6 and seeing no regressions
> compared to the same suite on 10.5.x seems a reasonable necessary
> requirement.  We have the resources to do that.  Beyond that, as  
> Ronald
> suggests, I think it important to go through the open issues in the  
> next
> couple of days and identify and flag any potential release-blockers
> (besides the IDLE problem already mentioned).

The IDLE issue is IMHO a release blocker, as is issue 6851.

>
> One other open issue is 64-bit support in the python.org OS X  
> installer.
> There have been discussions and requests in the past and, with Apple
> providing 64-bit out of the box in 10.6, it seems like it's time to
> provide something on python.org as well.   One option: continue to
> provide a 32-bit only installer for ppc and i386 for 10.3.9 and beyond
> and add a second installer image with 3-way (ppc, i386, x86_64 but no
> ppc64) 32/64 for 10.5 and beyond.   Ronald, is that your current
> thinking?

64-bit support can wait until after 2.6.3 is released. I need time to  
work out what's needed go create a good installer (and not just  
running the current build-installer.py script because that includes to  
much for a binary that doesn't run on 10.3.9). That won't happen  
before 2.6.3 is released because I'm too thinly stretched even without  
working on that.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090915/47226121/attachment.bin>

From solipsis at pitrou.net  Tue Sep 15 16:28:13 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 14:28:13 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
Message-ID: <loom.20090915T162717-42@post.gmane.org>

Andrew McNamara <andrewm <at> object-craft.com.au> writes:
> 
>     >>> ipaddr.IPv4Network('192.168.1.1/16').network
>     IPv4Address('192.168.0.0')

Er, does this mean that taking the `network` attribute from a network object
actually gives an address object (not a network)?
It looks horribly misleading to me.




From rdmurray at bitdance.com  Tue Sep 15 16:36:08 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 10:36:08 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090915T162717-42@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 14:28, Antoine Pitrou wrote:
> Andrew McNamara <andrewm <at> object-craft.com.au> writes:
>>
>>    >>> ipaddr.IPv4Network('192.168.1.1/16').network
>>     IPv4Address('192.168.0.0')
>
> Er, does this mean that taking the `network` attribute from a network object
> actually gives an address object (not a network)?
> It looks horribly misleading to me.

That was my opinion, too.  I've always called that number the 'zero'
of the network, but other people said that it is usually called the
'network'.

I also find it odd to see a 'network' with an IP address of 192.168.1.1.
That looks like a host-address-plus-netmask to me, not a network
address.

--David

From eric.pruitt at gmail.com  Tue Sep 15 18:25:35 2009
From: eric.pruitt at gmail.com (Eric Pruitt)
Date: Tue, 15 Sep 2009 12:25:35 -0400
Subject: [Python-Dev] PEP 3145 (With Contents)
In-Reply-To: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com>
References: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com>
Message-ID: <171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com>

I'm bumping this PEP again in hopes of getting some feedback.

Thanks,
Eric

On Tue, Sep 8, 2009 at 23:52, Eric Pruitt <eric.pruitt at gmail.com> wrote:
> PEP: 3145
> Title: Asynchronous I/O For subprocess.Popen
> Author: (James) Eric Pruitt, Charles R. McCreary, Josiah Carlson
> Type: Standards Track
> Content-Type: text/plain
> Created: 04-Aug-2009
> Python-Version: 3.2
>
> Abstract:
>
> ? ?In its present form, the subprocess.Popen implementation is prone to
> ? ?dead-locking and blocking of the parent Python script while waiting on data
> ? ?from the child process.
>
> Motivation:
>
> ? ?A search for "python asynchronous subprocess" will turn up numerous
> ? ?accounts of people wanting to execute a child process and communicate with
> ? ?it from time to time reading only the data that is available instead of
> ? ?blocking to wait for the program to produce data [1] [2] [3]. ?The current
> ? ?behavior of the subprocess module is that when a user sends or receives
> ? ?data via the stdin, stderr and stdout file objects, dead locks are common
> ? ?and documented [4] [5]. ?While communicate can be used to alleviate some of
> ? ?the buffering issues, it will still cause the parent process to block while
> ? ?attempting to read data when none is available to be read from the child
> ? ?process.
>
> Rationale:
>
> ? ?There is a documented need for asynchronous, non-blocking functionality in
> ? ?subprocess.Popen [6] [7] [2] [3]. ?Inclusion of the code would improve the
> ? ?utility of the Python standard library that can be used on Unix based and
> ? ?Windows builds of Python. ?Practically every I/O object in Python has a
> ? ?file-like wrapper of some sort. ?Sockets already act as such and for
> ? ?strings there is StringIO. ?Popen can be made to act like a file by simply
> ? ?using the methods attached the the subprocess.Popen.stderr, stdout and
> ? ?stdin file-like objects. ?But when using the read and write methods of
> ? ?those options, you do not have the benefit of asynchronous I/O. ?In the
> ? ?proposed solution the wrapper wraps the asynchronous methods to mimic a
> ? ?file object.
>
> Reference Implementation:
>
> ? ?I have been maintaining a Google Code repository that contains all of my
> ? ?changes including tests and documentation [9] as well as blog detailing
> ? ?the problems I have come across in the development process [10].
>
> ? ?I have been working on implementing non-blocking asynchronous I/O in the
> ? ?subprocess.Popen module as well as a wrapper class for subprocess.Popen
> ? ?that makes it so that an executed process can take the place of a file by
> ? ?duplicating all of the methods and attributes that file objects have.
>
> ? ?There are two base functions that have been added to the subprocess.Popen
> ? ?class: Popen.send and Popen._recv, each with two separate implementations,
> ? ?one for Windows and one for Unix based systems. ?The Windows
> ? ?implementation uses ctypes to access the functions needed to control pipes
> ? ?in the kernel 32 DLL in an asynchronous manner. ?On Unix based systems,
> ? ?the Python interface for file control serves the same purpose. ?The
> ? ?different implementations of Popen.send and Popen._recv have identical
> ? ?arguments to make code that uses these functions work across multiple
> ? ?platforms.
>
> ? ?When calling the Popen._recv function, it requires the pipe name be
> ? ?passed as an argument so there exists the Popen.recv function that passes
> ? ?selects stdout as the pipe for Popen._recv by default. ?Popen.recv_err
> ? ?selects stderr as the pipe by default. "Popen.recv" and "Popen.recv_err"
> ? ?are much easier to read and understand than "Popen._recv('stdout' ..." and
> ? ?"Popen._recv('stderr' ..." respectively.
>
> ? ?Since the Popen._recv function does not wait on data to be produced
> ? ?before returning a value, it may return empty bytes. ?Popen.asyncread
> ? ?handles this issue by returning all data read over a given time
> ? ?interval.
>
> ? ?The ProcessIOWrapper class uses the asyncread and asyncwrite functions to
> ? ?allow a process to act like a file so that there are no blocking issues
> ? ?that can arise from using the stdout and stdin file objects produced from
> ? ?a subprocess.Popen call.
>
>
> References:
>
> ? ?[1] [ python-Feature Requests-1191964 ] asynchronous Subprocess
> ? ? ? ?http://mail.python.org/pipermail/python-bugs-list/2006-December/
> ? ? ? ? ?036524.html
>
> ? ?[2] Daily Life in an Ivory Basement : /feb-07/problems-with-subprocess
> ? ? ? ?http://ivory.idyll.org/blog/feb-07/problems-with-subprocess
>
> ? ?[3] How can I run an external command asynchronously from Python? - Stack
> ? ? ? ?Overflow
> ? ? ? ?http://stackoverflow.com/questions/636561/how-can-i-run-an-external-
> ? ? ? ? ?command-asynchronously-from-python
>
> ? ?[4] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation
> ? ? ? ?http://docs.python.org/library/subprocess.html#subprocess.Popen.wait
>
> ? ?[5] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation
> ? ? ? ?http://docs.python.org/library/subprocess.html#subprocess.Popen.kill
>
> ? ?[6] Issue 1191964: asynchronous Subprocess - Python tracker
> ? ? ? ?http://bugs.python.org/issue1191964
>
> ? ?[7] Module to allow Asynchronous subprocess use on Windows and Posix
> ? ? ? ?platforms - ActiveState Code
> ? ? ? ?http://code.activestate.com/recipes/440554/
>
> ? ?[8] subprocess.rst - subprocdev - Project Hosting on Google Code
> ? ? ? ?http://code.google.com/p/subprocdev/source/browse/doc/subprocess.rst?spec=svn2c925e935cad0166d5da85e37c742d8e7f609de5&r=2c925e935cad0166d5da85e37c742d8e7f609de5#437
>
> ? ?[9] subprocdev - Project Hosting on Google Code
> ? ? ? ?http://code.google.com/p/subprocdev
>
> ? ?[10] Python Subprocess Dev
> ? ? ? ? http://subdev.blogspot.com/
>
> Copyright:
>
> ? ?This P.E.P. is licensed under the Open Publication License;
> ? ?http://www.opencontent.org/openpub/.

From phd at phd.pp.ru  Tue Sep 15 18:56:54 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Tue, 15 Sep 2009 20:56:54 +0400
Subject: [Python-Dev] PEP 3145 (With Contents)
In-Reply-To: <171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com>
References: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com>
	<171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com>
Message-ID: <20090915165654.GA27798@phd.pp.ru>

On Tue, Sep 15, 2009 at 12:25:35PM -0400, Eric Pruitt wrote:
> A search for "python asynchronous subprocess" will turn up numerous
> accounts of people

   IMHO there is no need to refer to a search. It'd be enough to say "There
are many people...".

> kernel 32 DLL

   Why not just name it kernel32.dll?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From scott+python-dev at scottdial.com  Tue Sep 15 19:16:06 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 15 Sep 2009 13:16:06 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<20090915021130.4E57A600115@longblack.object-craft.com.au>	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
Message-ID: <4AAFCBD6.5090104@scottdial.com>

R. David Murray wrote:
> On Tue, 15 Sep 2009 at 14:28, Antoine Pitrou wrote:
>> Andrew McNamara <andrewm <at> object-craft.com.au> writes:
>>>
>>>    >>> ipaddr.IPv4Network('192.168.1.1/16').network
>>>     IPv4Address('192.168.0.0')
>>
>> Er, does this mean that taking the `network` attribute from a network
>> object
>> actually gives an address object (not a network)?
>> It looks horribly misleading to me.
> 
> That was my opinion, too.  I've always called that number the 'zero'
> of the network, but other people said that it is usually called the
> 'network'.
> 
> I also find it odd to see a 'network' with an IP address of 192.168.1.1.
> That looks like a host-address-plus-netmask to me, not a network
> address.
> 

It just happened that I needed a tool today to calculate the gateway IP
for an interface, and I took it as an opportunity to try out this ipaddr
module. I'll attempt to relate my experience below...

I have to concur with the opinions above. I was very confused by the
following error:

>>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0")
...
ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not
a valid address (hint, it's probably a network)

Because, it *is* a address of a host on a network. I gave in and said:

>>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0")

But then, I was dumbfounded as to how I could get the gateway IP from
this IPNetwork object. It took me a while to figure out that you can
iterate over IPNetwork instances:

>>> gateway = net[1]

I was then confused, because:

>>> print(type(gateway))
<class 'ipaddr.IPv4Address'>

Which sorta blew my mind.. I fully expected to receive an IPNetwork back
from that operation. It is unclear to me why the network information
gets chucked by that operation. I foresee having to work around that in
real applications by doing something obnoxious like:

>>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask))

But since I only actually needed the IP address for the gateway, it
fulfilled my needs.

In the end, I found the names IPNetwork/IPAddress and their
instantiations confusing. ISTM that IPNetwork is overloaded and plays
two roles of being an IPNetwork and being an IPAddressWithNetwork. And
finally, it's unclear to me why iterating over a IPNetwork would not
produce a sequence of IPNetwork(/IPAddressWithNetwork) objects.

ISTM that if I started with an IPNetwork object, the API should always
return IPNetwork objects. If I want just an IPAddress object, then I can
always fetch the "ip" attribute to get that. Perhaps there is some
compelling conceptual argument as to why this is not correct, but it
seems like the API destroys information needlessly.

Just my opinion..

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From mcguire at google.com  Tue Sep 15 19:16:29 2009
From: mcguire at google.com (Jake McGuire)
Date: Tue, 15 Sep 2009 10:16:29 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
Message-ID: <77c780b40909151016n4675292craab781d115723383@mail.gmail.com>

On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum <guido at python.org> wrote:

> What's the opinion of the other interested party or parties? I don't
> want a repeat of the events last time, where we had to pull it at the
> last time because there hadn't been enough discussion.


How many other people are using this library?  I think it's hard to give
really useful feedback on an API without using it for some non-trivial task,
but maybe other people don't have this problem.

-jake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090915/43b8d4f5/attachment.htm>

From peter at hda3.com  Tue Sep 15 19:36:13 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 15 Sep 2009 10:36:13 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
Message-ID: <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>

On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire <mcguire at google.com> wrote:
> On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum <guido at python.org> wrote:
>>
>> What's the opinion of the other interested party or parties? I don't
>> want a repeat of the events last time, where we had to pull it at the
>> last time because there hadn't been enough discussion.
>
> How many other people are using this library? ?I think it's hard to give
> really useful feedback on an API without using it for some non-trivial task,
> but maybe other people don't have this problem.
> -jake

188 (check that, 190) people have downloaded the 2.0 release in the
last week (numbers publicly available from the code.google.com). I
can't tell you how many (if any) have downloaded it via svn.

Cheers,
/peter

From peter at hda3.com  Tue Sep 15 19:56:35 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 15 Sep 2009 10:56:35 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AAFCBD6.5090104@scottdial.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
Message-ID: <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>

On Tue, Sep 15, 2009 at 10:16 AM, Scott Dial
<scott+python-dev at scottdial.com> wrote:
> R. David Murray wrote:
>> On Tue, 15 Sep 2009 at 14:28, Antoine Pitrou wrote:
>>> Andrew McNamara <andrewm <at> object-craft.com.au> writes:
>>>>
>>>> ? ?>>> ipaddr.IPv4Network('192.168.1.1/16').network
>>>> ? ? IPv4Address('192.168.0.0')
>>>
>>> Er, does this mean that taking the `network` attribute from a network
>>> object
>>> actually gives an address object (not a network)?
>>> It looks horribly misleading to me.
>>
>> That was my opinion, too. ?I've always called that number the 'zero'
>> of the network, but other people said that it is usually called the
>> 'network'.
>>
>> I also find it odd to see a 'network' with an IP address of 192.168.1.1.
>> That looks like a host-address-plus-netmask to me, not a network
>> address.
>>
>
> It just happened that I needed a tool today to calculate the gateway IP
> for an interface, and I took it as an opportunity to try out this ipaddr
> module. I'll attempt to relate my experience below...
>
> I have to concur with the opinions above. I was very confused by the
> following error:
>
>>>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0")
> ...
> ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not
> a valid address (hint, it's probably a network)

are you actually getting '98.223.189.24/255.255.240.0' back? that
doesn't look right.

> Because, it *is* a address of a host on a network. I gave in and said:
>
>>>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0")
>
> But then, I was dumbfounded as to how I could get the gateway IP from
> this IPNetwork object. It took me a while to figure out that you can
> iterate over IPNetwork instances:
>
>>>> gateway = net[1]
>
> I was then confused, because:
>
>>>> print(type(gateway))
> <class 'ipaddr.IPv4Address'>
>
> Which sorta blew my mind.. I fully expected to receive an IPNetwork back
> from that operation. It is unclear to me why the network information
> gets chucked by that operation. I foresee having to work around that in
> real applications by doing something obnoxious like:
>
>>>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask))
>
> But since I only actually needed the IP address for the gateway, it
> fulfilled my needs.
>
> In the end, I found the names IPNetwork/IPAddress and their
> instantiations confusing. ISTM that IPNetwork is overloaded and plays
> two roles of being an IPNetwork and being an IPAddressWithNetwork. And
> finally, it's unclear to me why iterating over a IPNetwork would not
> produce a sequence of IPNetwork(/IPAddressWithNetwork) objects.

What you're describing, at least for the networks, is basically what
ipaddr was.  It seemed, after much heated discussion, that the
community felt that

ipaddr.IPv4Network('1.1.1.0/24')[0]

was actually an individual address. ie, each of the addresses in
1.1.1.0/24 are individual addresses, rather than networks with /32
prefix lengths.

having said that, I understand your confusion; I spent a year+
treating addresses exactly as you've mentioned. It didn't seem as
though the majority of the community felt the same, though, so ipaddr
evolved.

Cheers,
/peter

> ISTM that if I started with an IPNetwork object, the API should always
> return IPNetwork objects. If I want just an IPAddress object, then I can
> always fetch the "ip" attribute to get that. Perhaps there is some
> compelling conceptual argument as to why this is not correct, but it
> seems like the API destroys information needlessly.
>
> Just my opinion..
>
> --
> Scott Dial
> scott at scottdial.com
> scodial at cs.indiana.edu
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From solipsis at pitrou.net  Tue Sep 15 20:07:23 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 18:07:23 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<20090915021130.4E57A600115@longblack.object-craft.com.au>	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
Message-ID: <loom.20090915T200602-36@post.gmane.org>

Scott Dial <scott+python-dev <at> scottdial.com> writes:
> 
> >>> gateway = net[1]
> 
> I was then confused, because:
> 
> >>> print(type(gateway))
> <class 'ipaddr.IPv4Address'>
> 
> Which sorta blew my mind.. I fully expected to receive an IPNetwork back
> from that operation.

Speaking as a non-network specialist, it actually looks logical to me to be
given an address if I iterate over a network (the same way that, if I iterate on
a list, I get individual elements, not 1-element sublists).

I don't understand why you think the network mask should be part of an address.
An address is just an address, it specifies the location of a host, not the
configuration of the network surrounding it. If you look at an IP header, the
destination address doesn't specify a netmask.



From scott+python-dev at scottdial.com  Tue Sep 15 20:16:45 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 15 Sep 2009 14:16:45 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<20090915021130.4E57A600115@longblack.object-craft.com.au>	<loom.20090915T162717-42@post.gmane.org>	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>	<4AAFCBD6.5090104@scottdial.com>
	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
Message-ID: <4AAFDA0D.5040602@scottdial.com>

Peter Moody wrote:
> On Tue, Sep 15, 2009 at 10:16 AM, Scott Dial
> <scott+python-dev at scottdial.com> wrote:
>> In the end, I found the names IPNetwork/IPAddress and their
>> instantiations confusing. ISTM that IPNetwork is overloaded and plays
>> two roles of being an IPNetwork and being an IPAddressWithNetwork. And
>> finally, it's unclear to me why iterating over a IPNetwork would not
>> produce a sequence of IPNetwork(/IPAddressWithNetwork) objects.
> 
> What you're describing, at least for the networks, is basically what
> ipaddr was.  It seemed, after much heated discussion, that the
> community felt that
> 
> ipaddr.IPv4Network('1.1.1.0/24')[0]
> 
> was actually an individual address. ie, each of the addresses in
> 1.1.1.0/24 are individual addresses, rather than networks with /32
> prefix lengths.
> 

For clarity, I am going to call the current design "A":

ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Address('1.1.1.0')

And what itt sounds like what you are describing as the old behavior is
this (design "B"):

ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Network('1.1.1.0/32')

Which is different than what I was trying to describe. I expected
(design "C"):

ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Network('1.1.1.0/24')

My summarization of these designs would be that "B" is wrong. And that
"A" is better than "B", certainly. At least "A" is not saying something
untrue. However, "C" would be truthful and retains a superset of
information that "A" provides (the "ip" attribute of IPNetwork).

In other words, I don't see why obtaining a host address would *not*
retain the hostmask from the network it was obtained from. I am not
disagreeing with it being an individual address. I am disagreeing that
IPNetwork itself already does represent individual addresses (hence my
aliasing it with IPAddressWithNetwork). And wrt, the logical return
would be another IPAddressWithNetwork retaining the same mask.

--
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From solipsis at pitrou.net  Tue Sep 15 20:23:20 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 18:23:20 +0000 (UTC)
Subject: [Python-Dev] PEP 3145 (With Contents)
References: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com>
	<171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com>
Message-ID: <h8om2o$1mi$1@ger.gmane.org>


Hello,

I would like to know if your approach is based on Python 2.x or 3.x.
Python 3.x has new API provisions, in the I/O layer, for non-blocking I/O 
and it would be nice if your work could fit in that framework.

> >    Popen can be made to act like a file by simply
> >    using the methods attached the the subprocess.Popen.stderr, stdout 
and
> >    stdin file-like objects.  But when using the read and write 
methods of
> >    those options, you do not have the benefit of asynchronous I/O.

I'm not sure I understand the latter sentence. Do you imply that, with 
your work, read() and write() do allow you to benefit from async I/O? If 
so, how?

Another question: what mechanism does it use internally? Is this mechanism
accessible from the outside, so that people can e.g. integrate this 
inside a third-party event loop (Twisted, asyncore or whatever else)?

The PEP should probably outline the additional APIs a bit more precisely 
and formally than it currently does.

> >    The Windows
> >    implementation uses ctypes to access the functions needed to 
control pipes
> >    in the kernel 32 DLL in an asynchronous manner.

Sorry for the naive question (I'm not a Windows specialist), but does the
allusion to "kernel32.dll" mean that it doesn't function on 64-bit 
variants?

Thanks for your work,

Regards

Antoine.



From exarkun at twistedmatrix.com  Tue Sep 15 20:24:56 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Tue, 15 Sep 2009 18:24:56 -0000
Subject: [Python-Dev] PEP 3145 (With Contents)
In-Reply-To: <171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com>
References: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com>
	<171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com>
Message-ID: <20090915182456.12215.1594428165.divmod.xquotient.17@localhost.localdomain>

On 04:25 pm, eric.pruitt at gmail.com wrote:
>I'm bumping this PEP again in hopes of getting some feedback.
>
>Thanks,
>Eric
>
>On Tue, Sep 8, 2009 at 23:52, Eric Pruitt <eric.pruitt at gmail.com> 
>wrote:
>>PEP: 3145
>>Title: Asynchronous I/O For subprocess.Popen
>>Author: (James) Eric Pruitt, Charles R. McCreary, Josiah Carlson
>>Type: Standards Track
>>Content-Type: text/plain
>>Created: 04-Aug-2009
>>Python-Version: 3.2
>>
>>Abstract:
>>
>>? ?In its present form, the subprocess.Popen implementation is prone 
>>to
>>? ?dead-locking and blocking of the parent Python script while waiting 
>>on data
>>? ?from the child process.
>>
>>Motivation:
>>
>>? ?A search for "python asynchronous subprocess" will turn up numerous
>>? ?accounts of people wanting to execute a child process and 
>>communicate with
>>? ?it from time to time reading only the data that is available 
>>instead of
>>? ?blocking to wait for the program to produce data [1] [2] [3]. ?The 
>>current
>>? ?behavior of the subprocess module is that when a user sends or 
>>receives
>>? ?data via the stdin, stderr and stdout file objects, dead locks are 
>>common
>>? ?and documented [4] [5]. ?While communicate can be used to alleviate 
>>some of
>>? ?the buffering issues, it will still cause the parent process to 
>>block while
>>? ?attempting to read data when none is available to be read from the 
>>child
>>? ?process.
>>
>>Rationale:
>>
>>? ?There is a documented need for asynchronous, non-blocking 
>>functionality in
>>? ?subprocess.Popen [6] [7] [2] [3]. ?Inclusion of the code would 
>>improve the
>>? ?utility of the Python standard library that can be used on Unix 
>>based and
>>? ?Windows builds of Python. ?Practically every I/O object in Python 
>>has a
>>? ?file-like wrapper of some sort. ?Sockets already act as such and 
>>for
>>? ?strings there is StringIO. ?Popen can be made to act like a file by 
>>simply
>>? ?using the methods attached the the subprocess.Popen.stderr, stdout 
>>and
>>? ?stdin file-like objects. ?But when using the read and write methods 
>>of
>>? ?those options, you do not have the benefit of asynchronous I/O. ?In 
>>the
>>? ?proposed solution the wrapper wraps the asynchronous methods to 
>>mimic a
>>? ?file object.
>>
>>Reference Implementation:
>>
>>? ?I have been maintaining a Google Code repository that contains all 
>>of my
>>? ?changes including tests and documentation [9] as well as blog 
>>detailing
>>? ?the problems I have come across in the development process [10].
>>
>>? ?I have been working on implementing non-blocking asynchronous I/O 
>>in the
>>? ?subprocess.Popen module as well as a wrapper class for 
>>subprocess.Popen
>>? ?that makes it so that an executed process can take the place of a 
>>file by
>>? ?duplicating all of the methods and attributes that file objects 
>>have.

"Non-blocking" and "asynchronous" are actually two different things. 
 From the rest of this PEP, I think only a non-blocking API is being 
introduced.  I haven't looked beyond the PEP, though, so I might be 
missing something.
>>? ?There are two base functions that have been added to the 
>>subprocess.Popen
>>? ?class: Popen.send and Popen._recv, each with two separate 
>>implementations,
>>? ?one for Windows and one for Unix based systems. ?The Windows
>>? ?implementation uses ctypes to access the functions needed to 
>>control pipes
>>? ?in the kernel 32 DLL in an asynchronous manner. ?On Unix based 
>>systems,
>>? ?the Python interface for file control serves the same purpose. ?The
>>? ?different implementations of Popen.send and Popen._recv have 
>>identical
>>? ?arguments to make code that uses these functions work across 
>>multiple
>>? ?platforms.

Why does the method for non-blocking read from a pipe start with an "_"? 
This is the convention (widely used) for a private API.  The name also 
doesn't suggest that this is the non-blocking version of reading. 
Similarly, the name "send" doesn't suggest that this is the non-blocking 
version of writing.
>>? ?When calling the Popen._recv function, it requires the pipe name be
>>? ?passed as an argument so there exists the Popen.recv function that 
>>passes
>>? ?selects stdout as the pipe for Popen._recv by default. 
>>?Popen.recv_err
>>? ?selects stderr as the pipe by default. "Popen.recv" and 
>>"Popen.recv_err"
>>? ?are much easier to read and understand than "Popen._recv('stdout' 
>>..." and
>>? ?"Popen._recv('stderr' ..." respectively.

What about reading from other file descriptors?  subprocess.Popen allows 
arbitrary file descriptors to be used.  Is there any provision here for 
reading and writing non-blocking from or to those?
>>? ?Since the Popen._recv function does not wait on data to be produced
>>? ?before returning a value, it may return empty bytes. 
>>?Popen.asyncread
>>? ?handles this issue by returning all data read over a given time
>>? ?interval.

Oh.  Popen.asyncread?   What's that?  This is the first time the PEP 
mentions it.
>>? ?The ProcessIOWrapper class uses the asyncread and asyncwrite 
>>functions to
>>? ?allow a process to act like a file so that there are no blocking 
>>issues
>>? ?that can arise from using the stdout and stdin file objects 
>>produced from
>>? ?a subprocess.Popen call.

What's the ProcessIOWrapper class?  And what's the asyncwrite function? 
Again, this is the first time it's mentioned.

So, to sum up, I think my main comment is that the PEP seems to be 
missing a significant portion of the details of what it's actually 
proposing.  I suspect that this information is present in the 
implementation, which I have not looked at, but it probably belongs in 
the PEP.

Jean-Paul

From daniel at stutzbachenterprises.com  Tue Sep 15 20:32:53 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 15 Sep 2009 13:32:53 -0500
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AAFCBD6.5090104@scottdial.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
Message-ID: <eae285400909151132t1e84aac4ua1e8228a2dee583a@mail.gmail.com>

On Tue, Sep 15, 2009 at 12:16 PM, Scott Dial
<scott+python-dev at scottdial.com<scott%2Bpython-dev at scottdial.com>
> wrote:

> >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0")
> ...
> ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not
> a valid address (hint, it's probably a network)
>
> Because, it *is* a address of a host on a network. I gave in and said:
>

10.1.2.3 is the address of a host on a network.  255.255.240.0 is a subnet
mask.

>>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0")
>
> But then, I was dumbfounded as to how I could get the gateway IP from
> this IPNetwork object. It took me a while to figure out that you can
> iterate over IPNetwork instances:
>

Any IP address on an IP network could be the gateway, so there is no
reliable way to determine the gateway IP from the network address and subnet
mask alone.

Technicalities aside, I agree with you that IPNetwork appears to be doing
double-duty as an IPNetwork type and an IPAddressWithNetwork type, which I
find confusing.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090915/c4898aaa/attachment.htm>

From rdmurray at bitdance.com  Tue Sep 15 20:33:50 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 14:33:50 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AAFDA0D.5040602@scottdial.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
Message-ID: <Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 14:16, Scott Dial wrote:
> In other words, I don't see why obtaining a host address would *not*
> retain the hostmask from the network it was obtained from. I am not
> disagreeing with it being an individual address. I am disagreeing that
> IPNetwork itself already does represent individual addresses (hence my
> aliasing it with IPAddressWithNetwork). And wrt, the logical return
> would be another IPAddressWithNetwork retaining the same mask.

In other other words, maybe we have three data types:

     IPv4Address
     IPv4AddressWithMask
     IPv4Network

Where myAddressWithMask.network would return an IPv4Network object,
and an IPv4Network object would always have the zero of the network
as its representation:

     x = IPv4AddressWithMask('192.168.1.1/24')
     x.network == IPv4Network('192.168.1.0/24')
     x.network[1] == x

In this scheme, IPv4Network('192.168.1.1/24') would raise a ValueError.

Although you could probably have what I called IPv4AddressWithMask be
called IPv4Address, and have what is now IPv4Address just have
netmask and network attributes of None.

If this were done, I would expect IPv4Network.network to be either
an attribute error or return self.

--David

From mcguire at google.com  Tue Sep 15 20:33:16 2009
From: mcguire at google.com (Jake McGuire)
Date: Tue, 15 Sep 2009 11:33:16 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
	<8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>
Message-ID: <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>

On Tue, Sep 15, 2009 at 10:36 AM, Peter Moody <peter at hda3.com> wrote:

> On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire <mcguire at google.com> wrote:
> > On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum <guido at python.org>
> wrote:
> >>
> >> What's the opinion of the other interested party or parties? I don't
> >> want a repeat of the events last time, where we had to pull it at the
> >> last time because there hadn't been enough discussion.
> >
> > How many other people are using this library?  I think it's hard to give
> > really useful feedback on an API without using it for some non-trivial
> task,
> > but maybe other people don't have this problem.
> > -jake
>
> 188 (check that, 190) people have downloaded the 2.0 release in the
> last week (numbers publicly available from the code.google.com). I
> can't tell you how many (if any) have downloaded it via svn.
>

Downloading and using are not the same thing.

-jake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090915/3ba8f3e6/attachment.htm>

From peter at hda3.com  Tue Sep 15 20:36:44 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 15 Sep 2009 11:36:44 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
	<8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>
	<77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>
Message-ID: <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>

On Tue, Sep 15, 2009 at 11:33 AM, Jake McGuire <mcguire at google.com> wrote:
> On Tue, Sep 15, 2009 at 10:36 AM, Peter Moody <peter at hda3.com> wrote:
>>
>> On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire <mcguire at google.com> wrote:
>> > On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum <guido at python.org>
>> > wrote:
>> >>
>> >> What's the opinion of the other interested party or parties? I don't
>> >> want a repeat of the events last time, where we had to pull it at the
>> >> last time because there hadn't been enough discussion.
>> >
>> > How many other people are using this library? ?I think it's hard to give
>> > really useful feedback on an API without using it for some non-trivial
>> > task,
>> > but maybe other people don't have this problem.
>> > -jake
>>
>> 188 (check that, 190) people have downloaded the 2.0 release in the
>> last week (numbers publicly available from the code.google.com). I
>> can't tell you how many (if any) have downloaded it via svn.
>
> Downloading and using are not the same thing.

Correct, but there is a strong positive correlation between the two.
If you have a better method for determining what you would consider an
appropriate level of usage, I'm all ears.

> -jake

From stephen at xemacs.org  Tue Sep 15 20:49:46 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 16 Sep 2009 03:49:46 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090915T200602-36@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<loom.20090915T200602-36@post.gmane.org>
Message-ID: <87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > Speaking as a non-network specialist, it actually looks logical to
 > me to be given an address if I iterate over a network (the same way
 > that, if I iterate on a list, I get individual elements, not
 > 1-element sublists).

But if you iterate over a string you get one character strings.  So
it's an issue of convenience of representation.




From stephen at xemacs.org  Tue Sep 15 20:54:35 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 16 Sep 2009 03:54:35 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AAFDA0D.5040602@scottdial.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
Message-ID: <87skeo11jo.fsf@uwakimon.sk.tsukuba.ac.jp>

Scott Dial writes:

 > ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Network('1.1.1.0/24')

So foo returns True?

def foo():
    a = ipaddr.IPv4Network('1.1.1.0/24')
    return a[0] == a

That seems ... weird.  Maybe you committed a typo?



From solipsis at pitrou.net  Tue Sep 15 20:43:59 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 18:43:59 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
Message-ID: <loom.20090915T203952-716@post.gmane.org>

R. David Murray <rdmurray <at> bitdance.com> writes:
> 
>      x = IPv4AddressWithMask('192.168.1.1/24')
>      x.network == IPv4Network('192.168.1.0/24')
>      x.network[1] == x

I don't think we need an IPAddressWithMask which would just complicate the API
without any obvious benefit.
We just need a factory function which returns a tuple after parsing:

    >>> addr, net = parse_address_with_mask('192.168.1.1/24')
    >>> addr == IPv4Address('192.168.1.1')
    True
    >>> net == IPv4Network('192.168.1.0/24')
    True

Regards

Antoine.



From peter at hda3.com  Tue Sep 15 20:46:01 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 15 Sep 2009 11:46:01 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<loom.20090915T200602-36@post.gmane.org>
	<87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <8517e9350909151146k77261851i35890b2f1b0e902a@mail.gmail.com>

On Tue, Sep 15, 2009 at 11:49 AM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> Antoine Pitrou writes:
>
> ?> Speaking as a non-network specialist, it actually looks logical to
> ?> me to be given an address if I iterate over a network (the same way
> ?> that, if I iterate on a list, I get individual elements, not
> ?> 1-element sublists).
>
> But if you iterate over a string you get one character strings. ?So
> it's an issue of convenience of representation.

True, but you don't get a string of equal length back.

from the PEP:

 - Treat network elements as lists (in so far as it's possible).

(that should probably say "treat networks as lists")

you will no be able, with a string, to do something like

>>> 'this is my string'[0][1][2][3]

but that's what's being suggested here for networks.

>>> ipaddr.IPv4Network('1.1.1.1/24')[0][1][2][3]...

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From rdmurray at bitdance.com  Tue Sep 15 21:09:08 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 15:09:08 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090915T203952-716@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
	<loom.20090915T203952-716@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 18:43, Antoine Pitrou wrote:
> R. David Murray <rdmurray <at> bitdance.com> writes:
>>
>>      x = IPv4AddressWithMask('192.168.1.1/24')
>>      x.network == IPv4Network('192.168.1.0/24')
>>      x.network[1] == x
>
> I don't think we need an IPAddressWithMask which would just complicate the API
> without any obvious benefit.
> We just need a factory function which returns a tuple after parsing:
>
>    >>> addr, net = parse_address_with_mask('192.168.1.1/24')
>    >>> addr == IPv4Address('192.168.1.1')
>    True
>    >>> net == IPv4Network('192.168.1.0/24')
>    True

I would find that acceptable but sub-optimal.  Most of my use cases
(which involve manipulating router and firewall configuration files) would
then start by making a little class named AddressWithNetwork to hold the
tuple returned by your parse function, with attributes 'ip' and 'network'
and a representation that included the netmask.

Other people's use cases would look like addr, _ = parse_address...

An IPv4Address with 'network' and 'mask' attributes that could be
None would also not complicate the API, IMO, and would handle both
of these use cases.

--David

From scott+python-dev at scottdial.com  Tue Sep 15 21:13:27 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 15 Sep 2009 15:13:27 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151146k77261851i35890b2f1b0e902a@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<20090915021130.4E57A600115@longblack.object-craft.com.au>	<loom.20090915T162717-42@post.gmane.org>	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>	<4AAFCBD6.5090104@scottdial.com>	<loom.20090915T200602-36@post.gmane.org>	<87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp>
	<8517e9350909151146k77261851i35890b2f1b0e902a@mail.gmail.com>
Message-ID: <4AAFE757.9010503@scottdial.com>

Peter Moody wrote:
> but that's what's being suggested here for networks.
> 
>>>> ipaddr.IPv4Network('1.1.1.1/24')[0][1][2][3]...

This example here solidifies my support of RDM's suggestion of there
being 3 types:

    IPv4Address
    IPv4AddressWithNetwork (or as he called it: IPv4AddressWithMask)
    IPv4Network

The primary difference between IPv4Network and IPv4AddressWithNetwork
would be that IPv4AddressWithNetwork would not support
iteration/indexing. The example above makes no sense logically and my
original suggestion unknowingly opened that can of worms. If indexing a
IPv4Network returned IPv4AddressWithNetwork, then that would remove that
oddity.

This would also solve the weirdness that Stephen brought up in another
branch of this discussion:

Stephen J. Turnbull wrote:
> Scott Dial writes:
>  > ipaddr.IPv4Network('1.1.1.0/24')[0] ==
>  >     ipaddr.IPv4Network('1.1.1.0/24')
>
> So foo returns True?
>
> def foo():
>     a = ipaddr.IPv4Network('1.1.1.0/24')
>     return a[0] == a
>
> That seems ... weird.  Maybe you committed a typo?

The root of the weirdness is that my proposition would appear to make
IPv4Network play double-duty. However, it already does! That you can
instantiate a IPv4Network object with a non-zero host is a clue, and
that it is apparently the only way to input a host with a mask is the
second clue.

If I have an IP (10.2.3.4) and I know the netmask (say, 255.255.0.0),
then how do I get the network that it is on? As it stands, the only way
I see to do this is to do:

>>> net = ipaddr.IPv4Network("10.2.3.4/255.255.0.0")

Which is strange, because I didn't input a *network*, I inputted an
*address* on a network. Moreover, it's strange because:

>>> net[0] == net.ip
False

If I needed that identity to hold true, then I have to do:

>>> net = ipaddr.IPv4Network("%s/%s" % (net[0], net.netmask)

Although it is unclear why a "network" has an "ip" attribute. As far as
I am concerned, IPv4Network objects should *only* have a net.network.

Hopefully at this point, I have made the case that IPv4Network already
is playing double-duty as a IPv4Network and IPv4AddressWithNetwork. And
that the API would be well-served by splitting that role and that it
would be considerably less confusing.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From solipsis at pitrou.net  Tue Sep 15 21:20:15 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 19:20:15 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
	<loom.20090915T203952-716@post.gmane.org>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
Message-ID: <loom.20090915T211440-93@post.gmane.org>

R. David Murray <rdmurray <at> bitdance.com> writes:
> 
> I would find that acceptable but sub-optimal.  Most of my use cases
> (which involve manipulating router and firewall configuration files) would
> then start by making a little class named AddressWithNetwork to hold the
> tuple returned by your parse function, with attributes 'ip' and 'network'
> and a representation that included the netmask.

If the AddressWithNetwork class has useful functionality of its own, then why
not (although I think that conceptually, this functionality should belong to the
Network class instead).

If, however, the AddressWithNetwork class is only a recipient for a tuple, then
it sounds useless. It's like having an AddressWithPort to hold things like
"127.0.0.1:8080". Building little featureless recipients like that isn't really
idiomatic Python, IMO.

> An IPv4Address with 'network' and 'mask' attributes that could be
> None would also not complicate the API, IMO, and would handle both
> of these use cases.

But it would be confusing and conceptually bizarre, because an address (in
usually accepted terminology) doesn't have a network and a mask.
I think we should keep the API clean and reasonably logical, rather than try to
cover all use cases at the expense of weird shortcuts.

Regards

Antoine.



From srittau at jroger.in-berlin.de  Tue Sep 15 21:35:13 2009
From: srittau at jroger.in-berlin.de (Sebastian Rittau)
Date: Tue, 15 Sep 2009 21:35:13 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AAFCBD6.5090104@scottdial.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
Message-ID: <20090915193513.GA6606@jroger.in-berlin.de>

On Tue, Sep 15, 2009 at 01:16:06PM -0400, Scott Dial wrote:

> I have to concur with the opinions above. I was very confused by the
> following error:
> 
> >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0")
> ...
> ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not
> a valid address (hint, it's probably a network)
> 
> Because, it *is* a address of a host on a network.

To me, 10.1.2.3/255.255.240.0 is not a host address, but specifies a
network. I.e., 10.1.2.3/255.255.240.0 == 10.1.0.0/255.255.240.0 ==
10.1.35.200/20.

> >>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0")
> 
> But then, I was dumbfounded as to how I could get the gateway IP from
> this IPNetwork object.

Well, you can't. There is no way to determine a gateway, without querying
the network topology. This is clearly outside the scope of this module.
The only two known host addresses of a network are the network address
(10.1.0.0 in the example you gave) and the broadcast address (10.1.15.255).

> It took me a while to figure out that you can
> iterate over IPNetwork instances:
> 
> >>> gateway = net[1]
> 
> I was then confused, because:
> 
> >>> print(type(gateway))
> <class 'ipaddr.IPv4Address'>
> 
> Which sorta blew my mind.. I fully expected to receive an IPNetwork back
> from that operation. It is unclear to me why the network information
> gets chucked by that operation.

This makes perfect sense to me. An IP network consists of a list of IP
addresses. Returning /32 networks seems kind of pointless to me.

> I foresee having to work around that in
> real applications by doing something obnoxious like:
> 
> >>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask))

But a gateway is not an IP address plus hostmask. A gateway is just a single
IP address. What is the use of adding a hostmask to the gateway IP address
or some other IP address inside the network?

 - Sebastian

From rdmurray at bitdance.com  Tue Sep 15 21:48:07 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 15:48:07 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090915T211440-93@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
	<loom.20090915T203952-716@post.gmane.org>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 19:20, Antoine Pitrou wrote:
> R. David Murray <rdmurray <at> bitdance.com> writes:
>> I would find that acceptable but sub-optimal.  Most of my use cases
>> (which involve manipulating router and firewall configuration files) would
>> then start by making a little class named AddressWithNetwork to hold the
>> tuple returned by your parse function, with attributes 'ip' and 'network'
>> and a representation that included the netmask.
>
> If the AddressWithNetwork class has useful functionality of its own, then why
> not (although I think that conceptually, this functionality should belong to the
> Network class instead).

But it's not a network.  It's an address that is explicitly associated
with a network.

It's useful functionality is parsing/validating an address+mask, rendering
as address+mask, and being able to get the associated IP and network objects
from it.  It's a small class, but useful, IMO.

> If, however, the AddressWithNetwork class is only a recipient for a tuple, then
> it sounds useless. It's like having an AddressWithPort to hold things like
> "127.0.0.1:8080". Building little featureless recipients like that isn't really
> idiomatic Python, IMO.

I could see myself doing that, too, though I haven't had a reason to yet :)
Although I'd probably call it 'IPv4EndPoint' or some such, since it would
be representing one endpoint of a traffic flow...

>> An IPv4Address with 'network' and 'mask' attributes that could be
>> None would also not complicate the API, IMO, and would handle both
>> of these use cases.
>
> But it would be confusing and conceptually bizarre, because an address (in
> usually accepted terminology) doesn't have a network and a mask.
> I think we should keep the API clean and reasonably logical, rather than try to
> cover all use cases at the expense of weird shortcuts.

OK.  I'll withdrawal that suggestion.

If AddressWithMask doesn't get added I won't be heartbroken; it is easy
enough to write in the programs where I need it (at least it would be
if the parsing/validation is in the library).  However, I do not think
that the proposed API should accept, eg, IPv4Network('192.168.1.1/24')
as valid.  That's just too confusing and error prone.

Oh, and if people don't like 'zero' as the name for the IPv4Address
at the beginning of the network IP range, how about 'network_ip' or
'network_address' instead of just network, to make it clearer that it
is an address?

--David

From solipsis at pitrou.net  Tue Sep 15 21:58:31 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 15 Sep 2009 21:58:31 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
	<loom.20090915T203952-716@post.gmane.org>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
Message-ID: <1253044711.5678.56.camel@localhost>

Le mardi 15 septembre 2009 ? 15:48 -0400, R. David Murray a ?crit :
> 
> It's useful functionality is parsing/validating an address+mask, rendering
> as address+mask, and being able to get the associated IP and network objects
> from it.  It's a small class, but useful, IMO.

If it's only about parsing and validating, then a tuple works just fine.
Getting the associated IP objects is obtained from iterating over the
network.
Getting the associated network objects I don't understand. There is a
single network, not a bunch of them.

> However, I do not think
> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24')
> as valid.  That's just too confusing and error prone.

Indeed, it should throw some kind of ValueError instead.

> Oh, and if people don't like 'zero' as the name for the IPv4Address
> at the beginning of the network IP range, how about 'network_ip' or
> 'network_address' instead of just network, to make it clearer that it
> is an address?

How about something more explicit about how it's obtained, like
'lower_bound'?




From martin at v.loewis.de  Tue Sep 15 22:12:46 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Sep 2009 22:12:46 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
Message-ID: <4AAFF53E.8090704@v.loewis.de>

> What's the opinion of the other interested party or parties? I don't
> want a repeat of the events last time, where we had to pull it at the
> last time because there hadn't been enough discussion.

I'm slightly interested, but didn't have the time to review the PEP at
all yet. I may be able to do so before the end of the year, but can't
promise that.

In any case, I don't see a reason to hurry - if the PEP is adopted
two or three months before the release of 2.7, it will still be in
time, IMO.

Regards,
Martin

From tjreedy at udel.edu  Tue Sep 15 22:30:42 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 15 Sep 2009 16:30:42 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>	<8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>	<77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>
	<8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
Message-ID: <h8othi$rhj$1@ger.gmane.org>

Peter Moody wrote:
> On Tue, Sep 15, 2009 at 11:33 AM, Jake McGuire <mcguire at google.com> wrote:

>> Downloading and using are not the same thing.
> 
> Correct, but there is a strong positive correlation between the two.
> If you have a better method for determining what you would consider an
> appropriate level of usage, I'm all ears.

You sneak in a bit of code that sends a message to home base "In use 
from <IPaddress>";-)

Kidding aside, there are usage reports, but those tend to be lower if 
the code satifies than if not. I think 200 indicates fair interest. Let 
us hope there is more feedback, pro and con.

tjr


From fetchinson at googlemail.com  Tue Sep 15 22:34:28 2009
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Tue, 15 Sep 2009 13:34:28 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
	<8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>
	<77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>
	<8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
Message-ID: <fbe2e2100909151334k62686719veac45c75c644858e@mail.gmail.com>

>>> 188 (check that, 190) people have downloaded the 2.0 release in the
>>> last week (numbers publicly available from the code.google.com). I
>>> can't tell you how many (if any) have downloaded it via svn.
>>
>> Downloading and using are not the same thing.
>
> Correct, but there is a strong positive correlation between the two.
> If you have a better method for determining what you would consider an
> appropriate level of usage, I'm all ears.

A good way of determining the level of usage would be pointing to open
source projects that are popular in the python community and which
incorporate your module.

Cheers,
Daniel

-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown

From peter at hda3.com  Tue Sep 15 22:43:26 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 15 Sep 2009 13:43:26 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <fbe2e2100909151334k62686719veac45c75c644858e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
	<8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>
	<77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>
	<8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
	<fbe2e2100909151334k62686719veac45c75c644858e@mail.gmail.com>
Message-ID: <8517e9350909151343q4eb419d1v35ad5468bc2798a8@mail.gmail.com>

On Tue, Sep 15, 2009 at 1:34 PM, Daniel Fetchinson
<fetchinson at googlemail.com> wrote:
>>>> 188 (check that, 190) people have downloaded the 2.0 release in the
>>>> last week (numbers publicly available from the code.google.com). I
>>>> can't tell you how many (if any) have downloaded it via svn.
>>>
>>> Downloading and using are not the same thing.
>>
>> Correct, but there is a strong positive correlation between the two.
>> If you have a better method for determining what you would consider an
>> appropriate level of usage, I'm all ears.
>
> A good way of determining the level of usage would be pointing to open
> source projects that are popular in the python community and which
> incorporate your module.

well, the 2.0 release is still new. codesearch.google.com shows some
projects using the 1.x release; hopefully some of those 200
downloaders will put up some publicly indexable python code at some
point.

> Cheers,
> Daniel
>
> --
> Psss, psss, put it down! - http://www.cafepress.com/putitdown
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From rdmurray at bitdance.com  Wed Sep 16 01:11:34 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 15 Sep 2009 19:11:34 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <1253044711.5678.56.camel@localhost>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
	<loom.20090915T203952-716@post.gmane.org>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
	<1253044711.5678.56.camel@localhost>
Message-ID: <Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>

On Tue, 15 Sep 2009 at 21:58, Antoine Pitrou wrote:
> Le mardi 15 septembre 2009 ?? 15:48 -0400, R. David Murray a ??crit :
>>
>> It's useful functionality is parsing/validating an address+mask, rendering
>> as address+mask, and being able to get the associated IP and network objects
>> from it.  It's a small class, but useful, IMO.
>
> If it's only about parsing and validating, then a tuple works just fine.
> Getting the associated IP objects is obtained from iterating over the
> network.
> Getting the associated network objects I don't understand. There is a
> single network, not a bunch of them.

The plural was for the two attributes.  I guess I just like accessing
things by name better than accessing them by indexes.  I suppose a
NamedTuple would work.  Anyway, like I said I'm not attached to that
class.

>> However, I do not think
>> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24')
>> as valid.  That's just too confusing and error prone.
>
> Indeed, it should throw some kind of ValueError instead.

Peter, what do you think?

>> Oh, and if people don't like 'zero' as the name for the IPv4Address
>> at the beginning of the network IP range, how about 'network_ip' or
>> 'network_address' instead of just network, to make it clearer that it
>> is an address?
>
> How about something more explicit about how it's obtained, like
> 'lower_bound'?

Well, when this came up earlier 'first' and 'last' were suggested to
replace 'network' and 'broadcast', but that proposal didn't get much
traction, the argument being that in normal technical parlance
the first IP in the netblock is (apparently, this isn't my personal
experience(*)) called the network, and the broadcast is almost invariably
the last IP in the netblock (and that does agree with my experience).

--David

(*) in my experience it is called the "network number", which I gather
derives from the pre-CIDR days.  "network identifier" also appears
to be used.  In my experience "network" refers to something that has
a netmask, although the netmask may be implicit in its class in the
classful routing sense, or through other context.

Consider this article, for example:
http://compnetworking.about.com/od/workingwithipaddresses/l/aa042900a.htm

From scott+python-dev at scottdial.com  Wed Sep 16 01:34:09 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 15 Sep 2009 19:34:09 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<20090915021130.4E57A600115@longblack.object-craft.com.au>	<loom.20090915T162717-42@post.gmane.org>	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>	<4AAFCBD6.5090104@scottdial.com>	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>	<4AAFDA0D.5040602@scottdial.com>	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>	<loom.20090915T203952-716@post.gmane.org>	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>	<loom.20090915T211440-93@post.gmane.org>	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>	<1253044711.5678.56.camel@localhost>
	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>
Message-ID: <4AB02471.7000701@scottdial.com>

R. David Murray wrote:
> On Tue, 15 Sep 2009 at 21:58, Antoine Pitrou wrote:
>> Le mardi 15 septembre 2009 ? 15:48 -0400, R. David Murray a ?crit :
>>> However, I do not think
>>> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24')
>>> as valid.  That's just too confusing and error prone.
>>
>> Indeed, it should throw some kind of ValueError instead.
> 
> Peter, what do you think?

This change would have to be dependent on Antoine's suggestion of:

    >>> addr, net = parse_address_with_mask('192.168.1.1/24')
    >>> addr == IPv4Address('192.168.1.1')
    True
    >>> net == IPv4Network('192.168.1.0/24')
    True

Otherwise, I am ok with this change too. It resloves the weird duality
of IPv4Network. And as RDM says, anyone who wants the
IPv4AddressWithNetwork functionality can just role it themselves. At the
moment of iteration, you have access to what network it is and can build
your own IPv4AddressWithNetwork objects.

net = IPv4Network('10.0.0.0/24')
netaddrs = []
for addr in net:
    netaddrs.append(IPv4AddressWithNetwork(addr, net))

I guess I am ok with this. It seems sub-optimal (why not just return a
IPv4AddressWithNetwork to begin with?) but I suppose it is no less
efficient since the same objects would be constructed.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From scott+python-dev at scottdial.com  Wed Sep 16 01:38:40 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 15 Sep 2009 19:38:40 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090915193513.GA6606@jroger.in-berlin.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<20090915021130.4E57A600115@longblack.object-craft.com.au>	<loom.20090915T162717-42@post.gmane.org>	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>	<4AAFCBD6.5090104@scottdial.com>
	<20090915193513.GA6606@jroger.in-berlin.de>
Message-ID: <4AB02580.4060502@scottdial.com>

Sebastian Rittau wrote:
> On Tue, Sep 15, 2009 at 01:16:06PM -0400, Scott Dial wrote:
>>>>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0")
>> But then, I was dumbfounded as to how I could get the gateway IP from
>> this IPNetwork object.
> 
> Well, you can't. There is no way to determine a gateway, without querying
> the network topology. This is clearly outside the scope of this module.
> The only two known host addresses of a network are the network address
> (10.1.0.0 in the example you gave) and the broadcast address (10.1.15.255).

I was not writing a general utility for the world to use. On *my*
network it is always the [1]th address used for the gateway. But, I
agree in general that is not true.

>> Which sorta blew my mind.. I fully expected to receive an IPNetwork back
>> from that operation. It is unclear to me why the network information
>> gets chucked by that operation.
> 
> This makes perfect sense to me. An IP network consists of a list of IP
> addresses. Returning /32 networks seems kind of pointless to me.

As I clarified later, I expected it to still be a /24 network, not a
/32. /32 makes no sense.

> But a gateway is not an IP address plus hostmask. A gateway is just a single
> IP address. What is the use of adding a hostmask to the gateway IP address
> or some other IP address inside the network?

My argument was from a destruction of information standpoint. Why is the
network chucked away? Wouldn't it be handy to be able to inspected
addr.network to see what network it was on? Perhaps because you are
building routing rules and need to actually know that. As it is, you
will just have to build your own IPAddressWithNetwork objects.

Hopefully I have covered all of you inquiries, but I feel like that a
lot of this is covered by many other replies in this thread.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From peter at hda3.com  Wed Sep 16 02:05:46 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 15 Sep 2009 17:05:46 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB02471.7000701@scottdial.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
	<loom.20090915T203952-716@post.gmane.org>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
	<1253044711.5678.56.camel@localhost>
	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>
	<4AB02471.7000701@scottdial.com>
Message-ID: <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>

On Tue, Sep 15, 2009 at 4:34 PM, Scott Dial
<scott+python-dev at scottdial.com> wrote:
> R. David Murray wrote:
>> On Tue, 15 Sep 2009 at 21:58, Antoine Pitrou wrote:
>>> Le mardi 15 septembre 2009 ? 15:48 -0400, R. David Murray a ?crit :
>>>> However, I do not think
>>>> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24')
>>>> as valid. ?That's just too confusing and error prone.
>>>
>>> Indeed, it should throw some kind of ValueError instead.
>>
>> Peter, what do you think?

I disagree. It seems overly pedantic to throw a valueerror on this.
IPy does (used to do?) this and it's one of the primary reasons I
wrote ipaddr.

> This change would have to be dependent on Antoine's suggestion of:
>
> ? ?>>> addr, net = parse_address_with_mask('192.168.1.1/24')
> ? ?>>> addr == IPv4Address('192.168.1.1')
> ? ?True
> ? ?>>> net == IPv4Network('192.168.1.0/24')
> ? ?True
>
> Otherwise, I am ok with this change too. It resloves the weird duality
> of IPv4Network. And as RDM says, anyone who wants the
> IPv4AddressWithNetwork functionality can just role it themselves. At the
> moment of iteration, you have access to what network it is and can build
> your own IPv4AddressWithNetwork objects.

now, I understand that as the author, I have certain biases, but I'm
not having the same difficulty with this duality as others.  I really
don't even see it as a duality.

It seems as though your issue could more easily be solved by having an
iterator in BaseNet which returns a new network object with the
original prefix length.

eg:

for addr in ipaddr.IPv4Network('1.1.1.0/24').iterwithnetmask():
  # 1.1.1.0/24, 1.1.1.1/24, 1.1.1.2/24...

(or better yet, .iterwithprefix)

this can be done much more cleanly/easily than adding two new
(redundant in my eyes) classes and much less pedantically than raising
exceptions w/ IPv?Network() constructor received something other than
base network address.

Is this sufficient? If not, perhaps someone could help me better
understand the confusion over IPv?Network objects.

Cheers,
/peter

> net = IPv4Network('10.0.0.0/24')
> netaddrs = []
> for addr in net:
> ? ?netaddrs.append(IPv4AddressWithNetwork(addr, net))
>
> I guess I am ok with this. It seems sub-optimal (why not just return a
> IPv4AddressWithNetwork to begin with?) but I suppose it is no less
> efficient since the same objects would be constructed.
>
> --
> Scott Dial
> scott at scottdial.com
> scodial at cs.indiana.edu
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From solipsis at pitrou.net  Wed Sep 16 02:20:41 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 00:20:41 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
	<loom.20090915T203952-716@post.gmane.org>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
	<1253044711.5678.56.camel@localhost>
	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>
	<4AB02471.7000701@scottdial.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
Message-ID: <loom.20090916T021036-577@post.gmane.org>

Peter Moody <peter <at> hda3.com> writes:
> 
> >>>> However, I do not think
> >>>> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24')
> >>>> as valid. ?That's just too confusing and error prone.
> >>>
> >>> Indeed, it should throw some kind of ValueError instead.
> >>
> >> Peter, what do you think?
> 
> I disagree. It seems overly pedantic to throw a valueerror on this.
> IPy does (used to do?) this and it's one of the primary reasons I
> wrote ipaddr.

Python is not PHP and does not try to be overly "smart" or tolerant when faced
with bizarrely-formatted input.

I don't see any valid reason for entering a network as "192.168.1.1/24" rather
than the canonical "192.168.1.0/24". The former might indicate a typing error or
a mental slip, so let's be helpful and signal it to the user.

> for addr in ipaddr.IPv4Network('1.1.1.0/24').iterwithnetmask():
>   # 1.1.1.0/24, 1.1.1.1/24, 1.1.1.2/24...

I don't have any strong feelings about it, although I don't really think it is
needed, and a plethoric API is not a good thing (neither clarity-wise nor
maintenance-wise).

> this can be done much more cleanly/easily than adding two new
> (redundant in my eyes) classes and much less pedantically than raising
> exceptions w/ IPv?Network() constructor received something other than
> base network address.

I don't think anybody suggested adding two classes. As for the "pedantry" of
thoroughly validating inputs, see above.

Regards

Antoine.



From solipsis at pitrou.net  Wed Sep 16 02:33:56 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 00:33:56 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
	<loom.20090915T203952-716@post.gmane.org>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
	<1253044711.5678.56.camel@localhost>
	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>
	<4AB02471.7000701@scottdial.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<loom.20090916T021036-577@post.gmane.org>
Message-ID: <loom.20090916T023248-272@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> I don't see any valid reason for entering a network as "192.168.1.1/24" rather
> than the canonical "192.168.1.0/24". The former might indicate a typing error
> or
> a mental slip, so let's be helpful and signal it to the user.

Or perhaps there can be an optional "strict=True" (or "strict=False") argument
to the constructor / parsing function.

Regards

Antoine.



From eric at trueblade.com  Wed Sep 16 03:02:01 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 15 Sep 2009 21:02:01 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090916T021036-577@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AAFDA0D.5040602@scottdial.com>	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>	<loom.20090915T203952-716@post.gmane.org>	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>	<loom.20090915T211440-93@post.gmane.org>	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>	<1253044711.5678.56.camel@localhost>	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>	<4AB02471.7000701@scottdial.com>	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<loom.20090916T021036-577@post.gmane.org>
Message-ID: <4AB03909.3090608@trueblade.com>

Antoine Pitrou wrote:
> Peter Moody <peter <at> hda3.com> writes:
>>>>>> However, I do not think
>>>>>> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24')
>>>>>> as valid.  That's just too confusing and error prone.
>>>>> Indeed, it should throw some kind of ValueError instead.
>>>> Peter, what do you think?
>> I disagree. It seems overly pedantic to throw a valueerror on this.
>> IPy does (used to do?) this and it's one of the primary reasons I
>> wrote ipaddr.
> 
> Python is not PHP and does not try to be overly "smart" or tolerant when faced
> with bizarrely-formatted input.
> 
> I don't see any valid reason for entering a network as "192.168.1.1/24" rather
> than the canonical "192.168.1.0/24". The former might indicate a typing error or
> a mental slip, so let's be helpful and signal it to the user.

I completely agree. I don't know of any situation where I'd want a 
network of "192.168.1.1/24" to be anything other than an error.

Eric.

From eric at trueblade.com  Wed Sep 16 03:02:50 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 15 Sep 2009 21:02:50 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090916T023248-272@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AAFDA0D.5040602@scottdial.com>	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>	<loom.20090915T203952-716@post.gmane.org>	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>	<loom.20090915T211440-93@post.gmane.org>	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>	<1253044711.5678.56.camel@localhost>	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>	<4AB02471.7000701@scottdial.com>	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>	<loom.20090916T021036-577@post.gmane.org>
	<loom.20090916T023248-272@post.gmane.org>
Message-ID: <4AB0393A.5020906@trueblade.com>

Antoine Pitrou wrote:
> Antoine Pitrou <solipsis <at> pitrou.net> writes:
>> I don't see any valid reason for entering a network as "192.168.1.1/24" rather
>> than the canonical "192.168.1.0/24". The former might indicate a typing error
>> or
>> a mental slip, so let's be helpful and signal it to the user.
> 
> Or perhaps there can be an optional "strict=True" (or "strict=False") argument
> to the constructor / parsing function.

If so, I hope that we'd default to strict=True.

From peter at hda3.com  Wed Sep 16 03:05:26 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 15 Sep 2009 18:05:26 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB03909.3090608@trueblade.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
	<1253044711.5678.56.camel@localhost>
	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>
	<4AB02471.7000701@scottdial.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<loom.20090916T021036-577@post.gmane.org>
	<4AB03909.3090608@trueblade.com>
Message-ID: <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>

On Tue, Sep 15, 2009 at 6:02 PM, Eric Smith <eric at trueblade.com> wrote:
> Antoine Pitrou wrote:
>>
>> Peter Moody <peter <at> hda3.com> writes:
>>>>>>>
>>>>>>> However, I do not think
>>>>>>> that the proposed API should accept, eg,
>>>>>>> IPv4Network('192.168.1.1/24')
>>>>>>> as valid. ?That's just too confusing and error prone.
>>>>>>
>>>>>> Indeed, it should throw some kind of ValueError instead.
>>>>>
>>>>> Peter, what do you think?
>>>
>>> I disagree. It seems overly pedantic to throw a valueerror on this.
>>> IPy does (used to do?) this and it's one of the primary reasons I
>>> wrote ipaddr.
>>
>> Python is not PHP and does not try to be overly "smart" or tolerant when
>> faced
>> with bizarrely-formatted input.
>>
>> I don't see any valid reason for entering a network as "192.168.1.1/24"
>> rather
>> than the canonical "192.168.1.0/24". The former might indicate a typing
>> error or
>> a mental slip, so let's be helpful and signal it to the user.
>
> I completely agree. I don't know of any situation where I'd want a network
> of "192.168.1.1/24" to be anything other than an error.

when you're entering the address of your nic.

> Eric.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From peter at hda3.com  Wed Sep 16 03:08:05 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 15 Sep 2009 18:08:05 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090916T023248-272@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
	<1253044711.5678.56.camel@localhost>
	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>
	<4AB02471.7000701@scottdial.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<loom.20090916T021036-577@post.gmane.org>
	<loom.20090916T023248-272@post.gmane.org>
Message-ID: <8517e9350909151808x173c81a3p8ff2b5d30c06f021@mail.gmail.com>

On Tue, Sep 15, 2009 at 5:33 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Antoine Pitrou <solipsis <at> pitrou.net> writes:
>>
>> I don't see any valid reason for entering a network as "192.168.1.1/24" rather
>> than the canonical "192.168.1.0/24". The former might indicate a typing error
>> or
>> a mental slip, so let's be helpful and signal it to the user.
>
> Or perhaps there can be an optional "strict=True" (or "strict=False") argument
> to the constructor / parsing function.

I can live w/ a default of strict=False.  there are plenty of cases
where it's not an error and easy enough ways to check, if the
developer is concerned, with or without an option.  eg if addr.ip !=
addr.network:

Cheers,
/peter

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From amk at amk.ca  Wed Sep 16 03:48:10 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 15 Sep 2009 21:48:10 -0400
Subject: [Python-Dev] Python Language Summit #2 in February
Message-ID: <20090916014810.GA318@andrew-kuchlings-macbook.local>

PyCon 2010 will be February 19-21 2010 in Atlanta, Georgia (US).
Van Lindberg, PyCon chair, has approved having another Python Language
Summit on Thursday, February 18 2010.  The web page for it is
<http://us.pycon.org/2010/about/summits/language/>

The Python Language Summit is an invitation-only event for developers
of Python implementations and the standard library, to discuss issues
of common concern to all implementors.

It will mostly be organized similarly to last year's event.  I'm going
to drop the initial open discussion, which wasn't very useful, and
we'll just have three 1.5-hour discussion sessions.  The fourth time
slot will be left open for chatting, development, or whatever the
attendees want to do.

We therefore need to decide what those three sessions should be about.
Please discuss on python-dev and hopefully we can arrive at some
consensus on topics of reasonably wide current interest.  (See
http://us.pycon.org/2009/about/summits/language/ for a reminder of
last year's topics.)

Feel free to raise discussions on other lists such as jython-dev,
catalog-sig, or wherever, but please summarize the results here on
python-dev; I won't see discussions on other lists.

Like last time, invitations will be sent to the committers for Python,
Jython, IronPython, PyPy, Pynie, plus a few extra people and projects.
If you want to suggest another project or person who should be
invited, please send me a private e-mail.


Andrew M. Kuchling
amk at amk.ca

From andrewm at object-craft.com.au  Wed Sep 16 04:32:31 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Wed, 16 Sep 2009 12:32:31 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151808x173c81a3p8ff2b5d30c06f021@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
	<1253044711.5678.56.camel@localhost>
	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>
	<4AB02471.7000701@scottdial.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<loom.20090916T021036-577@post.gmane.org>
	<loom.20090916T023248-272@post.gmane.org>
	<8517e9350909151808x173c81a3p8ff2b5d30c06f021@mail.gmail.com>
Message-ID: <20090916023231.5A00D5AC0D6@longblack.object-craft.com.au>

>>> I don't see any valid reason for entering a network as "192.168.1.1/24"
>>> rather than the canonical "192.168.1.0/24". The former might indicate a
>>> typing error or a mental slip, so let's be helpful and signal it to the
>>> user.
>>
>> Or perhaps there can be an optional "strict=True" (or "strict=False")
>> argument to the constructor / parsing function.
>
>I can live w/ a default of strict=False.  there are plenty of cases
>where it's not an error and easy enough ways to check, if the
>developer is concerned, with or without an option.  eg if addr.ip !=
>addr.network:

I agree - there are definitely times when it is not an error, but I don't
like the idea of a "strict" flag.

I've done a bit of everything - router configs with a national ISP,
scripts to manage host configuration, user interfaces, you name it.
The way I see it, we need:

 * Two address classes that describe a single IP end-point - "Address" with
   no mask and "AddressWithMask" (the later being the current Network
   class, minus the container-like behaviour).

 * A "Network" container-like class. Same as the current Network class,
   but addresses with masked bits would be considered an error.

This is along the lines that RDM was suggesting, except that we remove the
container behaviour from AddressWithMask.

Additionally:

 * The .network attribute on an AddressWithMask would return a Network
   instance.

 * An Address class would not have a .network attribute

 * Network.__contains__() would accept Network, Address and
   AddressWithMask. Only Network implements __contains__ -
   an AddressWithMask can't contain another address, although it's
   .network can.

 * Maybe an Address should compare equal with an AddressWithMask if
   the address is identical and the mask is equivalent to /32?

Personally, I don't see a strong use-case for the list-like indexing and
iteration behaviour - I think it's enough to implement some basic
container behaviour, but I won't object to the iterator and indexing,
provided they don't distort the rest of the design (which I fear they
are doing now). Iterating or indexing a network should return Address
or AddressWithMask instances - if the later, the mask should match the
parent network's mask.

I'm not particularly wedded to the name "AddressWithMask" - maybe it
could be NetworkAddress or MaskedAddress or ?

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From jonrascal at gmail.com  Wed Sep 16 04:49:38 2009
From: jonrascal at gmail.com (rascal)
Date: Tue, 15 Sep 2009 19:49:38 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151808x173c81a3p8ff2b5d30c06f021@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090915T211440-93@post.gmane.org>
	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
	<1253044711.5678.56.camel@localhost>
	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>
	<4AB02471.7000701@scottdial.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<loom.20090916T021036-577@post.gmane.org>
	<loom.20090916T023248-272@post.gmane.org>
	<8517e9350909151808x173c81a3p8ff2b5d30c06f021@mail.gmail.com>
Message-ID: <b331d0640909151949t1cc852b9m570c701f70182242@mail.gmail.com>

I'd like to see more clear direction for this. Is it a string manipulation
library, or an IP address library?

There are ONLY TWO types of IP representations:
1. Network = 32 bit number AND 32 bit mask. That is the binary AND. The mask
needs to be preserved, but not the network portion. Any library should
return the ANDed result. Whether the number is 192.186.1.1/24 or
192.168.1.0/24, the result after 'and' is the same. Add a strict=false in
case people need it for validation, better yet, add a validation helper
function, but always return the proper form.

2. A host address is an IP address with no network info.  No mask
information, ever.


What about Broadcast / Network IDs?
The "Network-ID or Network Number" and "Broadcast" are meaningless without
context.
1. A network number by itself it's just another address. Its use depends on
context. When routing, it needs a mask.

> route 10.0.0.0 255.0.0.0 192.168.1.1
>
When writing firewall ACLs, it is a host.

> access-list aclname permit ip any 192.168.1.2 255.255.255.254
>
Since both are valid, preference shouldn't be given. Instead add a helper
for firstip.

2. Same story with broadcast. Use depends on context. Add the lastip helper.


What does iterating over a network mean?
Iterating over the set of addresses in a network must return host addresses.
If it returns a network, then the address should be ANDed, but that would
return the same number for every iteration. Only returning hosts makes
sense.

We need to set a solid line between IP String manipulation and IP
manipulation.
If you want an IP with a mask that isn't a network, then you're not doing IP
address manipulation, you're doing string manipulation. There are many use
cases for it, but it's a separate consideration to pure IP manipulation.

If string manipulation is desirable, then add another standard library
called ipstring (or a separate class if you want it all jumbled in the same
import).  This way real network computing can be done without weird quirks
of representation.

R.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090915/a2449da2/attachment.htm>

From stephen at xemacs.org  Wed Sep 16 07:30:03 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 16 Sep 2009 14:30:03 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AAFE757.9010503@scottdial.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<loom.20090915T200602-36@post.gmane.org>
	<87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp>
	<8517e9350909151146k77261851i35890b2f1b0e902a@mail.gmail.com>
	<4AAFE757.9010503@scottdial.com>
Message-ID: <87pr9r1mp0.fsf@uwakimon.sk.tsukuba.ac.jp>

I haven't looked closedly at the reference implementation, and my
experience is small.  So take this as "POLA to the naive new user of
the API" discussion.

Scott Dial writes:

 > This example here solidifies my support of RDM's suggestion of there
 > being 3 types:
 > 
 >     IPv4Address
 >     IPv4AddressWithNetwork (or as he called it: IPv4AddressWithMask)
 >     IPv4Network

IPv4AddressWithMask seems silly to me, implementation-wise.  IP
networks are conceptually globally unique (and even for unroutable
networks they're locally unique), and it seems to me that I want that
to be true internally (for example, I once implemented an IP network
type that maintains a hostname table).  If you need *the* network,
you'll have to look it up in a mask_to_network table (at least
internally).

Rather, don't you want to just give IPv4Address an attribute
'network'?  This could then be filled in by the indexing method on
IPv4Network.

Then deriving a mask from a network is trivial (networks need to know
their masks anyway), like so:

    IPv4Address('1.1.1.1').network.mask

giving you the mask if .network is known, else an attribute error if
.network is None.  But deriving *the* corresponding network from a
mask will require keeping a table of networks around.

 > that it is apparently the only way to input a host with a mask is the
 > second clue.

I don't see why you would want to do that.  Wouldn't you rather

    IPv4Address('1.1.1.1').network_from_mask('255.255.0.0')

and

    IPv4Address('1.1.1.1').set_network_from_mask('255.255.0.0')


 > If I have an IP (10.2.3.4) and I know the netmask (say, 255.255.0.0),
 > then how do I get the network that it is on? As it stands, the only way
 > I see to do this is to do:
 > 
 > >>> net = ipaddr.IPv4Network("10.2.3.4/255.255.0.0")
 > 
 > Which is strange, because I didn't input a *network*, I inputted an
 > *address* on a network.

No, you inputted a pair of dotted quads.  Dotted quads do triple duty
representing host addresses, network addresses, and network masks.
Since construction is a conversion anyway, I don't have any problem
with constructing a network from that pair so that your test:

 > >>> net[0] == net.ip

returns true.  Ie, the implementation can do

    class IPv4Network(...):
        def __init__(self, dotted_quad_pair_string):
            ip, mask = pair_string_to_int_pair(dotted_quad_pair_string)
            self.mask = mask
            self.ip = ip & mask

 > IPv4Network objects should *only* have a net.network.

That's a naming issue, though, isn't it?  I don't see a use for
net.ip != net.network, so you wouldn't have both of them.  If you *do*
have both for some reason, your test

    net[0] == net.ip

seems to have the semantics "How was I constructed?" and I don't see
why you can assert that should be True.

 > Hopefully at this point, I have made the case that IPv4Network already
 > is playing double-duty as a IPv4Network and IPv4AddressWithNetwork.

At this point, no, you've only made the point that you would try to
use it that way.  Maybe that's natural for you (and you're obviously
more expert than I), but it's not for me.  I haven't experienced a
need for the operations you're trying to do, but if I needed them,
I'd be looking for APIs on IPv4Address and IPv4Network like those I
described above, not an IPv4AddressWithNetwork type (ie, I would
expect that IPv4Address is implemented as IPv4AddressWithNetwork).


From stephen at xemacs.org  Wed Sep 16 07:48:58 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 16 Sep 2009 14:48:58 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090915T211440-93@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
	<loom.20090915T203952-716@post.gmane.org>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
Message-ID: <87ocpb1lth.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > > An IPv4Address with 'network' and 'mask' attributes that could be
 > > None would also not complicate the API, IMO, and would handle both
 > > of these use cases.
 > 
 > But it would be confusing and conceptually bizarre, because an address (in
 > usually accepted terminology) doesn't have a network and a mask.

It doesn't need a mask attribute because the network knows that.

OTOH, it would be useful for an address to be able to cache the
network it is a member of, because the object that represents a
particular network should be unique.  That means a lookup in a table,
and possibly constructing a new IPv4Network object to represent the
anwser if false, if you are given a netmask and asked to deduce *the*
network.  And addresses do, implicitly, have a network.  Otherwise
the're useless!<0.5 wink>


From martin at v.loewis.de  Wed Sep 16 10:00:21 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Sep 2009 10:00:21 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB0393A.5020906@trueblade.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AAFDA0D.5040602@scottdial.com>	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>	<loom.20090915T203952-716@post.gmane.org>	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>	<loom.20090915T211440-93@post.gmane.org>	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>	<1253044711.5678.56.camel@localhost>	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>	<4AB02471.7000701@scottdial.com>	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>	<loom.20090916T021036-577@post.gmane.org>	<loom.20090916T023248-272@post.gmane.org>
	<4AB0393A.5020906@trueblade.com>
Message-ID: <4AB09B15.80705@v.loewis.de>

>>> I don't see any valid reason for entering a network as
>>> "192.168.1.1/24" rather
>>> than the canonical "192.168.1.0/24". The former might indicate a
>>> typing error
>>> or
>>> a mental slip, so let's be helpful and signal it to the user.
>>
>> Or perhaps there can be an optional "strict=True" (or "strict=False")
>> argument
>> to the constructor / parsing function.
> 
> If so, I hope that we'd default to strict=True.

I don't see the point of enforcing strictness here. There are good use
cases: if you know your address is 82.94.164.162, how do you compute
what you should spell for 82.94.164.162/27? It's very easy to make a
mistake here, and doing the computation in your head is really not
something that should be necessary, given that a computer can get it
correct easily.

Regards,
Martin


From python-3000 at udmvt.ru  Wed Sep 16 10:26:30 2009
From: python-3000 at udmvt.ru (python-3000 at udmvt.ru)
Date: Wed, 16 Sep 2009 13:26:30 +0500
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090915193513.GA6606@jroger.in-berlin.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<20090915193513.GA6606@jroger.in-berlin.de>
Message-ID: <20090916082630.GF12810@ruber.office.udmvt.ru>

On Tue, Sep 15, 2009 at 09:35:13PM +0200, Sebastian Rittau wrote:
> On Tue, Sep 15, 2009 at 01:16:06PM -0400, Scott Dial wrote:
> 
> > I have to concur with the opinions above. I was very confused by the
> > following error:
> > 
> > >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0")
> > ...
> > ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not
> > a valid address (hint, it's probably a network)
> > 
> > Because, it *is* a address of a host on a network.
> 
> To me, 10.1.2.3/255.255.240.0 is not a host address, but specifies a
> network. I.e., 10.1.2.3/255.255.240.0 == 10.1.0.0/255.255.240.0 ==
> 10.1.35.200/20.
We shouldn't invent another bicycle here. It is a host address, that ALSO
specifies a network mask, that is not /32. Mask /32 just may be omitted when
written, but assumed, when stored by computer.

What you talk about is "address with mask". It is just IP address plus mask
information. IP address and mask just written together, one after another.
You MAY use mask to extract network part from an address, and have a "set" of
IP addresses, or range of IP addresses, or "network" as you say.
But you are not forced to do that with "MUST" statement, because you may also
use mask and IP address to compute widely used default value for a "broadcast"
address, or you may extract "host" part from the whole IP address.

There is no semantical difference between IPAddress for hosts and IPAddress for
networks, look at 'inet' type of PostgreSQL. Python shouldn't have separate
types for that - it is stupid, people will say 'python programmers are lamers'.
If you need only one IP address - you write AA.BB.CC.DD[/32] or /255.255.255.255
and the mask value may be omitted here. If you need to specify a larger range of
IP addresses, you specify another mask value, that is simple.


> 
> > >>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0")
> > 
> > But then, I was dumbfounded as to how I could get the gateway IP from
> > this IPNetwork object.
> 
> Well, you can't. There is no way to determine a gateway, without querying
> the network topology. This is clearly outside the scope of this module.
> The only two known host addresses of a network are the network address
> (10.1.0.0 in the example you gave) and the broadcast address (10.1.15.255).
> 
> > It took me a while to figure out that you can
> > iterate over IPNetwork instances:
> > 
> > >>> gateway = net[1]
> > 
> > I was then confused, because:
> > 
> > >>> print(type(gateway))
> > <class 'ipaddr.IPv4Address'>
> > 
> > Which sorta blew my mind.. I fully expected to receive an IPNetwork back
> > from that operation. It is unclear to me why the network information
> > gets chucked by that operation.
> 
> This makes perfect sense to me. An IP network consists of a list of IP
> addresses. Returning /32 networks seems kind of pointless to me.
Why? What differ IP Host address from IP Network address with mask /32?
Don't say "they have different python classes".
Well, I know, that an element is not a set containing only that element.
But that is about mathematical objects. What difference between /32 IP Network
and IP Host for Python?

Why IPNetwork can't be used where IPAddress is used?
When you only need IP address, you only use address part and ignore mask.
Why IPAddress can't be used where IPNetwork is accepted?
It is just IPNetwork with /32 mask.

It is counterintuitive for me, network administrator.

> 
> > I foresee having to work around that in
> > real applications by doing something obnoxious like:
> > 
> > >>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask))
> 
> But a gateway is not an IP address plus hostmask. A gateway is just a single
> IP address. What is the use of adding a hostmask to the gateway IP address
> or some other IP address inside the network?
Just to specify an address together with a mask, for short. It is widespread.
People are used to it.
When a gateway address written with a mask, it fully specifies you IP stack
configuration, it specifies (when computed as usual):
   1. netmask
   2. your subnetwork address
   3. host address
   4. address of default route
   5. subnetwork broadcast address
   6. I don't know, maybe something else?
   7. Oh, yes, it shows you the range of IP addresses, available for your use,
      if it was your subnetwork.
So, what is better to write on network diagrams or other documentation, those
five data pieces or just one gateway-with-a-mask?


> 
>  - Sebastian
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-3000%40udmvt.ru

-- 
Alexey S.

From python-3000 at udmvt.ru  Wed Sep 16 11:08:49 2009
From: python-3000 at udmvt.ru (Alexey S)
Date: Wed, 16 Sep 2009 14:08:49 +0500
Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for
	the	Python Standard Library
In-Reply-To: <d9047f780908191128p12107a1exc20dabc34ddff5de@mail.gmail.com>
References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com>
	<h6gjku$t2n$1@ger.gmane.org> <4A8C0267.5020408@wildenhain.de>
	<8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com>
	<Pine.LNX.4.64.0908191148530.30683@kimball.webabinitio.net>
	<8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com>
	<4A8C4264.2040708@trueblade.com>
	<d9047f780908191128p12107a1exc20dabc34ddff5de@mail.gmail.com>
Message-ID: <20090916090849.GH12810@ruber.office.udmvt.ru>

On Wed, Aug 19, 2009 at 02:28:38PM -0400, Glyph Lefkowitz wrote:
> On Wed, Aug 19, 2009 at 2:20 PM, Eric Smith <eric at trueblade.com> wrote:
> 
> 
> > I think using .network and .broadcast are pretty well understood to be the
> > [0] and [-1] of the network address block. I don't think we want to start
> > creating new terms or access patterns here.
> >
> > +1 on leaving .network and .broadcast as-is (including returning a
> > IPvXAddress object).
> >
> 
> -1.  I think 'network.number' or 'network.zero' is a lot clearer than
> 'network.network'.  Maybe '.broadcast' would be okay, as long as it *can* be
> adjusted for those unusual, or maybe even only hypothetical, networks where
> it is not the [-1].
Real life example: network with a /31 mask.
There are only two hosts: 0 and 1
first host configures the other's host as broadcast address and vice versa.
NOTE - broadcasts are different here!
Everything works, no one ever need to address "network" address, broadcasting
works as expected. It works well between two our routers.
What is wrong here? It just works for two Linuxes. It emulates point-to-point.
Well, some weird soft will not let you to configure that (on Windows?heh)? So
just let Python be off the shame list.

Another real life examples include /32 networks on PPP. Just a point-to-point.
No need for broadcasts and networks, a host just have one IP address and
send all traffic via point-to-point link, no addressing is required there.
This is a working dialup configuration, it works for me, it works for you, probably.
It is not weird, it is common, it is used for PPPoE, for ADSL, for dialup.


-- 
Alexey S.

From g.brandl at gmx.net  Wed Sep 16 11:53:48 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Sep 2009 11:53:48 +0200
Subject: [Python-Dev] displayhook behavior in pdb
Message-ID: <h8qcn8$f75$1@ger.gmane.org>

May I have a short vote on this issue:

   http://bugs.python.org/issue6903

In short, pdb (since 2.6) uses a separate displayhook in order to avoid
_ being reassigned (which screws up debugging apps that use _ as gettext).
In that displayhook, I did not add the suppression of printing None, as
it can be confusing to look at variables and get no output:

(Pdb) foo
1
(Pdb) bar
(Pdb)

(You could argue that this is what the "p" command is for though.)

Now in Python 3, where print is a function, if you call print in a loop
(e.g. to debug a list or dictionary, as it is advertised in the pdb docs
under the "alias" command), the output has the printed values interspersed
with "None"s.

Now, what is the lesser evil?

cheers,
Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From fuzzyman at voidspace.org.uk  Wed Sep 16 12:08:01 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Sep 2009 11:08:01 +0100
Subject: [Python-Dev] displayhook behavior in pdb
In-Reply-To: <h8qcn8$f75$1@ger.gmane.org>
References: <h8qcn8$f75$1@ger.gmane.org>
Message-ID: <4AB0B901.6090807@voidspace.org.uk>

Georg Brandl wrote:
> May I have a short vote on this issue:
>
>    http://bugs.python.org/issue6903
>
> In short, pdb (since 2.6) uses a separate displayhook in order to avoid
> _ being reassigned (which screws up debugging apps that use _ as gettext).
> In that displayhook, I did not add the suppression of printing None, as
> it can be confusing to look at variables and get no output:
>
> (Pdb) foo
> 1
> (Pdb) bar
> (Pdb)
>
> (You could argue that this is what the "p" command is for though.)
>
> Now in Python 3, where print is a function, if you call print in a loop
> (e.g. to debug a list or dictionary, as it is advertised in the pdb docs
> under the "alias" command), the output has the printed values interspersed
> with "None"s.
>
> Now, what is the lesser evil?
>
>   

IMO not showing the extraneous Nones is preferable.

Michael

> cheers,
> Georg
>
>   


-- 
http://www.ironpythoninaction.com/


From p.f.moore at gmail.com  Wed Sep 16 12:48:52 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 16 Sep 2009 11:48:52 +0100
Subject: [Python-Dev] displayhook behavior in pdb
In-Reply-To: <4AB0B901.6090807@voidspace.org.uk>
References: <h8qcn8$f75$1@ger.gmane.org> <4AB0B901.6090807@voidspace.org.uk>
Message-ID: <79990c6b0909160348n32edc221v8dc082a0c52eb357@mail.gmail.com>

2009/9/16 Michael Foord <fuzzyman at voidspace.org.uk>:
> Georg Brandl wrote:
>>
>> May I have a short vote on this issue:
>>
>> ? http://bugs.python.org/issue6903
>>
>> In short, pdb (since 2.6) uses a separate displayhook in order to avoid
>> _ being reassigned (which screws up debugging apps that use _ as gettext).
>> In that displayhook, I did not add the suppression of printing None, as
>> it can be confusing to look at variables and get no output:
>>
>> (Pdb) foo
>> 1
>> (Pdb) bar
>> (Pdb)
>>
>> (You could argue that this is what the "p" command is for though.)
>>
>> Now in Python 3, where print is a function, if you call print in a loop
>> (e.g. to debug a list or dictionary, as it is advertised in the pdb docs
>> under the "alias" command), the output has the printed values interspersed
>> with "None"s.
>>
>> Now, what is the lesser evil?
>>
>>
>
> IMO not showing the extraneous Nones is preferable.

I agree (although I don't use pdb, so my view shouldn't be given too
much weight...)
Paul.

From steve at pearwood.info  Wed Sep 16 12:52:59 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 16 Sep 2009 20:52:59 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
Message-ID: <200909162053.00354.steve@pearwood.info>

I've been skimming emails in this thread, since most of them go over my 
head and I have no current need for an ipaddress module. But one thing 
I noticed stands out and needs commenting on:

On Wed, 16 Sep 2009 11:05:26 am Peter Moody wrote:
> On Tue, Sep 15, 2009 at 6:02 PM, Eric Smith <eric at trueblade.com> 
wrote:
> > I completely agree. I don't know of any situation where I'd want a
> > network of "192.168.1.1/24" to be anything other than an error.
>
> when you're entering the address of your nic.

Eric is talking about a network. Peter replies by talking about an 
address. Perhaps I'm naive, but surely addresses aren't networks? If 
Peter's use-case is to treat addresses and networks interchangeably, I 
can't imagine that could be a good thing. Is it? What do the relevant 
RFCs say?

As an outsider to this argument, and judging from the lack of agreement 
here, it seems to me that some (many? most?) developers who have need 
of ipaddress functions are a little unclear about the difference, or 
lack thereof, between addresses and networks. Is it our aim to pander 
to such confusion, and have a module which will Just Work for such 
users, even at the risk of sometimes accepting invalid input. Or is it 
to have a module which is strict and forces the user to Do The Right 
Thing, even at the cost of being less easy to use?

For what it's worth, it seems to me that it would be better to have a 
strict module in the standard lib, and leave the DWIM code to third 
parties.



-- 
Steven D'Aprano

From fuzzyman at voidspace.org.uk  Wed Sep 16 12:59:57 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Sep 2009 11:59:57 +0100
Subject: [Python-Dev] Python Language Summit #2 in February
In-Reply-To: <20090916014810.GA318@andrew-kuchlings-macbook.local>
References: <20090916014810.GA318@andrew-kuchlings-macbook.local>
Message-ID: <4AB0C52D.5050608@voidspace.org.uk>

A.M. Kuchling wrote:
> PyCon 2010 will be February 19-21 2010 in Atlanta, Georgia (US).
> Van Lindberg, PyCon chair, has approved having another Python Language
> Summit on Thursday, February 18 2010.  The web page for it is
> <http://us.pycon.org/2010/about/summits/language/>
>
> The Python Language Summit is an invitation-only event for developers
> of Python implementations and the standard library, to discuss issues
> of common concern to all implementors.
>
> It will mostly be organized similarly to last year's event.  I'm going
> to drop the initial open discussion, which wasn't very useful, and
> we'll just have three 1.5-hour discussion sessions.  The fourth time
> slot will be left open for chatting, development, or whatever the
> attendees want to do.
>
> We therefore need to decide what those three sessions should be about.
> Please discuss on python-dev and hopefully we can arrive at some
> consensus on topics of reasonably wide current interest.  (See
> http://us.pycon.org/2009/about/summits/language/ for a reminder of
> last year's topics.)
>   

Given the long discussion on the stdlib-sig it seems like a discussion 
of the standard library would be useful. Potential topics include (some 
of which partially overlap each other):

* Clarifying the deprecation process of modules including whether we 
will *ever* remove deprecated modules
* Breaking the standard library into a separate development repository 
(which we decided to do at the last language summit but haven't yet - 
several developers are still keen to do this)
* Deprecating and removing more obsolete / unmaintained modules (I 
believe a new PEP on this will be coming soon)
* The goals of the standard library - some people want a 'dead' standard 
library (as in dead-stable) others want to see an actively improving 
standard library
* Bringing new best-of-breed modules into the standard library (and what 
to when they fully or partly duplicate existing functionality)

All the best,

Michael


> Feel free to raise discussions on other lists such as jython-dev,
> catalog-sig, or wherever, but please summarize the results here on
> python-dev; I won't see discussions on other lists.
>
> Like last time, invitations will be sent to the committers for Python,
> Jython, IronPython, PyPy, Pynie, plus a few extra people and projects.
> If you want to suggest another project or person who should be
> invited, please send me a private e-mail.
>
>
> Andrew M. Kuchling
> amk at amk.ca
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From p.f.moore at gmail.com  Wed Sep 16 13:04:07 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 16 Sep 2009 12:04:07 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909162053.00354.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info>
Message-ID: <79990c6b0909160404p4a1d5b5bu773b815b995f73dd@mail.gmail.com>

2009/9/16 Steven D'Aprano <steve at pearwood.info>:
> I've been skimming emails in this thread, since most of them go over my
> head and I have no current need for an ipaddress module.

Same here.

> As an outsider to this argument, and judging from the lack of agreement
> here, it seems to me that some (many? most?) developers who have need
> of ipaddress functions are a little unclear about the difference, or
> lack thereof, between addresses and networks. Is it our aim to pander
> to such confusion, and have a module which will Just Work for such
> users, even at the risk of sometimes accepting invalid input. Or is it
> to have a module which is strict and forces the user to Do The Right
> Thing, even at the cost of being less easy to use?
>
> For what it's worth, it seems to me that it would be better to have a
> strict module in the standard lib, and leave the DWIM code to third
> parties.

As a non-expert, I find the confusion between network, address,
address with mask, etc to be extremely unhelpful. I haven't looked to
confirm if it's a confusion that only exists in the discussion, or if
it's in the code as well. However, from my POV, I'd rather have a
module that made the concepts clear and unambiguous, effectively
educating me in the details of correct usage while I used it, rather
than one that catered to my muddled thinking and allowed me to remain
confused.

Of course, the discussion seems to imply that even the experts have a
confused view, so maybe I'm being too ambitious here :-)

Paul.

From ncoghlan at gmail.com  Wed Sep 16 13:10:33 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 16 Sep 2009 21:10:33 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909162053.00354.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB03909.3090608@trueblade.com>	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info>
Message-ID: <4AB0C7A9.3030001@gmail.com>

Steven D'Aprano wrote:
> I've been skimming emails in this thread, since most of them go over my 
> head and I have no current need for an ipaddress module. But one thing 
> I noticed stands out and needs commenting on:
> 
> On Wed, 16 Sep 2009 11:05:26 am Peter Moody wrote:
>> On Tue, Sep 15, 2009 at 6:02 PM, Eric Smith <eric at trueblade.com> 
> wrote:
>>> I completely agree. I don't know of any situation where I'd want a
>>> network of "192.168.1.1/24" to be anything other than an error.
>> when you're entering the address of your nic.
> 
> Eric is talking about a network. Peter replies by talking about an 
> address.

Martin explained it better in another part of the thread:
> if you know your address is 82.94.164.162, how do you compute
> what you should spell for 82.94.164.162/27? 

Or, to put it another way, given an arbitrary host in a network (e.g.
your own machine or the default gateway) and the netmask for that
network, calculate the network address.

With a "lax" parser on IPNetwork this is a trivial task - just create
the network object and then retrieve the network address from it.

If, on the other hand, IPNetwork demands that you already know the
network address before allowing you to create an IPNetwork object, then
you're pretty much out of luck - if all you have to work with are the IP
strings then this is actually a tricky calculation.

If the default IPNetwork constructor was made more strict, then this
functionality would have to be made available another way (probably as
an alternate constructor like IPNetwork.from_host_address rather than as
a boolean 'strict' option)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Wed Sep 16 13:41:43 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 11:41:43 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090915021130.4E57A600115@longblack.object-craft.com.au>
	<loom.20090915T162717-42@post.gmane.org>
	<Pine.LNX.4.64.0909151033270.6303@kimball.webabinitio.net>
	<4AAFCBD6.5090104@scottdial.com>
	<loom.20090915T200602-36@post.gmane.org>
	<87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp>
	<8517e9350909151146k77261851i35890b2f1b0e902a@mail.gmail.com>
	<4AAFE757.9010503@scottdial.com>
	<87pr9r1mp0.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <loom.20090916T133605-853@post.gmane.org>

Stephen J. Turnbull <stephen <at> xemacs.org> writes:
> 
> Rather, don't you want to just give IPv4Address an attribute
> 'network'?  This could then be filled in by the indexing method on
> IPv4Network.

It doesn't make sense to me. An address, conceptually, doesn't have a "network".
If I say "213.5.4.68", it isn't part of a specific network /a priori/.

Not only doesn't it make sense to me, but it creates ambiguities.
Do two similar addresses, but with different "networks", compare and hash
equally? Both answers are unsatisfactory and will create nasty surprises for
users.

I don't see what's difficult in remembering the network object by yourself if
you need it. Python is not Java, it has lightweight syntax, useful datatypes and
notations (e.g. tuples). You don't need to have every use case taken care of by
a dedicated library API.

(to make an analogy: if I take an item from a list, it doesn't hold a reference
to the list just in case it might be practical to do so)

Regards

Antoine.



From mcguire at google.com  Tue Sep 15 20:44:19 2009
From: mcguire at google.com (Jake McGuire)
Date: Tue, 15 Sep 2009 11:44:19 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
	<8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>
	<77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>
	<8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
Message-ID: <77c780b40909151144u17731a44na7b60e3f9debcd8a@mail.gmail.com>

On Tue, Sep 15, 2009 at 11:36 AM, Peter Moody <peter at hda3.com> wrote:

> On Tue, Sep 15, 2009 at 11:33 AM, Jake McGuire <mcguire at google.com> wrote:
> > On Tue, Sep 15, 2009 at 10:36 AM, Peter Moody <peter at hda3.com> wrote:
> >>
> >> On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire <mcguire at google.com>
> wrote:
> >> > On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum <guido at python.org>
> >> > wrote:
> >> >>
> >> >> What's the opinion of the other interested party or parties? I don't
> >> >> want a repeat of the events last time, where we had to pull it at the
> >> >> last time because there hadn't been enough discussion.
> >> >
> >> > How many other people are using this library?  I think it's hard to
> give
> >> > really useful feedback on an API without using it for some non-trivial
> >> > task,
> >> > but maybe other people don't have this problem.
> >> > -jake
> >>
> >> 188 (check that, 190) people have downloaded the 2.0 release in the
> >> last week (numbers publicly available from the code.google.com). I
> >> can't tell you how many (if any) have downloaded it via svn.
> >
> > Downloading and using are not the same thing.
>
> Correct, but there is a strong positive correlation between the two.
> If you have a better method for determining what you would consider an
> appropriate level of usage, I'm all ears.
>

Put something on the project page (or download page if possible) saying
"ipaddr is being considered for inclusion in the Python standard library.
 We want to make sure it meets your needs, but we need you to tell us.  If
you use ipaddr and like it, please let us know on ip-addr-dev."

I dunno, maybe it's too much work.  But no one else seems to have an opinion
strong enough to share, at least not at this point.

-jake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090915/7ef30687/attachment.htm>

From mcguire at google.com  Tue Sep 15 20:58:12 2009
From: mcguire at google.com (Jake McGuire)
Date: Tue, 15 Sep 2009 11:58:12 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <77c780b40909151144u17731a44na7b60e3f9debcd8a@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
	<8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>
	<77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>
	<8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
	<77c780b40909151144u17731a44na7b60e3f9debcd8a@mail.gmail.com>
Message-ID: <77c780b40909151158y17f20b65je41f934de35c0a38@mail.gmail.com>

On Tue, Sep 15, 2009 at 11:44 AM, Jake McGuire <mcguire at google.com> wrote:

> On Tue, Sep 15, 2009 at 11:36 AM, Peter Moody <peter at hda3.com> wrote:
>
>> On Tue, Sep 15, 2009 at 11:33 AM, Jake McGuire <mcguire at google.com>
>> wrote:
>> > On Tue, Sep 15, 2009 at 10:36 AM, Peter Moody <peter at hda3.com> wrote:
>> >>
>> >> On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire <mcguire at google.com>
>> wrote:
>> >> > On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum <guido at python.org>
>> >> > wrote:
>> >> >>
>> >> >> What's the opinion of the other interested party or parties? I don't
>> >> >> want a repeat of the events last time, where we had to pull it at
>> the
>> >> >> last time because there hadn't been enough discussion.
>> >> >
>> >> > How many other people are using this library?  I think it's hard to
>> give
>> >> > really useful feedback on an API without using it for some
>> non-trivial
>> >> > task,
>> >> > but maybe other people don't have this problem.
>> >> > -jake
>> >>
>> >> 188 (check that, 190) people have downloaded the 2.0 release in the
>> >> last week (numbers publicly available from the code.google.com). I
>> >> can't tell you how many (if any) have downloaded it via svn.
>> >
>> > Downloading and using are not the same thing.
>>
>> Correct, but there is a strong positive correlation between the two.
>> If you have a better method for determining what you would consider an
>> appropriate level of usage, I'm all ears.
>>
>
> Put something on the project page (or download page if possible) saying
> "ipaddr is being considered for inclusion in the Python standard library.
>  We want to make sure it meets your needs, but we need you to tell us.  If
> you use ipaddr and like it, please let us know on ip-addr-dev."
>
> I dunno, maybe it's too much work.  But no one else seems to have an
> opinion strong enough to share, at least not at this point.
>

To clarify: there will always be some group of people happy to tell you that
whatever you made is crap and that it should have been done differently.
 There may be many more people out there who think that what you did is
great.  But those people probably don't read python-dev so their voices
don't get heard when it comes to deciding what to do with PEP 3144.

The easiest way to get their voices heard is to ask them to speak up, and
the one place you know they visited was the ipaddr page on code.google.com,
so you may as well ask them there.

-jake
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090915/dbc7bc61/attachment.htm>

From solipsis at pitrou.net  Wed Sep 16 13:53:28 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 11:53:28 +0000 (UTC)
Subject: [Python-Dev] Python Language Summit #2 in February
References: <20090916014810.GA318@andrew-kuchlings-macbook.local>
	<4AB0C52D.5050608@voidspace.org.uk>
Message-ID: <loom.20090916T134328-925@post.gmane.org>

Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
> 
> Given the long discussion on the stdlib-sig it seems like a discussion 
> of the standard library would be useful. Potential topics include (some 
> of which partially overlap each other):
> 
[snip]

Let me suggest the following additional point:

* Clarifying maintainership. What is a maintainer, what can be expected from him
and what is *not* expected from him. Whether maintainers are, or are not,
"owners". What the relationship of other core developers to maintainers should
be, and should not be.

Of course, all this will perhaps have been discussed before the summit.

Regards

Antoine.



From fuzzyman at voidspace.org.uk  Wed Sep 16 13:58:44 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 16 Sep 2009 12:58:44 +0100
Subject: [Python-Dev] Python Language Summit #2 in February
In-Reply-To: <loom.20090916T134328-925@post.gmane.org>
References: <20090916014810.GA318@andrew-kuchlings-macbook.local>	<4AB0C52D.5050608@voidspace.org.uk>
	<loom.20090916T134328-925@post.gmane.org>
Message-ID: <4AB0D2F4.3080007@voidspace.org.uk>

Antoine Pitrou wrote:
> Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
>   
>> Given the long discussion on the stdlib-sig it seems like a discussion 
>> of the standard library would be useful. Potential topics include (some 
>> of which partially overlap each other):
>>
>>     
> [snip]
>
> Let me suggest the following additional point:
>
> * Clarifying maintainership. What is a maintainer, what can be expected from him
> and what is *not* expected from him. Whether maintainers are, or are not,
> "owners". What the relationship of other core developers to maintainers should
> be, and should not be.
>   

Yup, good addition.

> Of course, all this will perhaps have been discussed before the summit.
>
>   

When we add discussing the relationship of the other implementations to 
the standard library I think there is plenty to discuss, even if some of 
the points are clearer or closer to being settled (or as settled as they 
can possibly get) by the time of the summit.

Michael

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From barry at python.org  Wed Sep 16 14:07:00 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 16 Sep 2009 08:07:00 -0400
Subject: [Python-Dev] Python Language Summit #2 in February
In-Reply-To: <loom.20090916T134328-925@post.gmane.org>
References: <20090916014810.GA318@andrew-kuchlings-macbook.local>
	<4AB0C52D.5050608@voidspace.org.uk>
	<loom.20090916T134328-925@post.gmane.org>
Message-ID: <9A82D87C-9A95-4338-8661-2655F3DB0896@python.org>

On Sep 16, 2009, at 7:53 AM, Antoine Pitrou wrote:

> Of course, all this will perhaps have been discussed before the  
> summit.

Sure, but not resolved.  :)

I do think stdlib evolution should be high on the list of topics.

-B

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090916/2e75f1cd/attachment.pgp>

From exarkun at twistedmatrix.com  Wed Sep 16 14:50:08 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 16 Sep 2009 12:50:08 -0000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB0C7A9.3030001@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info>
	<4AB0C7A9.3030001@gmail.com>
Message-ID: <20090916125008.12215.453222785.divmod.xquotient.31@localhost.localdomain>

On 11:10 am, ncoghlan at gmail.com wrote:
>Steven D'Aprano wrote:
>>I've been skimming emails in this thread, since most of them go over 
>>my
>>head and I have no current need for an ipaddress module. But one thing
>>I noticed stands out and needs commenting on:
>>
>>On Wed, 16 Sep 2009 11:05:26 am Peter Moody wrote:
>>>On Tue, Sep 15, 2009 at 6:02 PM, Eric Smith <eric at trueblade.com>
>>wrote:
>>>>I completely agree. I don't know of any situation where I'd want a
>>>>network of "192.168.1.1/24" to be anything other than an error.
>>>when you're entering the address of your nic.
>>
>>Eric is talking about a network. Peter replies by talking about an
>>address.
>
>Martin explained it better in another part of the thread:
>>if you know your address is 82.94.164.162, how do you compute
>>what you should spell for 82.94.164.162/27?
>
>Or, to put it another way, given an arbitrary host in a network (e.g.
>your own machine or the default gateway) and the netmask for that
>network, calculate the network address.
>
>With a "lax" parser on IPNetwork this is a trivial task - just create
>the network object and then retrieve the network address from it.
>
>If, on the other hand, IPNetwork demands that you already know the
>network address before allowing you to create an IPNetwork object, then
>you're pretty much out of luck - if all you have to work with are the 
>IP
>strings then this is actually a tricky calculation.
>
>If the default IPNetwork constructor was made more strict, then this
>functionality would have to be made available another way (probably as
>an alternate constructor like IPNetwork.from_host_address rather than 
>as
>a boolean 'strict' option)

This seems to be the right solution to me, particularly the use of an 
alternate constructor rather than an ambiguously named flag.

Jean-Paul

From barry at python.org  Wed Sep 16 14:52:47 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 16 Sep 2009 08:52:47 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <F10025B7-8AEA-4FD2-88C4-231F8BD77458@mac.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-C736AE.09230710092009@ger.gmane.org>
	<F10025B7-8AEA-4FD2-88C4-231F8BD77458@mac.com>
Message-ID: <B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>

On Sep 15, 2009, at 9:53 AM, Ronald Oussoren wrote:

> The IDLE issue is IMHO a release blocker, as is issue 6851.

So that leaves 4 release blockers for 2.6.3.  Three of these are  
assigned to Ronald.  Ronald are you sure you will have time to fix  
these by then?  The one I'm still uncertain on is the IDLE hang.  If  
this only affects OS X 10.6 and there is no progress on it by the time  
2.6.3 is otherwise ready, I'm willing to knock this down in severity.

Martin, you have the other showstopper bug for 2.6.3.  Do you think  
you will be able to fix that one in time?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090916/9c707c16/attachment.pgp>

From ncoghlan at gmail.com  Wed Sep 16 16:35:31 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 17 Sep 2009 00:35:31 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090916125008.12215.453222785.divmod.xquotient.31@localhost.localdomain>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB03909.3090608@trueblade.com>	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>	<200909162053.00354.steve@pearwood.info>	<4AB0C7A9.3030001@gmail.com>
	<20090916125008.12215.453222785.divmod.xquotient.31@localhost.localdomain>
Message-ID: <4AB0F7B3.2090600@gmail.com>

exarkun at twistedmatrix.com wrote:
> On 11:10 am, ncoghlan at gmail.com wrote:
>> If the default IPNetwork constructor was made more strict, then this
>> functionality would have to be made available another way (probably as
>> an alternate constructor like IPNetwork.from_host_address rather than as
>> a boolean 'strict' option)
> 
> This seems to be the right solution to me, particularly the use of an
> alternate constructor rather than an ambiguously named flag.

Particular since someone pointed out that the "Network" object was
remembering which IP address happened to be passed to the constructor
(which is "Address with Mask" behaviour, as has been mentioned).

That said, I've been trying to refrain from commenting on specifics of
the API, as I haven't tried to use it for anything.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From eric at trueblade.com  Wed Sep 16 16:42:31 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 16 Sep 2009 10:42:31 -0400
Subject: [Python-Dev] Python Language Summit #2 in February
In-Reply-To: <20090916014810.GA318@andrew-kuchlings-macbook.local>
References: <20090916014810.GA318@andrew-kuchlings-macbook.local>
Message-ID: <4AB0F957.8020300@trueblade.com>

A.M. Kuchling wrote:
> We therefore need to decide what those three sessions should be about.
> Please discuss on python-dev and hopefully we can arrive at some
> consensus on topics of reasonably wide current interest.  (See
> http://us.pycon.org/2009/about/summits/language/ for a reminder of
> last year's topics.)

I'd like to have a discussion on the future of 2.x, at least as far as 
core development goes. Supporting both 2.x and 3.x cuts my contributions 
in half, at best. For example, in order to keep 2.x and 3.x nearly in 
sync, Mark and I back-ported the API's for short-float-repr to 2.x, even 
though 2.x itself didn't see any benefit from that work (well, at least 
not much of a benefit).

My initial thought is let's stop after 2.7 (if we even release 2.7). But 
it all depends on uptake of 3.x, and what stopping core development 
means to the community.

Can we discuss this at the language summit?

Eric.

From guido at python.org  Wed Sep 16 17:00:04 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Sep 2009 08:00:04 -0700
Subject: [Python-Dev] displayhook behavior in pdb
In-Reply-To: <79990c6b0909160348n32edc221v8dc082a0c52eb357@mail.gmail.com>
References: <h8qcn8$f75$1@ger.gmane.org> <4AB0B901.6090807@voidspace.org.uk> 
	<79990c6b0909160348n32edc221v8dc082a0c52eb357@mail.gmail.com>
Message-ID: <ca471dc20909160800x7f0c4ac6x1d382a761b1b3c4f@mail.gmail.com>

On Wed, Sep 16, 2009 at 3:48 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> 2009/9/16 Michael Foord <fuzzyman at voidspace.org.uk>:
> > Georg Brandl wrote:
> >>
> >> May I have a short vote on this issue:
> >>
> >>   http://bugs.python.org/issue6903
> >>
> >> In short, pdb (since 2.6) uses a separate displayhook in order to avoid
> >> _ being reassigned (which screws up debugging apps that use _ as
> gettext).
> >> In that displayhook, I did not add the suppression of printing None, as
> >> it can be confusing to look at variables and get no output:
> >>
> >> (Pdb) foo
> >> 1
> >> (Pdb) bar
> >> (Pdb)
> >>
> >> (You could argue that this is what the "p" command is for though.)
> >>
> >> Now in Python 3, where print is a function, if you call print in a loop
> >> (e.g. to debug a list or dictionary, as it is advertised in the pdb docs
> >> under the "alias" command), the output has the printed values
> interspersed
> >> with "None"s.
> >>
> >> Now, what is the lesser evil?
> >>
> >>
> >
> > IMO not showing the extraneous Nones is preferable.
>
> I agree (although I don't use pdb, so my view shouldn't be given too
> much weight...)
>

I *do* use pdb a lot, and I agree that the new behavior is weird.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090916/fa30e66a/attachment-0001.htm>

From rdmurray at bitdance.com  Wed Sep 16 17:25:09 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 16 Sep 2009 11:25:09 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090916125008.12215.453222785.divmod.xquotient.31@localhost.localdomain>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<20090916125008.12215.453222785.divmod.xquotient.31@localhost.localdomain>
Message-ID: <Pine.LNX.4.64.0909161123480.17020@kimball.webabinitio.net>

On Wed, 16 Sep 2009 at 12:50, exarkun at twistedmatrix.com wrote:
> On 11:10 am, ncoghlan at gmail.com wrote:
>> Or, to put it another way, given an arbitrary host in a network (e.g.
>> your own machine or the default gateway) and the netmask for that
>> network, calculate the network address.
>> 
>> With a "lax" parser on IPNetwork this is a trivial task - just create
>> the network object and then retrieve the network address from it.
>> 
>> If, on the other hand, IPNetwork demands that you already know the
>> network address before allowing you to create an IPNetwork object, then
>> you're pretty much out of luck - if all you have to work with are the IP
>> strings then this is actually a tricky calculation.
>> 
>> If the default IPNetwork constructor was made more strict, then this
>> functionality would have to be made available another way (probably as
>> an alternate constructor like IPNetwork.from_host_address rather than as
>> a boolean 'strict' option)
>
> This seems to be the right solution to me, particularly the use of an 
> alternate constructor rather than an ambiguously named flag.

+1

--David

From rdmurray at bitdance.com  Wed Sep 16 17:32:25 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 16 Sep 2009 11:32:25 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <79990c6b0909160404p4a1d5b5bu773b815b995f73dd@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info>
	<79990c6b0909160404p4a1d5b5bu773b815b995f73dd@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909161125190.17020@kimball.webabinitio.net>

On Wed, 16 Sep 2009 at 12:04, Paul Moore wrote:
> Of course, the discussion seems to imply that even the experts have a
> confused view, so maybe I'm being too ambitious here :-)

Part of the problem, as we discovered in the last go-round on
ipaddr, is that there are two types of experts:  those who primarily
deal with networks and for whom host addresses are just a special
case, and those who primarily deal with host addresses, for whom
networks are relevant only in certain contexts.

I hope we are converging on something that works for both.  That should
certainly be a goal, since the fact that it didn't was what scuttled
inclusion last time.

--David

From g.brandl at gmx.net  Wed Sep 16 18:35:59 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Sep 2009 18:35:59 +0200
Subject: [Python-Dev] displayhook behavior in pdb
In-Reply-To: <ca471dc20909160800x7f0c4ac6x1d382a761b1b3c4f@mail.gmail.com>
References: <h8qcn8$f75$1@ger.gmane.org> <4AB0B901.6090807@voidspace.org.uk>
	<79990c6b0909160348n32edc221v8dc082a0c52eb357@mail.gmail.com>
	<ca471dc20909160800x7f0c4ac6x1d382a761b1b3c4f@mail.gmail.com>
Message-ID: <h8r49d$3f5$1@ger.gmane.org>

Guido van Rossum schrieb:

>     >> Now, what is the lesser evil?
>     >>
>     >>
>     >
>     > IMO not showing the extraneous Nones is preferable.
> 
>     I agree (although I don't use pdb, so my view shouldn't be given too
>     much weight...)
> 
> 
> I *do* use pdb a lot, and I agree that the new behavior is weird. 

Okay, I've fixed it in trunk and will backport soon.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From brett at python.org  Wed Sep 16 21:57:28 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 16 Sep 2009 12:57:28 -0700
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> 
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> 
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> 
	<nad-C736AE.09230710092009@ger.gmane.org>
	<F10025B7-8AEA-4FD2-88C4-231F8BD77458@mac.com> 
	<B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>
Message-ID: <bbaeab100909161257p239228e3sf2b5b2e6d1974888@mail.gmail.com>

On Wed, Sep 16, 2009 at 05:52, Barry Warsaw <barry at python.org> wrote:
> On Sep 15, 2009, at 9:53 AM, Ronald Oussoren wrote:
>
>> The IDLE issue is IMHO a release blocker, as is issue 6851.
>
> So that leaves 4 release blockers for 2.6.3. ?Three of these are assigned to
> Ronald. ?Ronald are you sure you will have time to fix these by then? ?The
> one I'm still uncertain on is the IDLE hang. ?If this only affects OS X 10.6
> and there is no progress on it by the time 2.6.3 is otherwise ready, I'm
> willing to knock this down in severity.
>
> Martin, you have the other showstopper bug for 2.6.3. ?Do you think you will
> be able to fix that one in time?

That issue doesn't require Martin specifically. I just needed to know
if changing the BaseException struct would be bad (Georg answered w/
"yes"). It will probably fall on to me (or Georg if his proposed
solution pans out) to get this fixed.

-Brett

From ronaldoussoren at mac.com  Wed Sep 16 22:28:34 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 16 Sep 2009 22:28:34 +0200
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-C736AE.09230710092009@ger.gmane.org>
	<F10025B7-8AEA-4FD2-88C4-231F8BD77458@mac.com>
	<B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>
Message-ID: <C5347D93-F54C-49C0-963B-5FD95BE7FF9C@mac.com>


On 16 Sep, 2009, at 14:52, Barry Warsaw wrote:

> On Sep 15, 2009, at 9:53 AM, Ronald Oussoren wrote:
>
>> The IDLE issue is IMHO a release blocker, as is issue 6851.
>
> So that leaves 4 release blockers for 2.6.3.  Three of these are  
> assigned to Ronald.  Ronald are you sure you will have time to fix  
> these by then?  The one I'm still uncertain on is the IDLE hang.  If  
> this only affects OS X 10.6 and there is no progress on it by the  
> time 2.6.3 is otherwise ready, I'm willing to knock this down in  
> severity.

I don't know for sure if I'll be able to fix these issues, I'll  
basicly just have time to work on this during the weekend.

* IDLE hang on OSX 10.6:   I don't really know where to start looking,  
this may have something to do with Tk-Cocoa (which is used on 10.6 and  
not on earlier releases). There is a patch for Tk-Cocoa support on the  
tracker, but I haven't had time yet to fully test that.

* Compile error for ctypes on 10.6: Should be easy to fix. Apple has  
released the sources of the copy of libffi included with 10.6.1, I'll  
use that to work on a patch.

* Crash with urllib and threads on OSX 10.6: This probably requires  
rewriting some code using ctypes in C. The crash seems to occur when  
CoreFoundation tries to initialise itself after being loaded on a  
secondairy thread by ctypes. To make matters worse the ctypes type  
annotations in urllib are not correct for 64-bit code (but the crash  
happens in 32-bit mode as well)

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090916/89546ea9/attachment.bin>

From thomas at python.org  Wed Sep 16 22:34:36 2009
From: thomas at python.org (Thomas Wouters)
Date: Wed, 16 Sep 2009 22:34:36 +0200
Subject: [Python-Dev] pthreads, fork, import, and execvp
In-Reply-To: <4AA8249C.9010707@gmail.com>
References: <d2115f4f0605090639h72570784k1f4ef0802b5a8d51@mail.gmail.com>
	<446C6FC9.5060205@gmail.com> <446CB69E.4090609@v.loewis.de>
	<9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com>
	<3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com>
	<9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com>
	<9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com>
	<52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com>
	<9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com>
	<4AA8249C.9010707@gmail.com>
Message-ID: <9e804ac0909161334x3d5d8569je67fd484c1747c7f@mail.gmail.com>

On Wed, Sep 9, 2009 at 23:56, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Thomas Wouters wrote:
> >     Your idea of making this an API called a 'fork lock' or something
> >     sounds good (though I think it needs a better name.  PyBeforeFork &
> >     PyAfterFork?).  The subprocess module, for example, disables garbage
> >     collection before forking and restores it afterwards to avoid
> >     http://bugs.python.org/issue1336.  That type of thing could also be
> >     done in such a function.
> >
> >
> > Unfortunately it's rather hard to make those functions work correctly
> > with the current API -- we can't provide functions you can just use as
> > arguments to pthread_atfork because the global interpreter lock is not
> > re-entrant and we have no way of testing whether the current thread
> > holds the GIL.
>
> I thought "make sure I have the GIL, either by already having it or
> waiting for it if I don't yet have it" was the entire point of the
> PyGILState_Ensure() API? [1]
>

Hm, yeah. For some reason I was certain it was inappropriate, back when I
was trying to create a pthread_atfork-friendly set of functions. At the time
I was also hip-deep in the awfulness of Python/thread*.c and its unsafe
punning and unwarranted assumptions, so I may have overreacted. I added this
as a feature-request issue ( http://bugs.python.org/issue6923 ) and will
look at it some more.

In the mean time, I fixed the biggest source of issues by checking in the
change to make at least calls to fork() made by Python be safe, also
backported to 2.6.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090916/6e4fe89f/attachment-0001.htm>

From solipsis at pitrou.net  Wed Sep 16 22:35:30 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 16 Sep 2009 20:35:30 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
Message-ID: <h8ri6i$ip5$1@ger.gmane.org>

Le Mon, 14 Sep 2009 09:44:12 -0700, Peter Moody a ?crit?:
> Folks, Guido,
> 
> I believe PEP 3144 is ready for your review.  When you get a chance, can
> you take a look/make a pronouncement?

Besides what has already been said in the thread, I have a bunch of 
comments:

    It should be noted
    that __len__ doesn't work as expected since python internals has this
    limited to a 32 bit integer and it would need to be at least 128 bits 
to
    work with IPV6.

You should clarify what it means: does the result get truncated, or is an 
error thrown when it can't fit inside an int?

    1. all IP addresses and networks, both IPv4 and IPv6. (IPAddrBase)

    2. all IP addresses of both versions. (BaseIP)

    3. all IP networks of both version. (BaseNet)

    4. all IPv4 objects, both addresses and networks. (BaseV4)

    5. all IPv6 objects, both addresses and networks. (BaseV6)

Should those base classes be exposed publically? It may make life more 
difficult if an alternate implementation (say, a C accelerator) wants to 
adopt a different code sharing strategy.

If they are not to be part of the public API, their names should be 
prefixed with an underscore.



From peter at hda3.com  Wed Sep 16 22:48:06 2009
From: peter at hda3.com (Peter Moody)
Date: Wed, 16 Sep 2009 13:48:06 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <h8ri6i$ip5$1@ger.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<h8ri6i$ip5$1@ger.gmane.org>
Message-ID: <8517e9350909161348h63462089x7743826b91d3689@mail.gmail.com>

On Wed, Sep 16, 2009 at 1:35 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Mon, 14 Sep 2009 09:44:12 -0700, Peter Moody a ?crit?:
>> Folks, Guido,
>>
>> I believe PEP 3144 is ready for your review. ?When you get a chance, can
>> you take a look/make a pronouncement?
>
> Besides what has already been said in the thread, I have a bunch of
> comments:
>
> ? ?It should be noted
> ? ?that __len__ doesn't work as expected since python internals has this
> ? ?limited to a 32 bit integer and it would need to be at least 128 bits
> to
> ? ?work with IPV6.
>
> You should clarify what it means: does the result get truncated, or is an
> error thrown when it can't fit inside an int?

I'll clarify this, but it looks like this:

>>> len(xrange(0xFFFFFFFFFFFFFFFFFFFFFFFFF))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: long int too large to convert to int
>>>

basically, __len__(self): can only return an int. At least that's what
I recall from the discussion.

> ? ?1. all IP addresses and networks, both IPv4 and IPv6. (IPAddrBase)
>
> ? ?2. all IP addresses of both versions. (BaseIP)
>
> ? ?3. all IP networks of both version. (BaseNet)
>
> ? ?4. all IPv4 objects, both addresses and networks. (BaseV4)
>
> ? ?5. all IPv6 objects, both addresses and networks. (BaseV6)
>
> Should those base classes be exposed publically? It may make life more
> difficult if an alternate implementation (say, a C accelerator) wants to
> adopt a different code sharing strategy.
>
> If they are not to be part of the public API, their names should be
> prefixed with an underscore.

makes sense. I see no need for them to be public. private they go.

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From tjreedy at udel.edu  Wed Sep 16 23:50:39 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 16 Sep 2009 17:50:39 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909161348h63462089x7743826b91d3689@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<h8ri6i$ip5$1@ger.gmane.org>
	<8517e9350909161348h63462089x7743826b91d3689@mail.gmail.com>
Message-ID: <h8rmje$30p$1@ger.gmane.org>

Peter Moody wrote:
> On Wed, Sep 16, 2009 at 1:35 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Le Mon, 14 Sep 2009 09:44:12 -0700, Peter Moody a ?crit :
>>> Folks, Guido,
>>>
>>> I believe PEP 3144 is ready for your review.  When you get a chance, can
>>> you take a look/make a pronouncement?
>> Besides what has already been said in the thread, I have a bunch of
>> comments:
>>
>>    It should be noted
>>    that __len__ doesn't work as expected since python internals has this
>>    limited to a 32 bit integer and it would need to be at least 128 bits
>> to
>>    work with IPV6.
>>
>> You should clarify what it means: does the result get truncated, or is an
>> error thrown when it can't fit inside an int?
> 
> I'll clarify this, but it looks like this:
> 
>>>> len(xrange(0xFFFFFFFFFFFFFFFFFFFFFFFFF))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> OverflowError: long int too large to convert to int
> 
> basically, __len__(self): can only return an int. At least that's what
> I recall from the discussion.

FWIW, the limitation remains in Py3, with a modified error msg.
 >>> len(range(0xFFFFFFFFFFFFFFFFFFFFFFFFF))
Traceback (most recent call last):
   File "<pyshell#1>", line 1, in <module>
     len(range(0xFFFFFFFFFFFFFFFFFFFFFFFFF))
OverflowError: Python int too large to convert to C ssize_t


From greg.ewing at canterbury.ac.nz  Wed Sep 16 23:59:47 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Sep 2009 09:59:47 +1200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB0C7A9.3030001@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
Message-ID: <4AB15FD3.5040804@canterbury.ac.nz>

Nick Coghlan wrote:

> Or, to put it another way, given an arbitrary host in a network (e.g.
> your own machine or the default gateway) and the netmask for that
> network, calculate the network address.

Some people have claimed that the gateway address of a
network isn't necessarily the zero address in that network.
If that's true, then you *can't* calculate the network
address from a host address and a netmask -- there isn't
enough information. Furthermore, an IPNetwork object
needs to be able to represent a network address whose
address part contains bits that aren't in the mask.

-- 
Greg

From eric at trueblade.com  Thu Sep 17 00:04:32 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 16 Sep 2009 18:04:32 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
Message-ID: <rqtynlwmdy2yd4u242j47pj3.1253138665917@email.android.com>


Eric.

"Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote:

>Nick Coghlan wrote:
>
>> Or, to put it another way, given an arbitrary host in a network (e.g.
>> your own machine or the default gateway) and the netmask for that
>> network, calculate the network address.
>
>Some people have claimed that the gateway address of a
>network isn't necessarily the zero address in that network.
>If that's true, then you *can't* calculate the network
>address from a host address and a netmask -- there isn't
>enough information. Furthermore, an IPNetwork object
>needs to be able to represent a network address whose
>address part contains bits that aren't in the mask.

I don't see why that would be considered a network address, then. It sounds to me like that's a host address plus a netmask.

From cs at zip.com.au  Thu Sep 17 00:27:12 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 17 Sep 2009 08:27:12 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AAFCBD6.5090104@scottdial.com>
References: <4AAFCBD6.5090104@scottdial.com>
Message-ID: <20090916222712.GA2794@cskk.homeip.net>

On 15Sep2009 13:16, Scott Dial <scott+python-dev at scottdial.com> wrote:
| It just happened that I needed a tool today to calculate the gateway IP
| for an interface, and I took it as an opportunity to try out this ipaddr
| module. I'll attempt to relate my experience below...
| 
| I have to concur with the opinions above. I was very confused by the
| following error:
| 
| >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0")
| ...
| ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not
| a valid address (hint, it's probably a network)

It sure looks like a network to me, too.

| Because, it *is* a address of a host on a network. I gave in and said:

No, it is an IP address plus a network mask, and that notation is
generally used to describe a network (base IP of network plus the mask
to indicate its range).

| >>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0")
| 
| But then, I was dumbfounded as to how I could get the gateway IP from
| this IPNetwork object. It took me a while to figure out that you can
| iterate over IPNetwork instances:
| 
| >>> gateway = net[1]

Shrug. It is a common convention for a network's _default_ gateway to be the
network's base address + 1. But certainly not necessary. If that convention
is also your convention, then good.

| I was then confused, because:
| 
| >>> print(type(gateway))
| <class 'ipaddr.IPv4Address'>
| 
| Which sorta blew my mind.. I fully expected to receive an IPNetwork back
| from that operation.

I have no idea why you would expect that. A gateway is just the IP
_address_ of a host that will route packets to other networks. It is
inherently an address, and not a network.

Remember that a network (well, a host on a network) may have multiple
gateways; usually a single "default" gateway, and potentially others for
special routes to particular other networks. There's no need to have
just one gateway address.

Perhaps you are thinking of routing table entries (such as recited by
"netstat -r" on a UNIX box). They normally consist of a network
specification (base-addr/network-mask) and a gateway _address_.

| It is unclear to me why the network information
| gets chucked by that operation. I foresee having to work around that in
| real applications by doing something obnoxious like:
| 
| >>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask))
| 
| But since I only actually needed the IP address for the gateway, it
| fulfilled my needs.

And that is a clue. You only need the gateway IP address in your needs
because a gateway _is_ an IP address.

It is quite possible you are thinking the word "gateway", but actually
thinking about a "route".

| In the end, I found the names IPNetwork/IPAddress and their
| instantiations confusing.

I really believe that you are confused in this instance, not the module.

| ISTM that IPNetwork is overloaded and plays
| two roles of being an IPNetwork and being an IPAddressWithNetwork. And
| finally, it's unclear to me why iterating over a IPNetwork would not
| produce a sequence of IPNetwork(/IPAddressWithNetwork) objects.
| 
| ISTM that if I started with an IPNetwork object, the API should always
| return IPNetwork objects.

Not for methods that produce addresses!

| If I want just an IPAddress object, then I can
| always fetch the "ip" attribute to get that. Perhaps there is some
| compelling conceptual argument as to why this is not correct, but it
| seems like the API destroys information needlessly.

I really think you have the wrong mental image of what these terms mean.

Nothing in your examples seems surprising, confusing or wrong to me.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

From rdmurray at bitdance.com  Thu Sep 17 00:27:28 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 16 Sep 2009 18:27:28 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB15FD3.5040804@canterbury.ac.nz>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
Message-ID: <Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 09:59, Greg Ewing wrote:
> Nick Coghlan wrote:
>
>>  Or, to put it another way, given an arbitrary host in a network (e.g.
>>  your own machine or the default gateway) and the netmask for that
>>  network, calculate the network address.
>
> Some people have claimed that the gateway address of a
> network isn't necessarily the zero address in that network.
> If that's true, then you *can't* calculate the network
> address from a host address and a netmask -- there isn't
> enough information. Furthermore, an IPNetwork object
> needs to be able to represent a network address whose
> address part contains bits that aren't in the mask.

A 'gateway' is an IP (ie: host) in a network to which you route packets
for delivery elsewhere.  That's a very different thing from the 'network
number' (or 'network address' or 'network zero' or 'network identifier'
or whatever you want to call it).  Any IP in a network can be a gateway
IP, and in many networks there is more than one gateway (and thus the
need for routing tables...)

A network is conventionally represented by an IP address in which the
bits corresponding to the one bits in the netmask are set to zero, plus
the netmask.  That IP with the bits set to zero is the 'network number'
or zero for the network, and it is almost never used as the address for a
host in the network.  Thus it would be very surprising to find it being
used as a gateway address.

There is no reason why an IPNetwork object should record any bits from
the portion of the IP address that are covered by the mask bits.  To do
otherwise is to conflate Networks with individual IPs, which is what
got IPAddr rejected the first time around.  (And for those of you who
don't know/remember, I was initially an IPAddr booster in its original
incarnation, because I come from a network engineering background.)

This existence of this kind of confusion is why I voted +1 for the
IPNetwork constructor rejecting input with non-zero bits covered by the
mask, and a separate constructor method to return the network object
representing the network that a given IP address is in when a given
mask is applied to it.  (Which is easily computed by applying the mask
to the IP.)

--David

From daniel at stutzbachenterprises.com  Thu Sep 17 00:31:27 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 16 Sep 2009 17:31:27 -0500
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB15FD3.5040804@canterbury.ac.nz>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
Message-ID: <eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>

On Wed, Sep 16, 2009 at 4:59 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>wrote:

> Some people have claimed that the gateway address of a
> network isn't necessarily the zero address in that network.
>

I'll go further: I don't think it's even legal for the gateway address to be
the zero address of the network (and I used to program the embedded software
in routers for a living :) ).


> If that's true, then you *can't* calculate the network
> address from a host address and a netmask -- there isn't
> enough information.


In a router or operating system kernel, an IPv4 address or netmask is stored
in a 32-bit unsigned integer.  Host addresses, network addresses, and
netmasks satisfy the following properties, where & is the bitwise-AND
operation:

network address & netmask == network address
host address & netmask == network address of that host

A gateway is just a host that will forward packets.  A gateway address has
the same properties as a host address and isn't the same as the zero
address.

Every host has a routing table that contains a list of (network address,
gateway, netmask) tuples.  To see the list, just run "/sbin/route -n" on any
Linux system (and most other Unixes; root is not typically required) or
"route print" on a Windows system.

To route a packet, the operating system essentially performs the following
algorithm:

def find_gateway_for_destination(destination_address):
    for (network_address, gateway, netmask) in
list_of_routing_table_entires:
        if network_address == destination_address & netmask:
            return gateway
    raise DestinationUnreachable

Furthermore, an IPNetwork object
> needs to be able to represent a network address whose
> address part contains bits that aren't in the mask.
>

Network addresses never contain bits that aren't in the mask (due to the
rule: "network address & netmask == network address").

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090916/c15f752a/attachment.htm>

From greg.ewing at canterbury.ac.nz  Thu Sep 17 01:56:25 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Sep 2009 11:56:25 +1200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
Message-ID: <4AB17B29.8050708@canterbury.ac.nz>

R. David Murray wrote:

> A network is conventionally represented by an IP address in which the
> bits corresponding to the one bits in the netmask are set to zero, plus
> the netmask.

Okay, that's clarified things for me, thanks.

In that case, we shouldn't be talking about a "network address"
at all, but just a "network", and it makes sense to have

1) A class called IPNetwork that contains an IP number
and a mask, with the IP number constrained not to have
any ones where the mask has zeroes, and

2) A class called IPAddress which only contains an
IP number.

It seems that some people would also like to have

3) A class called IPAddressWithMask that contains an
IP number and a mask, with no constraints,

but I'm not enough of an expert to know whether such
a thing would be used often enough to be worth having
a special type for it, as opposed to using an
(IPNetwork, IPAddress) pair, or attaching a 'network'
attribute to an IPAddress, or some other solution
when the need arises.

-- 
Greg

From andrewm at object-craft.com.au  Thu Sep 17 02:30:27 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Thu, 17 Sep 2009 10:30:27 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB17B29.8050708@canterbury.ac.nz>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
Message-ID: <20090917003027.3AC245AC0D6@longblack.object-craft.com.au>

>R. David Murray wrote:
>
>> A network is conventionally represented by an IP address in which the
>> bits corresponding to the one bits in the netmask are set to zero, plus
>> the netmask.
>
>Okay, that's clarified things for me, thanks.

Put another way, an "Address" describes a single end-point and a "Network"
describes a set of (contiguous) Addresses.

Where things have become confused is that, for practical reasons, it is
convenient to have a representation for an Address and it's containing
Network (the later can be derived from the Address and a mask). We tried
to make the current Network entity do double-duty, but it is just leading
to confusion.

This is why I proprose there be three entities:

 * an Address entity (same as the current one)
 * a Network entity (like now, but requires masked bits to be zero)
 * an AddressWithMask entity (existing Network, but no container behaviour)

There is a school of thought that says we only need a single class
that behaves like the current Network entity - end-points are simply
represented by an all-ones mask. This is, I think, where we started. But
this scheme was rejected.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From andrewm at object-craft.com.au  Thu Sep 17 02:41:37 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Thu, 17 Sep 2009 10:41:37 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>
Message-ID: <20090917004138.2177C5AC0D6@longblack.object-craft.com.au>

>> Some people have claimed that the gateway address of a
>> network isn't necessarily the zero address in that network.

It almost never is - conventions vary, but it is often the network address
plus one, or the broadcast address minus one.

>I'll go further: I don't think it's even legal for the gateway address to be
>the zero address of the network (and I used to program the embedded software
>in routers for a living :) ).

I don't think the RFCs forbid the zero address being used, and
"enlightened" network stacks allow it (typically routers) to achieve
better utilisation of the limited IPv4 address space (for a /24 or larger,
wasting one address out of 255 isn't too bad, but it is now typical to
use much smaller nets - right down to /30).

>> If that's true, then you *can't* calculate the network
>> address from a host address and a netmask -- there isn't
>> enough information.

You can always calculate the network address from the IP address plus
mask - the network address is simply the bits that are not masked. 

In the olden days, the mask was spelled out in octets (eg
255.255.255.0). But we've moved to a more compact and logical notation
where the number of leading significant bits is specified (eg /24).

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From peter at hda3.com  Thu Sep 17 02:58:25 2009
From: peter at hda3.com (Peter Moody)
Date: Wed, 16 Sep 2009 17:58:25 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
Message-ID: <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>

On Wed, Sep 16, 2009 at 5:30 PM, Andrew McNamara
<andrewm at object-craft.com.au> wrote:
>>R. David Murray wrote:
>>
>>> A network is conventionally represented by an IP address in which the
>>> bits corresponding to the one bits in the netmask are set to zero, plus
>>> the netmask.
>>
>>Okay, that's clarified things for me, thanks.
>
> Put another way, an "Address" describes a single end-point and a "Network"
> describes a set of (contiguous) Addresses.
>
> Where things have become confused is that, for practical reasons, it is
> convenient to have a representation for an Address and it's containing
> Network (the later can be derived from the Address and a mask). We tried
> to make the current Network entity do double-duty, but it is just leading
> to confusion.
>
> This is why I proprose there be three entities:
>
> ?* an Address entity (same as the current one)
> ?* a Network entity (like now, but requires masked bits to be zero)
> ?* an AddressWithMask entity (existing Network, but no container behaviour)

This proposal actually leads to 6 entities (3 for IPv4 and 3 for IPv6).

It's still unclear to me what is gained by pulling AddressWithMask
functionality out of the current network classes. It's easy enough for
the concerned developer who to check if the entered network address
does actually have all of its host bits set to zero. It is not my
experience that this behavior is desired so often that having the
network classes behave as they do now leads to a great deal of
confusion.

> There is a school of thought that says we only need a single class
> that behaves like the current Network entity - end-points are simply
> represented by an all-ones mask. This is, I think, where we started. But
> this scheme was rejected.

I don't think anything has actually been rejected.

> --
> Andrew McNamara, Senior Developer, Object Craft
> http://www.object-craft.com.au/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From andrewm at object-craft.com.au  Thu Sep 17 03:32:19 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Thu, 17 Sep 2009 11:32:19 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
Message-ID: <20090917013219.6A3895AC0D6@longblack.object-craft.com.au>

>This proposal actually leads to 6 entities (3 for IPv4 and 3 for IPv6).

Yes, I know - I was just trying to keep to the point.

>It's still unclear to me what is gained by pulling AddressWithMask
>functionality out of the current network classes. It's easy enough for
>the concerned developer who to check if the entered network address
>does actually have all of its host bits set to zero. It is not my
>experience that this behavior is desired so often that having the
>network classes behave as they do now leads to a great deal of
>confusion.

I think we're in a painful middle ground now - we should either go back
to the idea of a single class (per protocol), or make the distinctions
clear (networks are containers and addresses are singletons).

Personally, I think I would be happy with a single class (but I suspect
that's just my laziness speaking). However, I think the structure and
discipline of three classes (per protocol) may actually make the concepts
easier to understand for non-experts.

A particular case in point - if you want to represent a single IP address
with netmask (say an interface), you use a Network class, not an Address
class. And the .network attribute returns a Address class!

The reason I suggest having the Network class assert that masked bits be
zero is two-fold:

 * it ensures the correct class is being used for the job
 * it ensures application-user errors are detected as early as possible

I also suggest the AddressWithMask classes not have any network/container
behaviours for a similar reason. If the developer needs these, the
.network attribute is only a lookup away.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From peter at hda3.com  Thu Sep 17 04:19:04 2009
From: peter at hda3.com (Peter Moody)
Date: Wed, 16 Sep 2009 19:19:04 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
Message-ID: <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>

On Wed, Sep 16, 2009 at 6:32 PM, Andrew McNamara
<andrewm at object-craft.com.au> wrote:
>>This proposal actually leads to 6 entities (3 for IPv4 and 3 for IPv6).
>
> Yes, I know - I was just trying to keep to the point.
>
>>It's still unclear to me what is gained by pulling AddressWithMask
>>functionality out of the current network classes. It's easy enough for
>>the concerned developer who to check if the entered network address
>>does actually have all of its host bits set to zero. It is not my
>>experience that this behavior is desired so often that having the
>>network classes behave as they do now leads to a great deal of
>>confusion.
>
> I think we're in a painful middle ground now - we should either go back
> to the idea of a single class (per protocol), or make the distinctions
> clear (networks are containers and addresses are singletons).
>
> Personally, I think I would be happy with a single class (but I suspect
> that's just my laziness speaking). However, I think the structure and
> discipline of three classes (per protocol) may actually make the concepts
> easier to understand for non-experts.

I think this is where we disagree. I don't think the added complexity
does make it any easier to understand.

> A particular case in point - if you want to represent a single IP address
> with netmask (say an interface), you use a Network class, not an Address
> class. And the .network attribute returns a Address class!

Right, and I don't see where the confusion lies.  You have an address
+ netmask. ergo, you have a Network object.  The single address that
defines the base address (most commonly referred to as the network
address) is an Address object. there is no netmask associated with
that single address, ergo, it's an Address object.

> The reason I suggest having the Network class assert that masked bits be
> zero is two-fold:
>
> ?* it ensures the correct class is being used for the job
> ?* it ensures application-user errors are detected as early as possible
>
> I also suggest the AddressWithMask classes not have any network/container
> behaviours for a similar reason. If the developer needs these, the
> .network attribute is only a lookup away.

the problem I have with this approach is that it seems like a long way
to go for a shortcut (of checking if addr.ip != addr.network: raise
Error).

Cheers,
/peter

> --
> Andrew McNamara, Senior Developer, Object Craft
> http://www.object-craft.com.au/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From greg.ewing at canterbury.ac.nz  Thu Sep 17 04:30:28 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Sep 2009 14:30:28 +1200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
Message-ID: <4AB19F44.8010102@canterbury.ac.nz>

Andrew McNamara wrote:

> I also suggest the AddressWithMask classes not have any network/container
> behaviours for a similar reason. If the developer needs these, the
> .network attribute is only a lookup away.

Another way to approach this would be for the Address
object to potentially have a 'network' attribute
referencing a Network object.

Then there are only two classes, but three use cases
are covered:

1) a Network

2) a plain, network-agnostic Address with network == None

3) an Address with an attached Network

An Address could be constructed in three ways:

   Address(ip_number)

   Address(ip_number, network = <Network instance>)

   Address(ip_number, mask = <mask>)
     # constructs and attaches a suitably-masked Network instance

We could also have some_network[n] return an Address
referring back to the network object it was obtained
from.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Thu Sep 17 04:48:59 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Sep 2009 14:48:59 +1200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
Message-ID: <4AB1A39B.7090003@canterbury.ac.nz>

Peter Moody wrote:
> I don't see where the confusion lies.  You have an address
> + netmask. ergo, you have a Network object.  The single address that
> defines the base address (most commonly referred to as the network
> address) is an Address object. there is no netmask associated with
> that single address, ergo, it's an Address object.

But if I understand correctly, you *don't* have a network,
you have something representing a particular host address
within a network, plus information that lets you deduce the
network to which it belongs. So calling the object a
'Network' is a misnomer.

What's more, I don't see the point of having a 'network'
attribute without a mask, because the zero address of a
network on its own doesn't define the network. You need
the zero address plus a mask to do that.

I'm not sure what usefulness the zero address on its own
has, but if it's considered useful enough to have an
attribute for it, calling it something like 'base_address'
would be less confusing.

-- 
Greg

From andrewm at object-craft.com.au  Thu Sep 17 05:21:26 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Thu, 17 Sep 2009 13:21:26 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
Message-ID: <20090917032126.66F896006DE@longblack.object-craft.com.au>

>> I think we're in a painful middle ground now - we should either go back
>> to the idea of a single class (per protocol), or make the distinctions
>> clear (networks are containers and addresses are singletons).
>>
>> Personally, I think I would be happy with a single class (but I suspect
>> that's just my laziness speaking). However, I think the structure and
>> discipline of three classes (per protocol) may actually make the concepts
>> easier to understand for non-experts.
>
>I think this is where we disagree. I don't think the added complexity
>does make it any easier to understand.

I argue that we're not actually adding any complexity: yes, we add
a class (per protocol), but we then merely relocate functionality to
clarify the intended use of the classes.

>> A particular case in point - if you want to represent a single IP address
>> with netmask (say an interface), you use a Network class, not an Address
>> class. And the .network attribute returns a Address class!
>
>Right, and I don't see where the confusion lies.  

I suggest you are too close to the implementation to be surprised by it. 8-)

>You have an address + netmask. ergo, you have a Network object.  

In a common use case, however, this instance will not represent a
network at all, but an address. It will have container-like behaviour,
but it should not (this is a property of networks, not addresses). So
the instance will be misnamed and have behaviours that are, at best,
misleading.

>The single address that defines the base address (most commonly referred
>to as the network address) is an Address object. there is no netmask
>associated with that single address, ergo, it's an Address object.

I would argue that a Network never has a single address - by definition,
it has two or more nodes. A .network attribute should return a Network
instance. If you want the base address, this probably should be called
.base_address or just .address (to parallel the .netmask attribute).

>> The reason I suggest having the Network class assert that masked bits be
>> zero is two-fold:
>>
>> * it ensures the correct class is being used for the job
>> * it ensures application-user errors are detected as early as possible
>>
>> I also suggest the AddressWithMask classes not have any network/container
>> behaviours for a similar reason. If the developer needs these, the
>> .network attribute is only a lookup away.
>
>the problem I have with this approach is that it seems like a long way
>to go for a shortcut (of checking if addr.ip != addr.network: raise
>Error).

This isn't about shortcuts, but about correctness... having the Network
object represent a network, and having Address objects represent
end-points, and having errors discovered as early as possible.

What I'm arguing here is that singletons should not simultaneously be
containers - it's not pythonic, and it leads to ambiguity. The underlying
IP concepts don't require it either: an IP address is a singleton, a
network is a container, and there is no overlap. Yes, an address may be a
member of a network, and having a reference to that network on the address
object is valuable, but the address should not behave like a network.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From peter at hda3.com  Thu Sep 17 05:26:27 2009
From: peter at hda3.com (Peter Moody)
Date: Wed, 16 Sep 2009 20:26:27 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB1A39B.7090003@canterbury.ac.nz>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
Message-ID: <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>

On Wed, Sep 16, 2009 at 7:48 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Peter Moody wrote:
>>
>> I don't see where the confusion lies. ?You have an address
>> + netmask. ergo, you have a Network object. ?The single address that
>> defines the base address (most commonly referred to as the network
>> address) is an Address object. there is no netmask associated with
>> that single address, ergo, it's an Address object.
>
> But if I understand correctly, you *don't* have a network,
> you have something representing a particular host address
> within a network, plus information that lets you deduce the
> network to which it belongs. So calling the object a
> 'Network' is a misnomer.
>
> What's more, I don't see the point of having a 'network'
> attribute without a mask, because the zero address of a
> network on its own doesn't define the network. You need
> the zero address plus a mask to do that.
>
> I'm not sure what usefulness the zero address on its own
> has, but if it's considered useful enough to have an
> attribute for it, calling it something like 'base_address'
> would be less confusing.

the address with all of the hosts bits masked to zero is most commonly
referred to as the network address. same as the address with all of
the host bits set to one is called the broadcast address. calling it
something like base_address or min_address will cause quite a bit more
confusion.

>
> --
> Greg
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From andrewm at object-craft.com.au  Thu Sep 17 05:27:43 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Thu, 17 Sep 2009 13:27:43 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB19F44.8010102@canterbury.ac.nz>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz>
Message-ID: <20090917032743.A231B6006DE@longblack.object-craft.com.au>

>Another way to approach this would be for the Address object to
>potentially have a 'network' attribute referencing a Network object.

Yes - that's reasonable.

>Then there are only two classes, but three use cases are covered:
>
>1) a Network
>
>2) a plain, network-agnostic Address with network == None
>
>3) an Address with an attached Network
>
>An Address could be constructed in three ways:
>
>   Address(ip_number)
>
>   Address(ip_number, network = <Network instance>)
>
>   Address(ip_number, mask = <mask>)
>     # constructs and attaches a suitably-masked Network instance

I think you still need to support the common notations:

    Address('10.0.0.1')                 # .network == None

    Address('10.0.0.1/255.255.255.0')
    Address('10.0.0.1/24')

>We could also have some_network[n] return an Address
>referring back to the network object it was obtained
>from.

Yes.

(Of course, we're simplifying - there would really be classes for each
protocol).

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From peter at hda3.com  Thu Sep 17 05:36:28 2009
From: peter at hda3.com (Peter Moody)
Date: Wed, 16 Sep 2009 20:36:28 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090917032126.66F896006DE@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<20090917032126.66F896006DE@longblack.object-craft.com.au>
Message-ID: <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>

On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara
<andrewm at object-craft.com.au> wrote:
>>> I think we're in a painful middle ground now - we should either go back
>>> to the idea of a single class (per protocol), or make the distinctions
>>> clear (networks are containers and addresses are singletons).
>>>
>>> Personally, I think I would be happy with a single class (but I suspect
>>> that's just my laziness speaking). However, I think the structure and
>>> discipline of three classes (per protocol) may actually make the concepts
>>> easier to understand for non-experts.
>>
>>I think this is where we disagree. I don't think the added complexity
>>does make it any easier to understand.
>
> I argue that we're not actually adding any complexity: yes, we add
> a class (per protocol), but we then merely relocate functionality to
> clarify the intended use of the classes.

And I argue the moving this functionality to new classes (and adding
new restrictions to existing classes) doesn't buy anything in the way
of overall functionality of the module or a developer's ability to
comprehend intended uses.

>>> A particular case in point - if you want to represent a single IP address
>>> with netmask (say an interface), you use a Network class, not an Address
>>> class. And the .network attribute returns a Address class!
>>
>>Right, and I don't see where the confusion lies.
>
> I suggest you are too close to the implementation to be surprised by it. 8-)

touche :)

>>You have an address + netmask. ergo, you have a Network object.
>
> In a common use case, however, this instance will not represent a
> network at all, but an address. It will have container-like behaviour,
> but it should not (this is a property of networks, not addresses). So
> the instance will be misnamed and have behaviours that are, at best,
> misleading.
>
>>The single address that defines the base address (most commonly referred
>>to as the network address) is an Address object. there is no netmask
>>associated with that single address, ergo, it's an Address object.
>
> I would argue that a Network never has a single address - by definition,
> it has two or more nodes. A .network attribute should return a Network
> instance. If you want the base address, this probably should be called
> .base_address or just .address (to parallel the .netmask attribute).

.network is shorthand for network address. are .network_address and
.broadcast_address less confusing?  I have to say, though,
.network/.broadcast are fairly common (IPy uses .net, netaddr and ipv4
use, IIRC .network...)

>>> The reason I suggest having the Network class assert that masked bits be
>>> zero is two-fold:
>>>
>>> * it ensures the correct class is being used for the job
>>> * it ensures application-user errors are detected as early as possible
>>>
>>> I also suggest the AddressWithMask classes not have any network/container
>>> behaviours for a similar reason. If the developer needs these, the
>>> .network attribute is only a lookup away.
>>
>>the problem I have with this approach is that it seems like a long way
>>to go for a shortcut (of checking if addr.ip != addr.network: raise
>>Error).
>
> This isn't about shortcuts, but about correctness... having the Network
> object represent a network, and having Address objects represent
> end-points, and having errors discovered as early as possible.

Then what I don't see is the purpose of your
network-only-network-object. essentially identical functionality can
be obtained with the module as is w/o the added complexity of new
classes.

Cheers,
/peter

> What I'm arguing here is that singletons should not simultaneously be
> containers - it's not pythonic, and it leads to ambiguity. The underlying
> IP concepts don't require it either: an IP address is a singleton, a
> network is a container, and there is no overlap. Yes, an address may be a
> member of a network, and having a reference to that network on the address
> object is valuable, but the address should not behave like a network.
>
> --
> Andrew McNamara, Senior Developer, Object Craft
> http://www.object-craft.com.au/
>

From peter at hda3.com  Thu Sep 17 05:41:47 2009
From: peter at hda3.com (Peter Moody)
Date: Wed, 16 Sep 2009 20:41:47 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<20090917032126.66F896006DE@longblack.object-craft.com.au>
	<8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
Message-ID: <8517e9350909162041n1685f80cmab96412fa3e47cb4@mail.gmail.com>

On Wed, Sep 16, 2009 at 8:36 PM, Peter Moody <peter at hda3.com> wrote:
> On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara
> <andrewm at object-craft.com.au> wrote:
>>>> I think we're in a painful middle ground now - we should either go back
>>>> to the idea of a single class (per protocol), or make the distinctions
>>>> clear (networks are containers and addresses are singletons).
>>>>
>>>> Personally, I think I would be happy with a single class (but I suspect
>>>> that's just my laziness speaking). However, I think the structure and
>>>> discipline of three classes (per protocol) may actually make the concepts
>>>> easier to understand for non-experts.
>>>
>>>I think this is where we disagree. I don't think the added complexity
>>>does make it any easier to understand.
>>
>> I argue that we're not actually adding any complexity: yes, we add
>> a class (per protocol), but we then merely relocate functionality to
>> clarify the intended use of the classes.
>
> And I argue the moving this functionality to new classes (and adding
> new restrictions to existing classes) doesn't buy anything in the way
> of overall functionality of the module or a developer's ability to
> comprehend intended uses.
>
>>>> A particular case in point - if you want to represent a single IP address
>>>> with netmask (say an interface), you use a Network class, not an Address
>>>> class. And the .network attribute returns a Address class!
>>>
>>>Right, and I don't see where the confusion lies.
>>
>> I suggest you are too close to the implementation to be surprised by it. 8-)
>
> touche :)
>
>>>You have an address + netmask. ergo, you have a Network object.
>>
>> In a common use case, however, this instance will not represent a
>> network at all, but an address. It will have container-like behaviour,
>> but it should not (this is a property of networks, not addresses). So
>> the instance will be misnamed and have behaviours that are, at best,
>> misleading.
>>
>>>The single address that defines the base address (most commonly referred
>>>to as the network address) is an Address object. there is no netmask
>>>associated with that single address, ergo, it's an Address object.
>>
>> I would argue that a Network never has a single address - by definition,
>> it has two or more nodes. A .network attribute should return a Network
>> instance. If you want the base address, this probably should be called
>> .base_address or just .address (to parallel the .netmask attribute).
>
> .network is shorthand for network address. are .network_address and
> .broadcast_address less confusing? ?I have to say, though,
> .network/.broadcast are fairly common (IPy uses .net, netaddr and ipv4
> use, IIRC .network...)
>
>>>> The reason I suggest having the Network class assert that masked bits be
>>>> zero is two-fold:
>>>>
>>>> * it ensures the correct class is being used for the job
>>>> * it ensures application-user errors are detected as early as possible
>>>>
>>>> I also suggest the AddressWithMask classes not have any network/container
>>>> behaviours for a similar reason. If the developer needs these, the
>>>> .network attribute is only a lookup away.
>>>
>>>the problem I have with this approach is that it seems like a long way
>>>to go for a shortcut (of checking if addr.ip != addr.network: raise
>>>Error).
>>
>> This isn't about shortcuts, but about correctness... having the Network
>> object represent a network, and having Address objects represent
>> end-points, and having errors discovered as early as possible.
>
> Then what I don't see is the purpose of your
> network-only-network-object. essentially identical functionality can
> be obtained with the module as is w/o the added complexity of new
> classes.

Since you mentioned correctness; it seems like we're coming back to an
option to the IPv?Network().__init__ methods which strictly checks for
host bits being masked to zero. this provides, I think, the
functionality/error checking you're looking for w/o the need for new
classes.

Cheers,
/peter


> Cheers,
> /peter
>
>> What I'm arguing here is that singletons should not simultaneously be
>> containers - it's not pythonic, and it leads to ambiguity. The underlying
>> IP concepts don't require it either: an IP address is a singleton, a
>> network is a container, and there is no overlap. Yes, an address may be a
>> member of a network, and having a reference to that network on the address
>> object is valuable, but the address should not behave like a network.
>>
>> --
>> Andrew McNamara, Senior Developer, Object Craft
>> http://www.object-craft.com.au/
>>
>

From scott+python-dev at scottdial.com  Thu Sep 17 05:53:12 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 16 Sep 2009 23:53:12 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>	<4AB17B29.8050708@canterbury.ac.nz>	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>	<20090917032126.66F896006DE@longblack.object-craft.com.au>
	<8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
Message-ID: <4AB1B2A8.9050204@scottdial.com>

Peter Moody wrote:
> On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara
> <andrewm at object-craft.com.au> wrote:
>>>> I think we're in a painful middle ground now - we should either go back
>>>> to the idea of a single class (per protocol), or make the distinctions
>>>> clear (networks are containers and addresses are singletons).
>>>>
>>>> Personally, I think I would be happy with a single class (but I suspect
>>>> that's just my laziness speaking). However, I think the structure and
>>>> discipline of three classes (per protocol) may actually make the concepts
>>>> easier to understand for non-experts.
>>> I think this is where we disagree. I don't think the added complexity
>>> does make it any easier to understand.
>> I argue that we're not actually adding any complexity: yes, we add
>> a class (per protocol), but we then merely relocate functionality to
>> clarify the intended use of the classes.
> 
> And I argue the moving this functionality to new classes (and adding
> new restrictions to existing classes) doesn't buy anything in the way
> of overall functionality of the module or a developer's ability to
> comprehend intended uses.

Speaking as the originator of this thread of discourse, the lack of a
3rd class was exactly the source of my confusion and my inability to
communicate my confusion to everyone. I clearly did not understand the
intended uses of the IPNetwork type because it was capable of playing
two roles that are decidedly different conceptually. So, I must
respectfully disagree with you.

>>> You have an address + netmask. ergo, you have a Network object.
>> In a common use case, however, this instance will not represent a
>> network at all, but an address. It will have container-like behaviour,
>> but it should not (this is a property of networks, not addresses). So
>> the instance will be misnamed and have behaviours that are, at best,
>> misleading.

This is exactly the confusion and duality that I struggled with.

>> This isn't about shortcuts, but about correctness... having the Network
>> object represent a network, and having Address objects represent
>> end-points, and having errors discovered as early as possible.
> 
> Then what I don't see is the purpose of your
> network-only-network-object. essentially identical functionality can
> be obtained with the module as is w/o the added complexity of new
> classes.

The purpose is to avoid conflating IPNetwork with an IPAddress that has
a mask. If the IPNetwork didn't accept a non-zero host and instead
required a developer to use a helper to construct a IPNetwork with a
proper address, then there would be less confusion about what exactly a
IPNetwork is meant to represent. As it stands, it's purposes is muddled
by accepting host addresses too.

I am rather indifferent whether there needs to be a IPAddressWithMask
type. If that is needed, then it is rather easy to create a type that
does that. And, if it is a common pattern, then it could be added to the
module later in life.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From tseaver at palladion.com  Thu Sep 17 06:14:12 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 17 Sep 2009 00:14:12 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB1B2A8.9050204@scottdial.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB0C7A9.3030001@gmail.com>	<4AB15FD3.5040804@canterbury.ac.nz>	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>	<4AB17B29.8050708@canterbury.ac.nz>	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>	<20090917032126.66F896006DE@longblack.object-craft.com.au>	<8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
	<4AB1B2A8.9050204@scottdial.com>
Message-ID: <h8sd2n$n1g$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Scott Dial wrote:

> The purpose is to avoid conflating IPNetwork with an IPAddress that has
> a mask.

I'm not confused by anything in this discussion except the repeated
harping on the (to me imaginary) concept of "address with a mask".
Conceptually:

 - Addresses don't have masks:  they are just 32- (128-) bit integers.

 - Any given address can belong to a number of networks, based on a
   given topology.

 - Masks are also 32- (128-) bit integers, which happen to have the
   property that their leftmost N bits are all zero and the rest are all
   one.

 - One can construct a network from the combination of an address and
   a mask.  Such a network *never* needs to know what the address was
   before masking.

 - Networks *do* know their "zero" address, as well as their "last
   address", and can do "containment" operations on addresses (I'm not
   clear on the need to do such operations on other networks, rather
   than just using their zero addresses).

At the API level, it is convenient to represent the combination
(address, mask) as a single string (e.g., '192.168.1.0/24'):  parsing
that string should return a network, not an address.  Let's keep the
"parsing human readable / writable strings" part separate from the clear
conceptual map.

> If the IPNetwork didn't accept a non-zero host and instead
> required a developer to use a helper to construct a IPNetwork with a
> proper address, then there would be less confusion about what exactly a
> IPNetwork is meant to represent. As it stands, it's purposes is muddled
> by accepting host addresses too.

I can't imagine why it would be useful to try to infer a network from an
address:  "In the face of ambiguity, refuse the temptation to guess."

> I am rather indifferent whether there needs to be a IPAddressWithMask
> type. If that is needed, then it is rather easy to create a type that
> does that. And, if it is a common pattern, then it could be added to the
> module later in life.

Agreed.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKsbeU+gerLs4ltQ4RAlG0AJ9ZvhuXHsTL2hheW/vlzeMmArs5rgCeLDLS
eKWSAdkdv++umepu+nK0/7I=
=N7ej
-----END PGP SIGNATURE-----


From andrewm at object-craft.com.au  Thu Sep 17 06:17:45 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Thu, 17 Sep 2009 14:17:45 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<20090917032126.66F896006DE@longblack.object-craft.com.au>
	<8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
Message-ID: <20090917041745.C331B6006DE@longblack.object-craft.com.au>

>> I argue that we're not actually adding any complexity: yes, we add
>> a class (per protocol), but we then merely relocate functionality to
>> clarify the intended use of the classes.
>
>And I argue the moving this functionality to new classes (and adding
>new restrictions to existing classes) doesn't buy anything in the way
>of overall functionality of the module or a developer's ability to
>comprehend intended uses.

It's mostly just minor refactoring and renaming, which I think makes
things clearer, although I agree this is merely an opinion. I would be
interest to hear what others think. To summarise:

 * an address is a singleton (a network endpoint), with no container
   behaviour. It may optionally reference it's network (via the .network
   attribute), .address returns mask-less address. 

 * a network is a container-like object. For consistency, .network should
   return self and raise an exception if the mask conflicts with the
   address, .address returns the base address, .mask returns an address
   object.

>> I would argue that a Network never has a single address - by definition,
>> it has two or more nodes. A .network attribute should return a Network
>> instance. If you want the base address, this probably should be called
>> .base_address or just .address (to parallel the .netmask attribute).
>
>.network is shorthand for network address. are .network_address and
>.broadcast_address less confusing?  I have to say, though,
>.network/.broadcast are fairly common (IPy uses .net, netaddr and ipv4
>use, IIRC .network...)

Yes, I understand your motivation, but I still think it's going to be more
confusing the way you have it.

>> This isn't about shortcuts, but about correctness... having the Network
>> object represent a network, and having Address objects represent
>> end-points, and having errors discovered as early as possible.
>
>Then what I don't see is the purpose of your
>network-only-network-object. essentially identical functionality can
>be obtained with the module as is w/o the added complexity of new
>classes.

Certainly, I'm not talking about adding functionality. What I am
suggesting is that if we wish to have a distinction between networks and
addresses, then that distinction should be clear and strong, such that
the choice of which to use is obvious, and if the wrong one is used,
the error is discovered as early as possible.

As the module stands, we have a pair of address-without-mask classes
called *Address, and a pair of address-with-mask classes called
*Network. So, sometimes when you want to record an *address* you use
a class called Network, and that class comes with a behaviours that
make no sense in the context of a singleton network end-point (it can't
"contain" other addresses, although it's .network can).

Sorry if I sound like a cracked record - these are subtle concepts,
and my ability to explain what I mean is less than is needed, but we'll
get there in the end.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From stephen at xemacs.org  Thu Sep 17 08:51:25 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 17 Sep 2009 15:51:25 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090917041745.C331B6006DE@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<20090917032126.66F896006DE@longblack.object-craft.com.au>
	<8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
	<20090917041745.C331B6006DE@longblack.object-craft.com.au>
Message-ID: <87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp>

Andrew McNamara writes:

 > As the module stands, we have a pair of address-without-mask classes
 > called *Address, and a pair of address-with-mask classes called
 > *Network. So, sometimes when you want to record an *address* you use
 > a class called Network, and that class comes with a behaviours that
 > make no sense in the context of a singleton network end-point (it can't
 > "contain" other addresses, although it's .network can).

I'm going to consistently use "address" to mean a singleton and
"network" to mean a container in the following.

I still don't see why an address-with-mask is useful, except that the
network is deducible as {'network': address & mask, 'mask': mask}.  Is
there *any* other way you would *ever* use that?

It seems to me that for some purposes (implementing dig(1), for
example), an IPv4Address can contain only the address (ie, a 32-bit
integer) as a data attribute, and (with methods for using that
attribute) that is the minimal implementation of IPv4Address.

However, there are other cases (eg, routing) where it's useful to
associate an address with its network, and I don't see much harm in
doing so by adding a 'network' attribute to the base class
IPv4Address, since addresses are hardly useful except in the context
of networks.  Of course that attribute is often going to be None (eg,
in implementing dig(1) the remote nameserver is unlikely to tell you
the netmask).  However, when iterating over an IPv4Network, the
iterator can automatically fill in the 'network' attribute, and that's
fairly cheap.

While to me neither the 'network' attribute nor the iterator behavior
just described seems amazing useful in the base classes, it seems to
me that precisely those behaviors will be reinvented over and over
again for derived classes.  Furthermore they are natural enough that
they won't bother people who don't need them.  (That's despite at
least one person (IIRC it was Antoine) firmly saying "an IPv4Address
should contain exactly one 32-bit int, no more, no less", so I could
be wrong.)  It seems to me that the only good reason for not having a
'network' attribute that contains an IPv4Network instance or None is
efficiency: the space for the attribute and the overhead of filling it
in the iterator.  I personally can't think of an application that
would care (from what I hear, Cisco has no interest in writing its
routers' IP stacks in Python, amazingly enough), but in theory ...

Finally, I agree that using IPv4Network as address-with-mask is a
confusing, undiscoverable abuse.  In particular, I think that every
time I went a week without using that idiom, I'd get nervous when I
saw it again: "Are you *sure* that won't raise an error or silently
get the lower bits masked off?!  If not now, in the next version?"

Obviously all the above applies to IPv6 classes, too.

From andrewm at object-craft.com.au  Thu Sep 17 09:15:16 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Thu, 17 Sep 2009 17:15:16 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<20090917032126.66F896006DE@longblack.object-craft.com.au>
	<8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
	<20090917041745.C331B6006DE@longblack.object-craft.com.au>
	<87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20090917071516.217655AC0D6@longblack.object-craft.com.au>

> > As the module stands, we have a pair of address-without-mask classes
> > called *Address, and a pair of address-with-mask classes called
> > *Network. So, sometimes when you want to record an *address* you use
> > a class called Network, and that class comes with a behaviours that
> > make no sense in the context of a singleton network end-point (it can't
> > "contain" other addresses, although it's .network can).
>
>I'm going to consistently use "address" to mean a singleton and
>"network" to mean a container in the following.

Ta. I think it's useful to have a common terminology.

>I still don't see why an address-with-mask is useful, except that the
>network is deducible as {'network': address & mask, 'mask': mask}.  Is
>there *any* other way you would *ever* use that?
>
>It seems to me that for some purposes (implementing dig(1), for
>example), an IPv4Address can contain only the address (ie, a 32-bit
>integer) as a data attribute, and (with methods for using that
>attribute) that is the minimal implementation of IPv4Address.
>
>However, there are other cases (eg, routing) where it's useful to
>associate an address with its network, and I don't see much harm in
>doing so by adding a 'network' attribute to the base class
>IPv4Address, since addresses are hardly useful except in the context
>of networks.  Of course that attribute is often going to be None (eg,
>in implementing dig(1) the remote nameserver is unlikely to tell you
>the netmask).  However, when iterating over an IPv4Network, the
>iterator can automatically fill in the 'network' attribute, and that's
>fairly cheap.

Conceptually, you sometimes need a bare address, and other times,
you need an address with an associated network (host interface
configs, router configs, etc). By AddressWithMask, I really mean
AddressWithEnoughInformationToDeriveNetworkWhenNeeded. Conveniently,
IPv4 and IPv6 addressing allows us to derive the network from the host
address combined with the netmask - in other words, we don't have to attach
a real Network object to Address objects until the user tries to access
it, and then we derive it from the address and mask.

>While to me neither the 'network' attribute nor the iterator behavior
>just described seems amazing useful in the base classes, it seems to
>me that precisely those behaviors will be reinvented over and over
>again for derived classes.  Furthermore they are natural enough that
>they won't bother people who don't need them.  (That's despite at
>least one person (IIRC it was Antoine) firmly saying "an IPv4Address
>should contain exactly one 32-bit int, no more, no less", so I could
>be wrong.)  

If you have a .network attribute on an address object, checking if an
address is in the same network as another address becomes:

    addr_a in addr_b.network

As the module stands, you write that as:

    addr_a in addr_b

I don't think the intent is as clear with the later.

>It seems to me that the only good reason for not having a
>'network' attribute that contains an IPv4Network instance or None is
>efficiency: the space for the attribute and the overhead of filling it
>in the iterator.  I personally can't think of an application that
>would care (from what I hear, Cisco has no interest in writing its
>routers' IP stacks in Python, amazingly enough), but in theory ...

The implementation already lazily creates most things like this.

>Finally, I agree that using IPv4Network as address-with-mask is a
>confusing, undiscoverable abuse.  In particular, I think that every
>time I went a week without using that idiom, I'd get nervous when I
>saw it again: "Are you *sure* that won't raise an error or silently
>get the lower bits masked off?!  If not now, in the next version?"

Yes.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From p.f.moore at gmail.com  Thu Sep 17 09:33:35 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 17 Sep 2009 08:33:35 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<20090917032126.66F896006DE@longblack.object-craft.com.au>
	<8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
Message-ID: <79990c6b0909170033p2b1897c2s8ad00229a0ce2dcd@mail.gmail.com>

2009/9/17 Peter Moody <peter at hda3.com>:
> On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara
> <andrewm at object-craft.com.au> wrote:
>>>> I think we're in a painful middle ground now - we should either go back
>>>> to the idea of a single class (per protocol), or make the distinctions
>>>> clear (networks are containers and addresses are singletons).
>>>>
>>>> Personally, I think I would be happy with a single class (but I suspect
>>>> that's just my laziness speaking). However, I think the structure and
>>>> discipline of three classes (per protocol) may actually make the concepts
>>>> easier to understand for non-experts.
>>>
>>>I think this is where we disagree. I don't think the added complexity
>>>does make it any easier to understand.
>>
>> I argue that we're not actually adding any complexity: yes, we add
>> a class (per protocol), but we then merely relocate functionality to
>> clarify the intended use of the classes.
>
> And I argue the moving this functionality to new classes (and adding
> new restrictions to existing classes) doesn't buy anything in the way
> of overall functionality of the module or a developer's ability to
> comprehend intended uses.

For what it's worth, Andrew's making the distinction clatified things
for *me*. Whether I am the target audience for the module (I'm not a
network expert, but on the odd occasion I need to manipulate IP
addresses in a program, and if the standard library had a module for
doing so, I'd reach for it) is up to someone else to decide, but if I
am, then my vote is for 3 classes based on the explanations I've seen
so far.

I don't see what having fewer classes buys me as the end user.

Paul.

From techtonik at gmail.com  Thu Sep 17 12:09:20 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Thu, 17 Sep 2009 13:09:20 +0300
Subject: [Python-Dev] Roundup dependency tree changes
Message-ID: <d34314100909170309x2fa50fbdjaffd905e26f722e4@mail.gmail.com>

Hello,

How about allowing all authenticated users update dependency field in
Python tracker?

http://bugs.python.org/issue?@sort0=dependencies&@sortdir0=on&@sort1=&@group0=priority&@group1=&@columns=title,id,activity,dependencies,status&@pagesize=150&@startwith=0

Looks like developers with access don't have enough time to track
issue dependencies.

--anatoly t.

From solipsis at pitrou.net  Thu Sep 17 13:49:57 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Sep 2009 11:49:57 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
Message-ID: <loom.20090917T134809-168@post.gmane.org>

Peter Moody <peter <at> hda3.com> writes:
> 
> the address with all of the hosts bits masked to zero is most commonly
> referred to as the network address. same as the address with all of
> the host bits set to one is called the broadcast address. calling it
> something like base_address or min_address will cause quite a bit more
> confusion.

Quite a bit less IMO. I'm not a network specialist, but the "network address" of
a network sounds quite vague to me, while things like "lower_bound", "first",
"first_address", "firstip" are immediately understandable and non-ambiguous.



From solipsis at pitrou.net  Thu Sep 17 13:58:09 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Sep 2009 11:58:09 +0000 (UTC)
Subject: [Python-Dev] conceptual clarity
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz>
Message-ID: <h8t88h$9q$1@ger.gmane.org>

Le Thu, 17 Sep 2009 14:30:28 +1200, Greg Ewing a ?crit?:
> 
> 3) an Address with an attached Network
> 
> An Address could be constructed in three ways:
> 
>    Address(ip_number)
> 
>    Address(ip_number, network = <Network instance>)
> 
>    Address(ip_number, mask = <mask>)
>      # constructs and attaches a suitably-masked Network instance
> 
> We could also have some_network[n] return an Address referring back to
> the network object it was obtained from.

It seems you are uselessly conflating two perfectly distinct concepts: 
Address and Network. You also haven't addresses the issue of comparing 
together (and hashing) two addresses with the same IP but pointing to a 
different network. No answer to this issue seems satisfactory and 
obviously right.

As it is, -1 from me. Either we only keep two concepts (Address and 
Network), or if we introduce a third one (AddressWithMask, whatever) for 
added practicality; but we shouldn't blur the line between the two former 
canonical concepts under the pretext that a platypus-like Address might 
be helpful in some particular situations.

Regards

Antoine.



From eric at trueblade.com  Thu Sep 17 14:14:34 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 17 Sep 2009 08:14:34 -0400
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <h8t88h$9q$1@ger.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB03909.3090608@trueblade.com>	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>	<200909162053.00354.steve@pearwood.info>
	<4AB0C7A9.3030001@gmail.com>	<4AB15FD3.5040804@canterbury.ac.nz>	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>	<4AB17B29.8050708@canterbury.ac.nz>	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>	<4AB19F44.8010102@canterbury.ac.nz>
	<h8t88h$9q$1@ger.gmane.org>
Message-ID: <4AB2282A.3050902@trueblade.com>

Antoine Pitrou wrote:
> Le Thu, 17 Sep 2009 14:30:28 +1200, Greg Ewing a ?crit :
>> 3) an Address with an attached Network
>>
>> An Address could be constructed in three ways:
>>
>>    Address(ip_number)
>>
>>    Address(ip_number, network = <Network instance>)
>>
>>    Address(ip_number, mask = <mask>)
>>      # constructs and attaches a suitably-masked Network instance
>>
>> We could also have some_network[n] return an Address referring back to
>> the network object it was obtained from.
> 
> It seems you are uselessly conflating two perfectly distinct concepts: 
> Address and Network. You also haven't addresses the issue of comparing 
> together (and hashing) two addresses with the same IP but pointing to a 
> different network. No answer to this issue seems satisfactory and 
> obviously right.
> 
> As it is, -1 from me. Either we only keep two concepts (Address and 
> Network), or if we introduce a third one (AddressWithMask, whatever) for 
> added practicality; but we shouldn't blur the line between the two former 
> canonical concepts under the pretext that a platypus-like Address might 
> be helpful in some particular situations.

I completely agree with this. By keeping the concepts distinct we can 
catch mis-uses earlier. If needed, convenience functions (or classes, or 
whatever) could be layered on top. But the underlying concepts need to 
be clear.

Eric.


From ncoghlan at gmail.com  Thu Sep 17 14:32:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 17 Sep 2009 22:32:49 +1000
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <4AB2282A.3050902@trueblade.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB03909.3090608@trueblade.com>	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>	<200909162053.00354.steve@pearwood.info>	<4AB0C7A9.3030001@gmail.com>	<4AB15FD3.5040804@canterbury.ac.nz>	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>	<4AB17B29.8050708@canterbury.ac.nz>	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>	<4AB19F44.8010102@canterbury.ac.nz>	<h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com>
Message-ID: <4AB22C71.1040205@gmail.com>

Eric Smith wrote:
> Antoine Pitrou wrote:
>> As it is, -1 from me. Either we only keep two concepts (Address and
>>  Network), or if we introduce a third one (AddressWithMask,
>> whatever) for added practicality; but we shouldn't blur the line
>> between the two former canonical concepts under the pretext that a
>> platypus-like Address might be helpful in some particular
>> situations.
> 
> I completely agree with this. By keeping the concepts distinct we can
>  catch mis-uses earlier. If needed, convenience functions (or
> classes, or whatever) could be layered on top. But the underlying
> concepts need to be clear.

That would be my view as well (this includes getting rid of the current
duality of IPNetwork by dropping the ip property, only accepting the
network address in the normal constructor and having a separate
constructor that allows a network object to be created from an arbitrary
  host address and a netmask)

Is-a-2.x-str-a-sequence-of-ASCII-characters-or-a-chunk-of-binary-data?'ly,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From eric at trueblade.com  Thu Sep 17 14:39:57 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 17 Sep 2009 08:39:57 -0400
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <4AB22C71.1040205@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB03909.3090608@trueblade.com>	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>	<200909162053.00354.steve@pearwood.info>	<4AB0C7A9.3030001@gmail.com>	<4AB15FD3.5040804@canterbury.ac.nz>	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>	<4AB17B29.8050708@canterbury.ac.nz>	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>	<4AB19F44.8010102@canterbury.ac.nz>	<h8t88h$9q$1@ger.gmane.org>	<4AB2282A.3050902@trueblade.com>
	<4AB22C71.1040205@gmail.com>
Message-ID: <4AB22E1D.30907@trueblade.com>

Nick Coghlan wrote:
> Eric Smith wrote:
>> Antoine Pitrou wrote:
>>> As it is, -1 from me. Either we only keep two concepts (Address and
>>>  Network), or if we introduce a third one (AddressWithMask,
>>> whatever) for added practicality; but we shouldn't blur the line
>>> between the two former canonical concepts under the pretext that a
>>> platypus-like Address might be helpful in some particular
>>> situations.
>> I completely agree with this. By keeping the concepts distinct we can
>>  catch mis-uses earlier. If needed, convenience functions (or
>> classes, or whatever) could be layered on top. But the underlying
>> concepts need to be clear.
> 
> That would be my view as well (this includes getting rid of the current
> duality of IPNetwork by dropping the ip property, only accepting the
> network address in the normal constructor and having a separate
> constructor that allows a network object to be created from an arbitrary
>   host address and a netmask)

Yes, I think that's the best plan. I could live without AddressWithMask 
(or whatever name), but it would be a nice convenience.

> Is-a-2.x-str-a-sequence-of-ASCII-characters-or-a-chunk-of-binary-data?'ly,

LOL! Perfect.

Eric.

From ncoghlan at gmail.com  Thu Sep 17 14:40:47 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 17 Sep 2009 22:40:47 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090917T134809-168@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>	<4AB17B29.8050708@canterbury.ac.nz>	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>	<4AB1A39B.7090003@canterbury.ac.nz>	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org>
Message-ID: <4AB22E4F.40509@gmail.com>

Antoine Pitrou wrote:
> Peter Moody <peter <at> hda3.com> writes:
>> the address with all of the hosts bits masked to zero is most commonly
>> referred to as the network address. same as the address with all of
>> the host bits set to one is called the broadcast address. calling it
>> something like base_address or min_address will cause quite a bit more
>> confusion.
> 
> Quite a bit less IMO. I'm not a network specialist, but the "network address"

Nah, network address is perfectly explicit - it's what you get when you
bitwise and any host within that network with the netmask.

Where it becomes confusing is that we have a property called "network"
that returns an IPAddress object rather than an IPNetwork object.

People that are network specialists would grasp that fairly easily, but
we try to design standard library APIs so that they're novice friendly
as well. And the above situation isn't novice friendly.

To be honest, given the indexing behaviour, I'm -1 on the idea of giving
the network address or broadcast address attribute names *at all*. Consider:

  network_address = my_net[0]
  broadcast_address = my_net[-1]

The only way the network address could match the number of characters in
the indexing method is if you just called it the network id (i.e.
my_net.id). For the broadcast address, there is no name that even comes
close to matching the indexing approach for brevity.

And since IPNetwork is a container for IPAddress instances, the indexing
approach means there is zero ambiguity regarding the return type.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From rdmurray at bitdance.com  Thu Sep 17 14:49:29 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 08:49:29 -0400 (EDT)
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <4AB22C71.1040205@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
Message-ID: <Pine.LNX.4.64.0909170843560.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 22:32, Nick Coghlan wrote:
> Eric Smith wrote:
>> Antoine Pitrou wrote:
>>> As it is, -1 from me. Either we only keep two concepts (Address and
>>>  Network), or if we introduce a third one (AddressWithMask,
>>> whatever) for added practicality; but we shouldn't blur the line
>>> between the two former canonical concepts under the pretext that a
>>> platypus-like Address might be helpful in some particular
>>> situations.
>>
>> I completely agree with this. By keeping the concepts distinct we can
>>  catch mis-uses earlier. If needed, convenience functions (or
>> classes, or whatever) could be layered on top. But the underlying
>> concepts need to be clear.
>
> That would be my view as well (this includes getting rid of the current
> duality of IPNetwork by dropping the ip property, only accepting the
> network address in the normal constructor and having a separate
> constructor that allows a network object to be created from an arbitrary
>  host address and a netmask)

I agree as well.  If we cannot get consensus on adding a third (or
rather fifth and sixth including IPv6) classes for AddressWithMask
(and I am no longer an advocate of doing so), then we leave it out and,
as suggested by Scott, wait and see what develops and add it later if
there is demand for it.

--David

From rdmurray at bitdance.com  Thu Sep 17 14:58:08 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 08:58:08 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909170850310.17020@kimball.webabinitio.net>

On Wed, 16 Sep 2009 at 20:26, Peter Moody wrote:
> On Wed, Sep 16, 2009 at 7:48 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> I'm not sure what usefulness the zero address on its own
>> has, but if it's considered useful enough to have an
>> attribute for it, calling it something like 'base_address'
>> would be less confusing.
>
> the address with all of the hosts bits masked to zero is most commonly
> referred to as the network address. same as the address with all of
> the host bits set to one is called the broadcast address. calling it
> something like base_address or min_address will cause quite a bit more
> confusion.

If 'network address' is the most common designation for this number
(and it may well be), then 'network.address' and 'network.broadcast'
would seem to be the best choices for names.  But there is definitely
an attraction to the idea (I think it was Nick's?) of dropping these
attributes entirely and just using indexing to get the addresses.
That will make perfect sense to network professionals, and by putting
appropriate words in the documentation will educate people new to IP
addressing concepts.  The advantage is that it is completely unambiguous:
the network address really is the first address in the network range,
and the broadcast really is the last, in common usage.  Exceptional cases
can then be handled by custom subclasses, without having someone who
has to handle weird broadcast addresses (for example) submitting a bug
request to make '.broadcast' settable.

--David

From eric at trueblade.com  Thu Sep 17 15:25:45 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 17 Sep 2009 09:25:45 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB22E4F.40509@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB0C7A9.3030001@gmail.com>	<4AB15FD3.5040804@canterbury.ac.nz>	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>	<4AB17B29.8050708@canterbury.ac.nz>	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>	<4AB1A39B.7090003@canterbury.ac.nz>	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>	<loom.20090917T134809-168@post.gmane.org>
	<4AB22E4F.40509@gmail.com>
Message-ID: <4AB238D9.1020204@trueblade.com>

Nick Coghlan wrote:
> To be honest, given the indexing behaviour, I'm -1 on the idea of giving
> the network address or broadcast address attribute names *at all*. Consider:
> 
>   network_address = my_net[0]
>   broadcast_address = my_net[-1]

My only concern with this is a possible performance issue with v6 
networks. Would this be implemented such that [-1] doesn't need to 
iterate through the (possibly large) address space of a v6 network?

I think it could (and would) be implemented efficiently, so I'm +1 on 
not giving these names, but adding this "recipe" to the documentation 
instead.

Eric.


From drkjam at gmail.com  Thu Sep 17 16:26:16 2009
From: drkjam at gmail.com (DrKJam)
Date: Thu, 17 Sep 2009 15:26:16 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org> <4AB22E4F.40509@gmail.com>
	<4AB238D9.1020204@trueblade.com>
	<538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com>
Message-ID: <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com>

Please can we have the following RFCs added to the references section that
cover many of the aspects covered by this PEP?

RFC 791 - Internet Protocol
RFC 1918 - Address Allocation for Private Internets
RFC 3330 - Special-Use IPv4 Addresses
RFC 4291 - IPv6 Addressing Architecture
RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet Address

Dave M.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090917/c7eb9bbc/attachment.htm>

From solipsis at pitrou.net  Thu Sep 17 16:32:05 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Sep 2009 14:32:05 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
Message-ID: <h8th95$2mh$1@ger.gmane.org>

Le Mon, 14 Sep 2009 09:44:12 -0700, Peter Moody a ?crit?:
> Folks, Guido,
> 
> I believe PEP 3144 is ready for your review.  When you get a chance, can
> you take a look/make a pronouncement?

I might add that, according to the whole discussion, it seems not all of 
the API is exposed in the PEP. Would it be possible to cover more of the 
API in the PEP, to be sure we don't overlook anything?


From daniel at stutzbachenterprises.com  Thu Sep 17 16:37:56 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Thu, 17 Sep 2009 09:37:56 -0500
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org> <4AB22E4F.40509@gmail.com>
	<4AB238D9.1020204@trueblade.com>
	<538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com>
	<538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com>
Message-ID: <eae285400909170737j6b67088bt72ca16d449cfbf7d@mail.gmail.com>

On Thu, Sep 17, 2009 at 9:26 AM, DrKJam <drkjam at gmail.com> wrote:

> Please can we have the following RFCs added to the references section that
> cover many of the aspects covered by this PEP?
>
> RFC 791 - Internet Protocol
> RFC 1918 - Address Allocation for Private Internets
> RFC 3330 - Special-Use IPv4 Addresses
> RFC 4291 - IPv6 Addressing Architecture
> RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet Address
>

+1

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090917/6a0aedf4/attachment.htm>

From peter at hda3.com  Thu Sep 17 16:40:47 2009
From: peter at hda3.com (Peter Moody)
Date: Thu, 17 Sep 2009 07:40:47 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org> <4AB22E4F.40509@gmail.com>
	<4AB238D9.1020204@trueblade.com>
	<538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com>
	<538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com>
Message-ID: <8517e9350909170740kf2c1d46t2b118c6835fbefce@mail.gmail.com>

On Thu, Sep 17, 2009 at 7:26 AM, DrKJam <drkjam at gmail.com> wrote:
> Please can we have the following RFCs added to the references section that
> cover many of the aspects covered by this PEP?
>
> RFC 791 - Internet Protocol
> RFC 1918 - Address Allocation for Private Internets
> RFC 3330 - Special-Use IPv4 Addresses
> RFC 4291 - IPv6 Addressing Architecture
> RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet Address

Sure, can you give me an idea of what specifically you're looking to
the rfc's to clarify? simply adding 5 rfc's to the the PEP wouldn't
seem to enhance its readability.

> Dave M.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>
>

From drkjam at gmail.com  Thu Sep 17 16:44:04 2009
From: drkjam at gmail.com (DrKJam)
Date: Thu, 17 Sep 2009 15:44:04 +0100
Subject: [Python-Dev] Fwd:  PEP 3144 review.
In-Reply-To: <538a660a0909170717t30834632q8b221b8654f8e8cf@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org> <4AB22E4F.40509@gmail.com>
	<538a660a0909170717t30834632q8b221b8654f8e8cf@mail.gmail.com>
Message-ID: <538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com>

>
> Antoine Pitrou wrote:
> > Peter Moody <peter <at> hda3.com> writes:
> >> the address with all of the hosts bits masked to zero is most commonly
> >> referred to as the network address. same as the address with all of
> >> the host bits set to one is called the broadcast address. calling it
> >> something like base_address or min_address will cause quite a bit more
> >> confusion.
> >
> > Quite a bit less IMO. I'm not a network specialist, but the "network
> address"
>
> Nah, network address is perfectly explicit - it's what you get when you
> bitwise and any host within that network with the netmask.
>
> Where it becomes confusing is that we have a property called "network"
> that returns an IPAddress object rather than an IPNetwork object.
>
> People that are network specialists would grasp that fairly easily, but
> we try to design standard library APIs so that they're novice friendly
> as well. And the above situation isn't novice friendly.
>
> To be honest, given the indexing behaviour, I'm -1 on the idea of giving
> the network address or broadcast address attribute names *at all*.
> Consider:
>
>  network_address = my_net[0]
>  broadcast_address = my_net[-1]
>
> The only way the network address could match the number of characters in
> the indexing method is if you just called it the network id (i.e.
> my_net.id). For the broadcast address, there is no name that even comes
> close to matching the indexing approach for brevity.
>
> And since IPNetwork is a container for IPAddress instances, the indexing
> approach means there is zero ambiguity regarding the return type.
>
> -1 from me. I'm happy to see indexing made available alongside specific
properties/methods that expose the network (without mask) and broadcast
addresses for a given network object, but not to remove them from the
interface entirely in favour of indexing alone.

While it seems like a good idea, in practice it just won't work or will at
least be sub-optimal and violates the rule of least surprise (for IP
libraries anyway). I struggled along with this approach in several early
versions of netaddr but had to cave in to pressure from users after repeated
questions about where to find the broadcast and network (without mask)
addresses!

Granted, there are decisions to be made about exactly what the
properties/methods should be named to avoid ambiguity, but they are
important enough to be given access to in their own right. Details in
docstrings help too ;-) 'network' and 'broadcast' are very much the
convention used pretty much everywhere (including libraries found in other
languages such as Ruby and Perl).

IPv6 doesn't support the notion of a broadcast address as part of a CIDR
network block at all. AFAIK, it is a perfect legitimate for the last address
in an IPv6 block to be used to configure a network interface. The IPv6
network object interface should possibly leave out the broadcast
property/method altogether although there are reasons to keep it in for the
sake of completeness and API consistency. The pros and cons of this need to
be considered.

BTW, has anyone considered use of the term *CIDR to refer to an address +
mask object? It would certainly be less controversial than *Network which
already has too many overlapping meanings elsewhere in the interface?
Obviously we'd still have the issue of what to do with the host bits to the
right of the supplied mask (keep or discard). This is not a clear cut choice
of one or the other as it is entirely based on context. For configuring
routes, you would likely always want to discard these bits (or at least
Cisco does when adding routes). For configuring a network interface you
would most certainly want to keep them!


> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/drkjam%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090917/8f65edc7/attachment.htm>

From peter at hda3.com  Thu Sep 17 16:38:08 2009
From: peter at hda3.com (Peter Moody)
Date: Thu, 17 Sep 2009 07:38:08 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <h8th95$2mh$1@ger.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<h8th95$2mh$1@ger.gmane.org>
Message-ID: <8517e9350909170738j6a5b805fp619006eb0a8bf5bb@mail.gmail.com>

On Thu, Sep 17, 2009 at 7:32 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Mon, 14 Sep 2009 09:44:12 -0700, Peter Moody a ?crit?:
>> Folks, Guido,
>>
>> I believe PEP 3144 is ready for your review. ?When you get a chance, can
>> you take a look/make a pronouncement?
>
> I might add that, according to the whole discussion, it seems not all of
> the API is exposed in the PEP. Would it be possible to cover more of the
> API in the PEP, to be sure we don't overlook anything?

sounds good.

todo += more api in the pep

Cheers,
/peter

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From peter at hda3.com  Thu Sep 17 16:48:23 2009
From: peter at hda3.com (Peter Moody)
Date: Thu, 17 Sep 2009 07:48:23 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB238D9.1020204@trueblade.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org> <4AB22E4F.40509@gmail.com>
	<4AB238D9.1020204@trueblade.com>
Message-ID: <8517e9350909170748g7ec8b1a0u926e26de6907b494@mail.gmail.com>

On Thu, Sep 17, 2009 at 6:25 AM, Eric Smith <eric at trueblade.com> wrote:
> Nick Coghlan wrote:
>>
>> To be honest, given the indexing behaviour, I'm -1 on the idea of giving
>> the network address or broadcast address attribute names *at all*.
>> Consider:
>>
>> ?network_address = my_net[0]
>> ?broadcast_address = my_net[-1]
>
> My only concern with this is a possible performance issue with v6 networks.
> Would this be implemented such that [-1] doesn't need to iterate through the
> (possibly large) address space of a v6 network?
>
> I think it could (and would) be implemented efficiently, so I'm +1 on not
> giving these names, but adding this "recipe" to the documentation instead.

indexing is plenty efficient, but the problem is that these names for
these attributes are common to the point of causing confusion if
they're omitted.

Cheers,
/peter

> Eric.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From eric at trueblade.com  Thu Sep 17 16:54:49 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 17 Sep 2009 10:54:49 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909170748g7ec8b1a0u926e26de6907b494@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>	
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>	
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>	
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>	
	<4AB1A39B.7090003@canterbury.ac.nz>	
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>	
	<loom.20090917T134809-168@post.gmane.org>
	<4AB22E4F.40509@gmail.com>	 <4AB238D9.1020204@trueblade.com>
	<8517e9350909170748g7ec8b1a0u926e26de6907b494@mail.gmail.com>
Message-ID: <4AB24DB9.9050605@trueblade.com>

Peter Moody wrote:

> indexing is plenty efficient, but the problem is that these names for
> these attributes are common to the point of causing confusion if
> they're omitted.

After thinking about it some more, I'm okay with names for [-1] and [0]. 
I like .broadcast, and I can live with .network (although it's 
confusing, since in my view it should return an address, not a network).

Eric.

From solipsis at pitrou.net  Thu Sep 17 16:56:57 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Sep 2009 14:56:57 +0000 (UTC)
Subject: [Python-Dev] Fwd:  PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org>
	<4AB22E4F.40509@gmail.com>
	<538a660a0909170717t30834632q8b221b8654f8e8cf@mail.gmail.com>
	<538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com>
Message-ID: <loom.20090917T165252-384@post.gmane.org>

DrKJam <drkjam <at> gmail.com> writes:
> 
> BTW, has anyone considered use of the term *CIDR to refer to an address + mask
object?

This sounds too obscure for a non-specialist.

> Obviously we'd still have the issue of what to do with the host bits to the
right of the supplied mask (keep or discard). This is not a clear cut choice of
one or the other as it is entirely based on context. For configuring routes, you
would likely always want to discard these bits (or at least Cisco does when
adding routes). For configuring a network interface you would most certainly
want to keep them!

When configuring a network interface you are not specifying a network, you are
specifying two things:
- a host address to which the interface is bound (all bits significant)
- a network to which the interface is connected (lower bits non-significant)

As someone pointed out, the fact that you are specifying a single string like
"10.0.1.7/8" is just a notational convenience.

Therefore, it seems clear to me that Network objects should ignore (and perhaps
zero out) the host bits at the right of the netmask. When you configure an
interface from the aforementioned notational convenience, you should retrieve
two objects, a Network, and an Address.

Regards

Antoine.



From rdmurray at bitdance.com  Thu Sep 17 17:23:19 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 11:23:19 -0400 (EDT)
Subject: [Python-Dev] Fwd:  PEP 3144 review.
In-Reply-To: <538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org> <4AB22E4F.40509@gmail.com>
	<538a660a0909170717t30834632q8b221b8654f8e8cf@mail.gmail.com>
	<538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909171117530.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 15:44, DrKJam wrote:
> Granted, there are decisions to be made about exactly what the
> properties/methods should be named to avoid ambiguity, but they are
> important enough to be given access to in their own right. Details in
> docstrings help too ;-) 'network' and 'broadcast' are very much the
> convention used pretty much everywhere (including libraries found in other
> languages such as Ruby and Perl).

Interesting.  Out of curiosity I searched for 'perl ip address' and
found Net::IP (http://search.cpan.org/dist/Net-IP/IP.pm).  This package
appears to use the same object class for both addresses and networks.
It has neither a 'network' nor a 'broadcast' attribute; instead it has
'ip' (it isn't clear from the docs if that returns the network address
or the IP that was passed in to the constructor) and 'last_ip'.

--David

From nate at binkert.org  Thu Sep 17 17:31:54 2009
From: nate at binkert.org (nathan binkert)
Date: Thu, 17 Sep 2009 08:31:54 -0700
Subject: [Python-Dev] re.Scanner and string.Template
Message-ID: <217accd40909170831p70b48c51te2273011cb516749@mail.gmail.com>

I've been working on code that works a lot like string.Template and in
the process, I stumbled on re.Scanner.  I have two questions.
1) Is re.Scanner ever going to be official?  Can I count on it being
in future versions of python?  It's been there for a really long time,
so I assume so, but something that's undocumented seems that it could
be dropped easily.
2) Is there a reason that string.Template doesn't use re.Scanner?  My
understanding of re.Scanner is that it is exactly what you want for
something like string.Template.

Thanks,

  Nate Binkert

From benjamin at python.org  Thu Sep 17 17:34:07 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 17 Sep 2009 10:34:07 -0500
Subject: [Python-Dev] re.Scanner and string.Template
In-Reply-To: <217accd40909170831p70b48c51te2273011cb516749@mail.gmail.com>
References: <217accd40909170831p70b48c51te2273011cb516749@mail.gmail.com>
Message-ID: <1afaf6160909170834o761694a6i733559cdba81f0f5@mail.gmail.com>

2009/9/17 nathan binkert <nate at binkert.org>:
> I've been working on code that works a lot like string.Template and in
> the process, I stumbled on re.Scanner. ?I have two questions.
> 1) Is re.Scanner ever going to be official? ?Can I count on it being
> in future versions of python? ?It's been there for a really long time,
> so I assume so, but something that's undocumented seems that it could
> be dropped easily.

See http://bugs.python.org/issue5337.

> 2) Is there a reason that string.Template doesn't use re.Scanner? ?My
> understanding of re.Scanner is that it is exactly what you want for
> something like string.Template.

Perhaps because it wasn't documented. :)



-- 
Regards,
Benjamin

From peter at hda3.com  Thu Sep 17 18:16:46 2009
From: peter at hda3.com (Peter Moody)
Date: Thu, 17 Sep 2009 09:16:46 -0700
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <4AB22C71.1040205@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
Message-ID: <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>

On Thu, Sep 17, 2009 at 5:32 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Eric Smith wrote:
>> Antoine Pitrou wrote:
>>> As it is, -1 from me. Either we only keep two concepts (Address and
>>> ?Network), or if we introduce a third one (AddressWithMask,
>>> whatever) for added practicality; but we shouldn't blur the line
>>> between the two former canonical concepts under the pretext that a
>>> platypus-like Address might be helpful in some particular
>>> situations.
>>
>> I completely agree with this. By keeping the concepts distinct we can
>> ?catch mis-uses earlier. If needed, convenience functions (or
>> classes, or whatever) could be layered on top. But the underlying
>> concepts need to be clear.
>
> That would be my view as well (this includes getting rid of the current
> duality of IPNetwork by dropping the ip property, only accepting the
> network address in the normal constructor and having a separate
> constructor that allows a network object to be created from an arbitrary
> ?host address and a netmask)

Again, the same error-catching functionality can be obtained through
an option to the constructor. network and broadcast attributes can be
renamed to .\1_address to alleviate confusion as well.

I mentioned before that IPy's insistence on receiving masked out
networks was one of the main reasons I wrote ipaddr to begin with.
Having ipaddr mimic this behavior would make it significantly less
useful. Removing functionality in the name of avoiding confusion
doesn't make sense when the same confusion can be alleviated w/o the
loss.

off to patch the pep and implement some of the non controversial changes.
/peter

> Is-a-2.x-str-a-sequence-of-ASCII-characters-or-a-chunk-of-binary-data?'ly,
> Nick.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From solipsis at pitrou.net  Thu Sep 17 18:26:11 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 17 Sep 2009 16:26:11 +0000 (UTC)
Subject: [Python-Dev] conceptual clarity
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
Message-ID: <loom.20090917T182317-245@post.gmane.org>

Peter Moody <peter <at> hda3.com> writes:
> 
> Again, the same error-catching functionality can be obtained through
> an option to the constructor. network and broadcast attributes can be
> renamed to .\1_address to alleviate confusion as well.

My remark was not targetting error-catching of non-zero low bits, it was about
the suggestion to have a "network" or "mask" attribute on Address objects.

Regards

Antoine.



From peter at hda3.com  Thu Sep 17 18:41:18 2009
From: peter at hda3.com (Peter Moody)
Date: Thu, 17 Sep 2009 09:41:18 -0700
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <loom.20090917T182317-245@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
	<loom.20090917T182317-245@post.gmane.org>
Message-ID: <8517e9350909170941o3f5ae0fdy90668d1df0515ca5@mail.gmail.com>

On Thu, Sep 17, 2009 at 9:26 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Peter Moody <peter <at> hda3.com> writes:
>>
>> Again, the same error-catching functionality can be obtained through
>> an option to the constructor. network and broadcast attributes can be
>> renamed to .\1_address to alleviate confusion as well.
>
> My remark was not targetting error-catching of non-zero low bits, it was about
> the suggestion to have a "network" or "mask" attribute on Address objects.

If I'm reading your reply correctly, then I agree that having a
network attribute on an address object doesn't make sense.

My remark was targeting Eric's reply of, "By keeping the concepts
distinct we can catch mis-uses earlier."

Cheers,
/peter

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From rdmurray at bitdance.com  Thu Sep 17 19:14:59 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 13:14:59 -0400 (EDT)
Subject: [Python-Dev] Misc/maintainers.rst
Message-ID: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>

I floated a proposal on stdlib-sig to create a file named
Misc/maintainers.rst.  The purpose of this file is to collect knowledge
about who knows which modules well enough to make decision about issues
in the tracker when the participants in the issue aren't sure, and to
write down the community knowledge about who has special interest and
expertise in specific topic areas.

This proposal was met with approval and, after a couple of small
modifications of the proposal, no dissent; so I've created the skeleton
of the file.  I've filled in some of the blanks where I had personal
knowledge, was told info on IRC, from the two referenced PEPs, and
from PEP 11.

Feel free to respond with comments about the header text, but more
importantly let me know what you know about who (especially if it is you)
should be listed for each module, platform, and topic, and feel free to
suggest additional topics.  My goal is to record the community knowledge.

Another topic of discussion that is orthogonal to filling in the table is
whether or not to publish it outside the repository.  Jesse would like to
see it included in the Python Documentation, and Georg has suggested the
possibility of creating a separate, sphinx-based, automatically-uploaded
document collection in the repository to contain this and related
information (Misc/devdocs?).

The module list was built from the py3k documentation module index, with
the addition of 'pybench' from PEP 291.  If there are other modules/tools
that are missing, or submodules that should be broken out into
separate lines, please let me know.

After the initial flurry of updates and comments dies down I will check
this in.

--David

----------------------------------------------------------------------
Maintainers Index
=================

This document cross references Python Modules (first table) and platforms
(second table) with the Tracker user names of people who are experts
and/or resources for that module or platform.  This list is intended
to be used by issue submitters, issue triage people, and other issue
participants to find people to add to the nosy list or to contact
directly by email for help and decisions on feature requests and bug
fixes.  People on this list may be asked to render final judgement on a
feature or bug.  If no active maintainer is listed for a given module,
then questionable changes should go to python-dev, while any other issues
can and should be decided by any committer.

The last part of this document is a third table, listing broader topic
areas in which various people have expertise.  These people can also
be contacted for help, opinions, and decisions when issues involve
their areas.

If a listed maintainer does not respond to requests for comment for an
extended period (three weeks or more), they should be marked as inactive
in this list by placing the word 'inactive' in parenthesis behind their
tracker id.  They are of course free to remove that inactive mark at
any time.

Committers should update this table as their areas of expertise widen.
New topics may be added to the third table at will.

The existence of this list is not meant to indicate that these people
*must* be contacted for decisions; it is, rather, a resource to be used
by non-committers to find responsible parties, and by committers who do
not feel qualified to make a decision in a particular context.

See also `PEP 291`_ and `PEP 360`_ for information about certain modules
with special rules.

.. _`PEP 291`: http://www.python.org/dev/peps/pep-0291/
.. _`PEP 360`: http://www.python.org/dev/peps/pep-0360/


==================  ===========
Module              Maintainers
==================  ===========
__future__
__main__            gvanrossum
_dummy_thread       brett.cannon
_thread
abc
aifc                r.david.murray
array
ast
asynchat            josiahcarlson
asyncore            josiahcarlson
atexit
audioop
base64
bdb
binascii
binhex
bisect
builtins
bz2
calendar
cgi
cgitb
chunk
cmath
cmd
code
codecs
codeop
collections
colorsys
compileall
configparser
contextlib
copy
copyreg
cProfile
crypt
csv
ctypes              theller
curses
datetime
dbm
decimal
difflib
dis
distutils           tarek
doctest
dummy_threading     brett.cannon
email               barry
encodings
errno
exceptions
fcntl
filecmp
fileinput
fnmatch
formatter
fpectl
fractions
ftplib
functools
gc
getopt
getpass
gettext
glob
grp
gzip
hashlib
heapq
hmac
html
http
imaplib
imghdr
imp
importlib           brett.cannon
inspect
io                  pitrou, benjamin.peterson
itertools
json
keyword
lib2to3             benjamin.peterson
linecache
locale
logging             vsajip
macpath
mailbox
mailcap
marshal
math
mimetypes
mmap
modulefinder        theller, jvr
msilib
msvcrt
multiprocessing     jnoller
netrc
nis
nntplib
numbers
operator
optparse            aronacher
os
ossaudiodev
parser
pdb
pickle
pickletools
pipes
pkgutil
platform            lemburg
plistlib
poplib
posix
pprint
pstats
pty
pwd
py_compile
pybench             lemburg
pyclbr
pydoc
queue
quopri
random
re
readline
reprlib
resource
rlcompleter
runpy
sched
select
shelve
shlex
shutil
signal
site
smtpd
smtplib
sndhdr
socket
socketserver
spwd
sqlite3
ssl
stat
string
stringprep
struct
subprocess          astrand (inactive)
sunau
symbol
symtable
sys
syslog
tabnanny
tarfile
telnetlib
tempfile
termios
test
textwrap
threading
time
timeit
tkinter             gpolo
token
tokenize
trace
traceback
tty
turtle              gregorlingl
types
unicodedata
unittest            michael.foord
urllib
uu
uuid
warnings
wave
weakref
webbrowser          georg.brandl
winreg
winsound
wsgiref             pje
xdrlib
xml                 loewis
xml.etree           effbot (inactive)
xmlrpc              loewis
zipfile
zipimport
zlib
==================  ===========


==================  ===========
Platform            Maintainer
------------------  -----------
AIX
Cygwin              jlt63
FreeBSD
Linux
Mac                 ronaldoussoren
NetBSD1
OS2/EMX             aimacintyre
Solaris
HP-UX
==================  ===========


==================  ===========
Interest Area       Maintainers
------------------  -----------
algorithms
ast/compiler
autoconf
bsd
buildbots
data formats
database
documentation       georg.brandl
GUI
i18n
import machinery    brett.cannon
io                  pitrou, benjamin.peterson
locale
makefiles
mathematics         marketdickinson, eric.smith
memory management
networking
packaging 
release management
time and dates
testing             michael.foord
threads
unicode 
windows
==================  ===========

From steve at pearwood.info  Thu Sep 17 19:27:33 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 18 Sep 2009 03:27:33 +1000
Subject: [Python-Dev] Fwd:  PEP 3144 review.
In-Reply-To: <538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<538a660a0909170717t30834632q8b221b8654f8e8cf@mail.gmail.com>
	<538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com>
Message-ID: <200909180327.34065.steve@pearwood.info>

On Fri, 18 Sep 2009 12:44:04 am DrKJam wrote:

> IPv6 doesn't support the notion of a broadcast address as part of a
> CIDR network block at all. AFAIK, it is a perfect legitimate for the
> last address in an IPv6 block to be used to configure a network
> interface. The IPv6 network object interface should possibly leave
> out the broadcast property/method altogether although there are
> reasons to keep it in for the sake of completeness and API
> consistency. The pros and cons of this need to be considered.

This seems to me to be rather weird: that for the sake of completeness 
and consistency you're considering to give an object which (allegedly) 
has no notion of "broadcast address" a broadcast address method.

If so, I can see three options. Given an appropriate IPv6 object:

obj.broadcast() always returns None.

obj.broadcast() always raises an exception.

obj.broadcast() always returns something which is not a broadcast 
address.

I would hope the third option isn't being seriously considered! Are 
there any reasons to consider the first two?



-- 
Steven D'Aprano

From rdmurray at bitdance.com  Thu Sep 17 19:32:13 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 13:32:13 -0400 (EDT)
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909171329450.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 09:16, Peter Moody wrote:
> I mentioned before that IPy's insistence on receiving masked out
> networks was one of the main reasons I wrote ipaddr to begin with.
> Having ipaddr mimic this behavior would make it significantly less
> useful. Removing functionality in the name of avoiding confusion
> doesn't make sense when the same confusion can be alleviated w/o the
> loss.

We're suggesting moving that functionality (accepting a non-masked IP
plus netmask and returning the corresponding Network object) into
a different constructor, not eliminating the functionality.

--David

From peter at hda3.com  Thu Sep 17 19:38:37 2009
From: peter at hda3.com (Peter Moody)
Date: Thu, 17 Sep 2009 10:38:37 -0700
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <Pine.LNX.4.64.0909171329450.17020@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
	<Pine.LNX.4.64.0909171329450.17020@kimball.webabinitio.net>
Message-ID: <8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com>

On Thu, Sep 17, 2009 at 10:32 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Thu, 17 Sep 2009 at 09:16, Peter Moody wrote:
>>
>> I mentioned before that IPy's insistence on receiving masked out
>> networks was one of the main reasons I wrote ipaddr to begin with.
>> Having ipaddr mimic this behavior would make it significantly less
>> useful. Removing functionality in the name of avoiding confusion
>> doesn't make sense when the same confusion can be alleviated w/o the
>> loss.
>
> We're suggesting moving that functionality (accepting a non-masked IP
> plus netmask and returning the corresponding Network object) into
> a different constructor, not eliminating the functionality.

Do you mean a new constructor function, ala IPAddress and IPNetwork
(so the current classes remain largely the same and the various
constructors enforce certain restrictions) or something else?

At this point, I'm getting confused with the intermingling of class
and constructor names.

Cheers,
/peter

> --David
>

From g.brandl at gmx.net  Thu Sep 17 19:38:50 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 17 Sep 2009 19:38:50 +0200
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
Message-ID: <h8tsb9$bkq$1@ger.gmane.org>

R. David Murray schrieb:
> I floated a proposal on stdlib-sig to create a file named
> Misc/maintainers.rst.  The purpose of this file is to collect knowledge
> about who knows which modules well enough to make decision about issues
> in the tracker when the participants in the issue aren't sure, and to
> write down the community knowledge about who has special interest and
> expertise in specific topic areas.
> 
> This proposal was met with approval and, after a couple of small
> modifications of the proposal, no dissent; so I've created the skeleton
> of the file.  I've filled in some of the blanks where I had personal
> knowledge, was told info on IRC, from the two referenced PEPs, and
> from PEP 11.
> 
> Feel free to respond with comments about the header text, but more
> importantly let me know what you know about who (especially if it is you)
> should be listed for each module, platform, and topic, and feel free to
> suggest additional topics.  My goal is to record the community knowledge.

One thing I'd like to see in the list are real names of committers,
not tracker names.  Of course, people looking for people to assign a bug
to should not have to search for the tracker name, so I'd like to make
another request (that Brett already made when we switched trackers):

  Could we *please* have tracker names that match the committer names?

(This doesn't even need to be done by the individual users, I would
volunteer to rename all committer accounts and notify them by email that
their name changed.  This will also be coordinated with the new names
used for Mercurial commits, if a change will be made.)

> Another topic of discussion that is orthogonal to filling in the table is
> whether or not to publish it outside the repository.  Jesse would like to
> see it included in the Python Documentation, and Georg has suggested the
> possibility of creating a separate, sphinx-based, automatically-uploaded
> document collection in the repository to contain this and related
> information (Misc/devdocs?).

For those who aren't on stdlib-sig, I'd like to elaborate a bit on that:
There are quite a few resources for and about Python core development,
but they aren't very accessible.  For example, there's Misc/developers.txt
and the upcoming maintainers.txt.  Then there's the dev FAQ, but it's not
maintained where developers usually look, but on the website.  Etc.
So the plan would be to consolidate these into another set of rst docs,
having them in the repo, editable by every committer, as well as published
somewhere on python.org (devdocs.python.org or somesuch).

cheers,
Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From steve at pearwood.info  Thu Sep 17 19:44:31 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 18 Sep 2009 03:44:31 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB22E4F.40509@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org>
	<4AB22E4F.40509@gmail.com>
Message-ID: <200909180344.31794.steve@pearwood.info>

On Thu, 17 Sep 2009 10:40:47 pm Nick Coghlan wrote:
> Antoine Pitrou wrote:
> > Peter Moody <peter <at> hda3.com> writes:
> >> the address with all of the hosts bits masked to zero is most
> >> commonly referred to as the network address. same as the address
> >> with all of the host bits set to one is called the broadcast
> >> address. calling it something like base_address or min_address
> >> will cause quite a bit more confusion.
> >
> > Quite a bit less IMO. I'm not a network specialist, but the
> > "network address"
>
> Nah, network address is perfectly explicit - it's what you get when
> you bitwise and any host within that network with the netmask.

[soapbox]
I really wish people would stop misusing the word "explicit" to mean 
(apparently) "I know how this is commonly defined".

To a non-specialist, "the network address" is ambiguous. There are many 
addresses in a network, and none of them are the entire network. It's 
like saying, given a list [2, 4, 8, 12], what's "the list item"?

If network specialists have a convention of calling hostadd&netmask "the 
network address", that's fine. It's jargon which seems illogical to me, 
but if it's useful and in common use, specialists will expect us to 
support it. But it's not explicit, it's a definition.


> Where it becomes confusing is that we have a property called
> "network" that returns an IPAddress object rather than an IPNetwork
> object.
>
> People that are network specialists would grasp that fairly easily,
> but we try to design standard library APIs so that they're novice
> friendly as well. And the above situation isn't novice friendly.

+1000 on this!


> To be honest, given the indexing behaviour, I'm -1 on the idea of
> giving the network address or broadcast address attribute names *at
> all*. Consider:
>
>   network_address = my_net[0]
>   broadcast_address = my_net[-1]
>
> The only way the network address could match the number of characters
> in the indexing method is if you just called it the network id (i.e.
> my_net.id). For the broadcast address, there is no name that even
> comes close to matching the indexing approach for brevity.

Is brevity so much more important than clarity and discoverability?




-- 
Steven D'Aprano

From drkjam at gmail.com  Thu Sep 17 19:50:22 2009
From: drkjam at gmail.com (David Moss)
Date: Thu, 17 Sep 2009 18:50:22 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909170740kf2c1d46t2b118c6835fbefce@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org>
	<4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com>
	<538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com>
	<538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com>
	<8517e9350909170740kf2c1d46t2b118c6835fbefce@mail.gmail.com>
Message-ID: <DD82F460-D62C-4EAA-8030-3890108820EC@gmail.com>

On 17 Sep 2009, at 15:40, Peter Moody <peter at hda3.com> wrote:

> On Thu, Sep 17, 2009 at 7:26 AM, DrKJam <drkjam at gmail.com> wrote:
>> Please can we have the following RFCs added to the references  
>> section that
>> cover many of the aspects covered by this PEP?
>>
>> RFC 791 - Internet Protocol
>> RFC 1918 - Address Allocation for Private Internets
>> RFC 3330 - Special-Use IPv4 Addresses
>> RFC 4291 - IPv6 Addressing Architecture
>> RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet  
>> Address
>
> Sure, can you give me an idea of what specifically you're looking to
> the rfc's to clarify? simply adding 5 rfc's to the the PEP wouldn't
> seem to enhance its readability.

I would hope that these RFCs form the technical basis upon which much  
of this library depends.

>
>> Dave M.
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>>
>>

From brett at python.org  Thu Sep 17 19:56:54 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 17 Sep 2009 10:56:54 -0700
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <h8tsb9$bkq$1@ger.gmane.org>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net> 
	<h8tsb9$bkq$1@ger.gmane.org>
Message-ID: <bbaeab100909171056l4a7c1845yd2a73bf01a423a49@mail.gmail.com>

On Thu, Sep 17, 2009 at 10:38, Georg Brandl <g.brandl at gmx.net> wrote:
> R. David Murray schrieb:
>> I floated a proposal on stdlib-sig to create a file named
>> Misc/maintainers.rst. ?The purpose of this file is to collect knowledge
>> about who knows which modules well enough to make decision about issues
>> in the tracker when the participants in the issue aren't sure, and to
>> write down the community knowledge about who has special interest and
>> expertise in specific topic areas.
>>
>> This proposal was met with approval and, after a couple of small
>> modifications of the proposal, no dissent; so I've created the skeleton
>> of the file. ?I've filled in some of the blanks where I had personal
>> knowledge, was told info on IRC, from the two referenced PEPs, and
>> from PEP 11.
>>
>> Feel free to respond with comments about the header text, but more
>> importantly let me know what you know about who (especially if it is you)
>> should be listed for each module, platform, and topic, and feel free to
>> suggest additional topics. ?My goal is to record the community knowledge.
>
> One thing I'd like to see in the list are real names of committers,
> not tracker names. ?Of course, people looking for people to assign a bug
> to should not have to search for the tracker name, so I'd like to make
> another request (that Brett already made when we switched trackers):
>
> ?Could we *please* have tracker names that match the committer names?
>
> (This doesn't even need to be done by the individual users, I would
> volunteer to rename all committer accounts and notify them by email that
> their name changed. ?This will also be coordinated with the new names
> used for Mercurial commits, if a change will be made.)
>
>> Another topic of discussion that is orthogonal to filling in the table is
>> whether or not to publish it outside the repository. ?Jesse would like to
>> see it included in the Python Documentation, and Georg has suggested the
>> possibility of creating a separate, sphinx-based, automatically-uploaded
>> document collection in the repository to contain this and related
>> information (Misc/devdocs?).
>
> For those who aren't on stdlib-sig, I'd like to elaborate a bit on that:
> There are quite a few resources for and about Python core development,
> but they aren't very accessible. ?For example, there's Misc/developers.txt
> and the upcoming maintainers.txt. ?Then there's the dev FAQ, but it's not
> maintained where developers usually look, but on the website. ?Etc.
> So the plan would be to consolidate these into another set of rst docs,
> having them in the repo, editable by every committer, as well as published
> somewhere on python.org (devdocs.python.org or somesuch).

dev.python.org would be nice to have, from which we can simply
redirect www.python.org/dev/ to dev.python.org. www.python.org/dev/
can then get cleaned up be made simpler to navigate and more obvious
for how people can get started.

-Brett

From brett at python.org  Thu Sep 17 19:57:55 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 17 Sep 2009 10:57:55 -0700
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
Message-ID: <bbaeab100909171057g4140695ey1acd0bf3bc2191d0@mail.gmail.com>

Looks great to me! Only thing missing that I can think of is sticking
Eric down as the guy who does str.format(). =)

On Thu, Sep 17, 2009 at 10:14, R. David Murray <rdmurray at bitdance.com> wrote:
> I floated a proposal on stdlib-sig to create a file named
> Misc/maintainers.rst. ?The purpose of this file is to collect knowledge
> about who knows which modules well enough to make decision about issues
> in the tracker when the participants in the issue aren't sure, and to
> write down the community knowledge about who has special interest and
> expertise in specific topic areas.
>
> This proposal was met with approval and, after a couple of small
> modifications of the proposal, no dissent; so I've created the skeleton
> of the file. ?I've filled in some of the blanks where I had personal
> knowledge, was told info on IRC, from the two referenced PEPs, and
> from PEP 11.
>
> Feel free to respond with comments about the header text, but more
> importantly let me know what you know about who (especially if it is you)
> should be listed for each module, platform, and topic, and feel free to
> suggest additional topics. ?My goal is to record the community knowledge.
>
> Another topic of discussion that is orthogonal to filling in the table is
> whether or not to publish it outside the repository. ?Jesse would like to
> see it included in the Python Documentation, and Georg has suggested the
> possibility of creating a separate, sphinx-based, automatically-uploaded
> document collection in the repository to contain this and related
> information (Misc/devdocs?).
>
> The module list was built from the py3k documentation module index, with
> the addition of 'pybench' from PEP 291. ?If there are other modules/tools
> that are missing, or submodules that should be broken out into
> separate lines, please let me know.
>
> After the initial flurry of updates and comments dies down I will check
> this in.
>
> --David
>
> ----------------------------------------------------------------------
> Maintainers Index
> =================
>
> This document cross references Python Modules (first table) and platforms
> (second table) with the Tracker user names of people who are experts
> and/or resources for that module or platform. ?This list is intended
> to be used by issue submitters, issue triage people, and other issue
> participants to find people to add to the nosy list or to contact
> directly by email for help and decisions on feature requests and bug
> fixes. ?People on this list may be asked to render final judgement on a
> feature or bug. ?If no active maintainer is listed for a given module,
> then questionable changes should go to python-dev, while any other issues
> can and should be decided by any committer.
>
> The last part of this document is a third table, listing broader topic
> areas in which various people have expertise. ?These people can also
> be contacted for help, opinions, and decisions when issues involve
> their areas.
>
> If a listed maintainer does not respond to requests for comment for an
> extended period (three weeks or more), they should be marked as inactive
> in this list by placing the word 'inactive' in parenthesis behind their
> tracker id. ?They are of course free to remove that inactive mark at
> any time.
>
> Committers should update this table as their areas of expertise widen.
> New topics may be added to the third table at will.
>
> The existence of this list is not meant to indicate that these people
> *must* be contacted for decisions; it is, rather, a resource to be used
> by non-committers to find responsible parties, and by committers who do
> not feel qualified to make a decision in a particular context.
>
> See also `PEP 291`_ and `PEP 360`_ for information about certain modules
> with special rules.
>
> .. _`PEP 291`: http://www.python.org/dev/peps/pep-0291/
> .. _`PEP 360`: http://www.python.org/dev/peps/pep-0360/
>
>
> ================== ?===========
> Module ? ? ? ? ? ? ?Maintainers
> ================== ?===========
> __future__
> __main__ ? ? ? ? ? ?gvanrossum
> _dummy_thread ? ? ? brett.cannon
> _thread
> abc
> aifc ? ? ? ? ? ? ? ?r.david.murray
> array
> ast
> asynchat ? ? ? ? ? ?josiahcarlson
> asyncore ? ? ? ? ? ?josiahcarlson
> atexit
> audioop
> base64
> bdb
> binascii
> binhex
> bisect
> builtins
> bz2
> calendar
> cgi
> cgitb
> chunk
> cmath
> cmd
> code
> codecs
> codeop
> collections
> colorsys
> compileall
> configparser
> contextlib
> copy
> copyreg
> cProfile
> crypt
> csv
> ctypes ? ? ? ? ? ? ?theller
> curses
> datetime
> dbm
> decimal
> difflib
> dis
> distutils ? ? ? ? ? tarek
> doctest
> dummy_threading ? ? brett.cannon
> email ? ? ? ? ? ? ? barry
> encodings
> errno
> exceptions
> fcntl
> filecmp
> fileinput
> fnmatch
> formatter
> fpectl
> fractions
> ftplib
> functools
> gc
> getopt
> getpass
> gettext
> glob
> grp
> gzip
> hashlib
> heapq
> hmac
> html
> http
> imaplib
> imghdr
> imp
> importlib ? ? ? ? ? brett.cannon
> inspect
> io ? ? ? ? ? ? ? ? ?pitrou, benjamin.peterson
> itertools
> json
> keyword
> lib2to3 ? ? ? ? ? ? benjamin.peterson
> linecache
> locale
> logging ? ? ? ? ? ? vsajip
> macpath
> mailbox
> mailcap
> marshal
> math
> mimetypes
> mmap
> modulefinder ? ? ? ?theller, jvr
> msilib
> msvcrt
> multiprocessing ? ? jnoller
> netrc
> nis
> nntplib
> numbers
> operator
> optparse ? ? ? ? ? ?aronacher
> os
> ossaudiodev
> parser
> pdb
> pickle
> pickletools
> pipes
> pkgutil
> platform ? ? ? ? ? ?lemburg
> plistlib
> poplib
> posix
> pprint
> pstats
> pty
> pwd
> py_compile
> pybench ? ? ? ? ? ? lemburg
> pyclbr
> pydoc
> queue
> quopri
> random
> re
> readline
> reprlib
> resource
> rlcompleter
> runpy
> sched
> select
> shelve
> shlex
> shutil
> signal
> site
> smtpd
> smtplib
> sndhdr
> socket
> socketserver
> spwd
> sqlite3
> ssl
> stat
> string
> stringprep
> struct
> subprocess ? ? ? ? ?astrand (inactive)
> sunau
> symbol
> symtable
> sys
> syslog
> tabnanny
> tarfile
> telnetlib
> tempfile
> termios
> test
> textwrap
> threading
> time
> timeit
> tkinter ? ? ? ? ? ? gpolo
> token
> tokenize
> trace
> traceback
> tty
> turtle ? ? ? ? ? ? ?gregorlingl
> types
> unicodedata
> unittest ? ? ? ? ? ?michael.foord
> urllib
> uu
> uuid
> warnings
> wave
> weakref
> webbrowser ? ? ? ? ?georg.brandl
> winreg
> winsound
> wsgiref ? ? ? ? ? ? pje
> xdrlib
> xml ? ? ? ? ? ? ? ? loewis
> xml.etree ? ? ? ? ? effbot (inactive)
> xmlrpc ? ? ? ? ? ? ?loewis
> zipfile
> zipimport
> zlib
> ================== ?===========
>
>
> ================== ?===========
> Platform ? ? ? ? ? ?Maintainer
> ------------------ ?-----------
> AIX
> Cygwin ? ? ? ? ? ? ?jlt63
> FreeBSD
> Linux
> Mac ? ? ? ? ? ? ? ? ronaldoussoren
> NetBSD1
> OS2/EMX ? ? ? ? ? ? aimacintyre
> Solaris
> HP-UX
> ================== ?===========
>
>
> ================== ?===========
> Interest Area ? ? ? Maintainers
> ------------------ ?-----------
> algorithms
> ast/compiler
> autoconf
> bsd
> buildbots
> data formats
> database
> documentation ? ? ? georg.brandl
> GUI
> i18n
> import machinery ? ?brett.cannon
> io ? ? ? ? ? ? ? ? ?pitrou, benjamin.peterson
> locale
> makefiles
> mathematics ? ? ? ? marketdickinson, eric.smith
> memory management
> networking
> packaging release management
> time and dates
> testing ? ? ? ? ? ? michael.foord
> threads
> unicode windows
> ================== ?===========
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From peter at hda3.com  Thu Sep 17 19:59:22 2009
From: peter at hda3.com (Peter Moody)
Date: Thu, 17 Sep 2009 10:59:22 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <DD82F460-D62C-4EAA-8030-3890108820EC@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org> <4AB22E4F.40509@gmail.com>
	<4AB238D9.1020204@trueblade.com>
	<538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com>
	<538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com>
	<8517e9350909170740kf2c1d46t2b118c6835fbefce@mail.gmail.com>
	<DD82F460-D62C-4EAA-8030-3890108820EC@gmail.com>
Message-ID: <8517e9350909171059h2a799786saa199e39b811b115@mail.gmail.com>

On Thu, Sep 17, 2009 at 10:50 AM, David Moss <drkjam at gmail.com> wrote:
> On 17 Sep 2009, at 15:40, Peter Moody <peter at hda3.com> wrote:
>
>> On Thu, Sep 17, 2009 at 7:26 AM, DrKJam <drkjam at gmail.com> wrote:
>>>
>>> Please can we have the following RFCs added to the references section
>>> that
>>> cover many of the aspects covered by this PEP?
>>>
>>> RFC 791 - Internet Protocol
>>> RFC 1918 - Address Allocation for Private Internets
>>> RFC 3330 - Special-Use IPv4 Addresses
>>> RFC 4291 - IPv6 Addressing Architecture
>>> RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet Address
>>
>> Sure, can you give me an idea of what specifically you're looking to
>> the rfc's to clarify? simply adding 5 rfc's to the the PEP wouldn't
>> seem to enhance its readability.
>
> I would hope that these RFCs form the technical basis upon which much of
> this library depends.

yes, and my question was is there something(s) in particular you wish
to see highlighted which will help clarify some confusion you
currently have, or do you feel that simply adding 5 rfc's to the
references section adds to the overall readability of the PEP?

see http://www.python.org/dev/peps/pep-0333/ for an example of what I mean.

>>
>>> Dave M.
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>>>
>>>
>

From brett at python.org  Thu Sep 17 19:59:10 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 17 Sep 2009 10:59:10 -0700
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <h8tsb9$bkq$1@ger.gmane.org>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net> 
	<h8tsb9$bkq$1@ger.gmane.org>
Message-ID: <bbaeab100909171059j58d336d1vc32b2591b0058895@mail.gmail.com>

On Thu, Sep 17, 2009 at 10:38, Georg Brandl <g.brandl at gmx.net> wrote:
> R. David Murray schrieb:
>> I floated a proposal on stdlib-sig to create a file named
>> Misc/maintainers.rst. ?The purpose of this file is to collect knowledge
>> about who knows which modules well enough to make decision about issues
>> in the tracker when the participants in the issue aren't sure, and to
>> write down the community knowledge about who has special interest and
>> expertise in specific topic areas.
>>
>> This proposal was met with approval and, after a couple of small
>> modifications of the proposal, no dissent; so I've created the skeleton
>> of the file. ?I've filled in some of the blanks where I had personal
>> knowledge, was told info on IRC, from the two referenced PEPs, and
>> from PEP 11.
>>
>> Feel free to respond with comments about the header text, but more
>> importantly let me know what you know about who (especially if it is you)
>> should be listed for each module, platform, and topic, and feel free to
>> suggest additional topics. ?My goal is to record the community knowledge.
>
> One thing I'd like to see in the list are real names of committers,
> not tracker names. ?Of course, people looking for people to assign a bug
> to should not have to search for the tracker name, so I'd like to make
> another request (that Brett already made when we switched trackers):
>
> ?Could we *please* have tracker names that match the committer names?
>
> (This doesn't even need to be done by the individual users, I would
> volunteer to rename all committer accounts and notify them by email that
> their name changed. ?This will also be coordinated with the new names
> used for Mercurial commits, if a change will be made.)

+1 from me. It would be nice to not have to look up what tracker name
someone uses since I know everyone's committer name.

-Brett

From rdmurray at bitdance.com  Thu Sep 17 20:04:11 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 14:04:11 -0400 (EDT)
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au> 
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> 
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au> 
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org> 
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> 
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> 
	<Pine.LNX.4.64.0909171329450.17020@kimball.webabinitio.net>
	<8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909171346550.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 10:38, Peter Moody wrote:
> On Thu, Sep 17, 2009 at 10:32 AM, R. David Murray <rdmurray at bitdance.com> wrote:
>> On Thu, 17 Sep 2009 at 09:16, Peter Moody wrote:
>>>
>>> I mentioned before that IPy's insistence on receiving masked out
>>> networks was one of the main reasons I wrote ipaddr to begin with.
>>> Having ipaddr mimic this behavior would make it significantly less
>>> useful. Removing functionality in the name of avoiding confusion
>>> doesn't make sense when the same confusion can be alleviated w/o the
>>> loss.
>>
>> We're suggesting moving that functionality (accepting a non-masked IP
>> plus netmask and returning the corresponding Network object) into
>> a different constructor, not eliminating the functionality.
>
> Do you mean a new constructor function, ala IPAddress and IPNetwork
> (so the current classes remain largely the same and the various
> constructors enforce certain restrictions) or something else?
>
> At this point, I'm getting confused with the intermingling of class
> and constructor names.

I mean, eg, IPv4Network.fromHostIP('192.168.1.1/24').

Or some such name.

Having another constructor that took an IPv4Address object and a mask
would probably be a good idea as well.  These could be exposed as
appropriately named module level functions instead/as well, of course.

I would have IPv4Address itself be strict, and thus the new constructors
would compute the network address and call the regular IPv4Address
constructor.(*)

--David

(*) If that's the part you object to (not recording the input IP
in the Network object), then clearly you see a need to have an
AddressWithMask object of _some_ sort....

From rdmurray at bitdance.com  Thu Sep 17 20:08:12 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 14:08:12 -0400 (EDT)
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <bbaeab100909171059j58d336d1vc32b2591b0058895@mail.gmail.com>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net> 
	<h8tsb9$bkq$1@ger.gmane.org>
	<bbaeab100909171059j58d336d1vc32b2591b0058895@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909171404490.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 10:59, Brett Cannon wrote:
> On Thu, Sep 17, 2009 at 10:38, Georg Brandl <g.brandl at gmx.net> wrote:
>> ??Could we *please* have tracker names that match the committer names?
>>
>> (This doesn't even need to be done by the individual users, I would
>> volunteer to rename all committer accounts and notify them by email that
>> their name changed. ??This will also be coordinated with the new names
>> used for Mercurial commits, if a change will be made.)
>
> +1 from me. It would be nice to not have to look up what tracker name
> someone uses since I know everyone's committer name.

+1 from me as well.  And speaking of names, people should feel free to
email me maintainer info for the table as real names.  Whether or
not Georg's proposal goes through I'll make the table reflect the
correct tracker ids.

--David

From fetchinson at googlemail.com  Thu Sep 17 20:10:51 2009
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Thu, 17 Sep 2009 11:10:51 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151343q4eb419d1v35ad5468bc2798a8@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
	<8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>
	<77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>
	<8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
	<fbe2e2100909151334k62686719veac45c75c644858e@mail.gmail.com>
	<8517e9350909151343q4eb419d1v35ad5468bc2798a8@mail.gmail.com>
Message-ID: <fbe2e2100909171110s523a3707w9d7d2196614326e3@mail.gmail.com>

>>>>> 188 (check that, 190) people have downloaded the 2.0 release in the
>>>>> last week (numbers publicly available from the code.google.com). I
>>>>> can't tell you how many (if any) have downloaded it via svn.
>>>>
>>>> Downloading and using are not the same thing.
>>>
>>> Correct, but there is a strong positive correlation between the two.
>>> If you have a better method for determining what you would consider an
>>> appropriate level of usage, I'm all ears.
>>
>> A good way of determining the level of usage would be pointing to open
>> source projects that are popular in the python community and which
>> incorporate your module.
>
> well, the 2.0 release is still new. codesearch.google.com shows some
> projects using the 1.x release; hopefully some of those 200
> downloaders will put up some publicly indexable python code at some
> point.

I think one first needs to wait until this happens, I meana large user
base is formed, before a meaningful discussion can be done on whether
to include it in the stdlib or not. The long and largely academic
thread here I think illustrates this point. Without a large user base
it's up to anybody's gut feelings what is 'right' and what 'feels
wrong'.

Cheers,
Daniel


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown

From steve at pearwood.info  Thu Sep 17 20:26:41 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 18 Sep 2009 04:26:41 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090917071516.217655AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20090917071516.217655AC0D6@longblack.object-craft.com.au>
Message-ID: <200909180426.41808.steve@pearwood.info>

On Thu, 17 Sep 2009 05:15:16 pm Andrew McNamara wrote:

> Conceptually, you sometimes need a bare address, and other times,
> you need an address with an associated network (host interface
> configs, router configs, etc). By AddressWithMask, I really mean 
> AddressWithEnoughInformationToDeriveNetworkWhenNeeded. Conveniently,
> IPv4 and IPv6 addressing allows us to derive the network from the
> host address combined with the netmask - in other words, we don't
> have to attach a real Network object to Address objects until the
> user tries to access it, and then we derive it from the address and
> mask.

To clarify: when you say "derive the network" are you talking about the 
network (which is a container) or the network address = host_address & 
netmask (which is not a container)? I think you're referring to the 
later.

If there's need for address+netmask, does it need to be a separate 
class? Perhaps Address objects could simply have a netmask property, 
defaulting to None. If you need an "address with mask" object, you 
create an Address and set the mask:

addr = Address(...)
addr.netmask = "255.255.255.0"

The caller is responsible for ensuring the netmask has been set before 
trying to use it:

network_address = addr & addr.netmask



> If you have a .network attribute on an address object, checking if an
> address is in the same network as another address becomes:
>
>     addr_a in addr_b.network
>
> As the module stands, you write that as:
>
>     addr_a in addr_b
>
> I don't think the intent is as clear with the later.

I would find the later completely unclear and disturbing -- how can one 
address contain another address?



-- 
Steven D'Aprano

From amk at amk.ca  Thu Sep 17 20:29:44 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 17 Sep 2009 14:29:44 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909171059h2a799786saa199e39b811b115@mail.gmail.com>
References: <4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org>
	<4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com>
	<538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com>
	<538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com>
	<8517e9350909170740kf2c1d46t2b118c6835fbefce@mail.gmail.com>
	<DD82F460-D62C-4EAA-8030-3890108820EC@gmail.com>
	<8517e9350909171059h2a799786saa199e39b811b115@mail.gmail.com>
Message-ID: <20090917182944.GA13389@amk-desktop.matrixgroup.net>

On Thu, Sep 17, 2009 at 10:59:22AM -0700, Peter Moody wrote:
> currently have, or do you feel that simply adding 5 rfc's to the
> references section adds to the overall readability of the PEP?

I would list them simply because it's not obvious which RFC specifies
the format of IP addresses or how they're allocated.  e.g. if I search
for "rfc ip address format", most of the hits are for RFC 3587,
"IPv6 Global Unicast Address Format", which doesn't actually explain
IPv6 address syntax.

--amk

From rdmurray at bitdance.com  Thu Sep 17 20:35:16 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 14:35:16 -0400 (EDT)
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <bbaeab100909171057g4140695ey1acd0bf3bc2191d0@mail.gmail.com>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
	<bbaeab100909171057g4140695ey1acd0bf3bc2191d0@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909171432310.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 10:57, Brett Cannon wrote:
> Looks great to me! Only thing missing that I can think of is sticking
> Eric down as the guy who does str.format(). =)

OK, I've added that one to the last table ;)

--David

From rdmurray at bitdance.com  Thu Sep 17 20:40:38 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 14:40:38 -0400 (EDT)
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <Pine.LNX.4.64.0909171404490.17020@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net> 
	<h8tsb9$bkq$1@ger.gmane.org>
	<bbaeab100909171059j58d336d1vc32b2591b0058895@mail.gmail.com>
	<Pine.LNX.4.64.0909171404490.17020@kimball.webabinitio.net>
Message-ID: <Pine.LNX.4.64.0909171435260.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 14:08, R. David Murray wrote:
> On Thu, 17 Sep 2009 at 10:59, Brett Cannon wrote:
>>  On Thu, Sep 17, 2009 at 10:38, Georg Brandl <g.brandl at gmx.net> wrote:
>> >  ??Could we *please* have tracker names that match the committer names?
>> > 
>> >  (This doesn't even need to be done by the individual users, I would
>> >  volunteer to rename all committer accounts and notify them by email that
>> >  their name changed. ??This will also be coordinated with the new names
>> >  used for Mercurial commits, if a change will be made.)
>>
>>  +1 from me. It would be nice to not have to look up what tracker name
>>  someone uses since I know everyone's committer name.
>
> +1 from me as well.  And speaking of names, people should feel free to
> email me maintainer info for the table as real names.  Whether or
> not Georg's proposal goes through I'll make the table reflect the
> correct tracker ids.

By the way, in case anyone isn't aware of it (Mark Dickinson and I weren't
initially), it is _really easy_ to change your tracker id if you want to
do it yourself.  Just go in 'your details' and change your login name.
It'll change everywhere in the tracker, since roundup tracks accounts
by an internal number, not the login name.

--David

From eric at trueblade.com  Thu Sep 17 20:43:13 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 17 Sep 2009 14:43:13 -0400
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <Pine.LNX.4.64.0909171432310.17020@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>	<bbaeab100909171057g4140695ey1acd0bf3bc2191d0@mail.gmail.com>
	<Pine.LNX.4.64.0909171432310.17020@kimball.webabinitio.net>
Message-ID: <4AB28341.5090908@trueblade.com>

R. David Murray wrote:
> On Thu, 17 Sep 2009 at 10:57, Brett Cannon wrote:
>> Looks great to me! Only thing missing that I can think of is sticking
>> Eric down as the guy who does str.format(). =)
> 
> OK, I've added that one to the last table ;)

Awesome! I get to spend even more time on formatting strings!

I kid. I'm happy to be the string formatting maintainer.



From amk at amk.ca  Thu Sep 17 20:44:10 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 17 Sep 2009 14:44:10 -0400
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <h8tsb9$bkq$1@ger.gmane.org>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
	<h8tsb9$bkq$1@ger.gmane.org>
Message-ID: <20090917184410.GB13389@amk-desktop.matrixgroup.net>

On Thu, Sep 17, 2009 at 07:38:50PM +0200, Georg Brandl wrote:
> So the plan would be to consolidate these into another set of rst docs,
> having them in the repo, editable by every committer, as well as published
> somewhere on python.org (devdocs.python.org or somesuch).

+1.

Should we do something similar with the FAQs at
http://www.python.org/doc/faq/ ?

--amk

From steve at pearwood.info  Thu Sep 17 20:47:35 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 18 Sep 2009 04:47:35 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090917004138.2177C5AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>
	<20090917004138.2177C5AC0D6@longblack.object-craft.com.au>
Message-ID: <200909180447.36475.steve@pearwood.info>

On Thu, 17 Sep 2009 10:41:37 am Andrew McNamara wrote:
> In the olden days, the mask was spelled out in octets (eg
> 255.255.255.0). But we've moved to a more compact and logical
> notation where the number of leading significant bits is specified
> (eg /24).

I hope you're not suggesting the older notation be unsupported? I would 
expect to be able to use a mask like 255.255.255.192 without having to 
count bits myself.


-- 
Steven D'Aprano

From p.f.moore at gmail.com  Thu Sep 17 21:11:51 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 17 Sep 2009 20:11:51 +0100
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
Message-ID: <79990c6b0909171211q697442f3n1948b946ffe31708@mail.gmail.com>

2009/9/17 R. David Murray <rdmurray at bitdance.com>:
> Feel free to respond with comments about the header text, but more
> importantly let me know what you know about who (especially if it is you)
> should be listed for each module, platform, and topic, and feel free to
> suggest additional topics. ?My goal is to record the community knowledge.

A couple more:

runpy - Nick Coghlan
itertools, collections - Raymond Hettinger

(obviously, if I've misrepresented their interest, they should speak up!)

Paul.

From fdrake at acm.org  Thu Sep 17 20:19:03 2009
From: fdrake at acm.org (Fred Drake)
Date: Thu, 17 Sep 2009 14:19:03 -0400
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <bbaeab100909171056l4a7c1845yd2a73bf01a423a49@mail.gmail.com>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
	<h8tsb9$bkq$1@ger.gmane.org>
	<bbaeab100909171056l4a7c1845yd2a73bf01a423a49@mail.gmail.com>
Message-ID: <E02AC97B-3ADB-4073-B173-A4C5ACC27A29@acm.org>

On Thu, Sep 17, 2009 at 10:38, Georg Brandl <g.brandl at gmx.net> wrote:
> So the plan would be to consolidate these into another set of rst  
> docs,
> having them in the repo, editable by every committer, as well as  
> published
> somewhere on python.org (devdocs.python.org or somesuch).

On Sep 17, 2009, at 1:56 PM, Brett Cannon wrote:
> dev.python.org would be nice to have, from which we can simply
> redirect www.python.org/dev/ to dev.python.org. www.python.org/dev/
> can then get cleaned up be made simpler to navigate and more obvious
> for how people can get started.

Many years ago, we decided to add docs.python.org with the "current"  
version of the documentation, so people would be able to find the docs  
more easily.  Since then, we've had problems with keeping  
docs.python.org and www.python.org/doc/ in sync, and with different  
styles being applied to the sites.

One of the reasons www.python.org/doc/ was considered less  
discoverable was the about of only-sometimes-interesting information  
there; docs.python.org contains only "current" docs (for some vague  
notion of current and only, given that dev builds and both Python 2  
and Python 3 versions).  Some claimed that having "docs" at the front  
of the URL helped, though I don't recall why.

I don't know whether users consider the docs more discoverable than  
they used to; if anyone can provide information about that (now sure  
what indicator would make even sense), that might be informative.

The overload problem is one we *don't* have for developer  
documentation; the difficulty is in discovering what exists at all.

+1 on moving the developer docs to subversion

+0 on exposing them online

-0 on adding another domain name


   -Fred

-- 
Fred Drake   <fdrake at pobox.com>


From mcguire at google.com  Thu Sep 17 20:58:29 2009
From: mcguire at google.com (Jake McGuire)
Date: Thu, 17 Sep 2009 11:58:29 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <fbe2e2100909171110s523a3707w9d7d2196614326e3@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
	<8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>
	<77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>
	<8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
	<fbe2e2100909151334k62686719veac45c75c644858e@mail.gmail.com>
	<8517e9350909151343q4eb419d1v35ad5468bc2798a8@mail.gmail.com>
	<fbe2e2100909171110s523a3707w9d7d2196614326e3@mail.gmail.com>
Message-ID: <77c780b40909171158u6117e47djf18508acc6e1bc18@mail.gmail.com>

On Thursday, September 17, 2009, Daniel Fetchinson
<fetchinson at googlemail.com> wrote:
>>>>>> 188 (check that, 190) people have downloaded the 2.0 release in the
>>>>>> last week (numbers publicly available from the http://code.google.com). I
>>>>>> can't tell you how many (if any) have downloaded it via svn.
>>>>>
>>>>> Downloading and using are not the same thing.
>>>>
>>>> Correct, but there is a strong positive correlation between the two.
>>>> If you have a better method for determining what you would consider an
>>>> appropriate level of usage, I'm all ears.
>>>
>>> A good way of determining the level of usage would be pointing to open
>>> source projects that are popular in the python community and which
>>> incorporate your module.
>>
>> well, the 2.0 release is still new. http://codesearch.google.com shows some
>> projects using the 1.x release; hopefully some of those 200
>> downloaders will put up some publicly indexable python code at some
>> point.
>
> I think one first needs to wait until this happens, I meana large user
> base is formed, before a meaningful discussion can be done on whether
> to include it in the stdlib or not. The long and largely academic
> thread here I think illustrates this point. Without a large user base
> it's up to anybody's gut feelings what is 'right' and what 'feels
> wrong'.

+1000

-jake

From rdmurray at bitdance.com  Thu Sep 17 21:15:41 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 15:15:41 -0400 (EDT)
Subject: [Python-Dev] maintainers.rst committed
Message-ID: <Pine.LNX.4.64.0909171513490.17020@kimball.webabinitio.net>

I decided to commit the draft of maintainers.rst in case people would
rather update it themselves.  I'm happy to continue collecting updates
and applying them as well.

--David

From jnoller at gmail.com  Thu Sep 17 21:40:23 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 17 Sep 2009 15:40:23 -0400
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <h8tsb9$bkq$1@ger.gmane.org>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
	<h8tsb9$bkq$1@ger.gmane.org>
Message-ID: <4222a8490909171240q4d7e1c81xb134c510af8acc54@mail.gmail.com>

On Thu, Sep 17, 2009 at 1:38 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> R. David Murray schrieb:
>> I floated a proposal on stdlib-sig to create a file named
>> Misc/maintainers.rst. ?The purpose of this file is to collect knowledge
>> about who knows which modules well enough to make decision about issues
>> in the tracker when the participants in the issue aren't sure, and to
>> write down the community knowledge about who has special interest and
>> expertise in specific topic areas.
>>
>> This proposal was met with approval and, after a couple of small
>> modifications of the proposal, no dissent; so I've created the skeleton
>> of the file. ?I've filled in some of the blanks where I had personal
>> knowledge, was told info on IRC, from the two referenced PEPs, and
>> from PEP 11.
>>
>> Feel free to respond with comments about the header text, but more
>> importantly let me know what you know about who (especially if it is you)
>> should be listed for each module, platform, and topic, and feel free to
>> suggest additional topics. ?My goal is to record the community knowledge.
>
> One thing I'd like to see in the list are real names of committers,
> not tracker names. ?Of course, people looking for people to assign a bug
> to should not have to search for the tracker name, so I'd like to make
> another request (that Brett already made when we switched trackers):
>
> ?Could we *please* have tracker names that match the committer names?
>
> (This doesn't even need to be done by the individual users, I would
> volunteer to rename all committer accounts and notify them by email that
> their name changed. ?This will also be coordinated with the new names
> used for Mercurial commits, if a change will be made.)
>
>> Another topic of discussion that is orthogonal to filling in the table is
>> whether or not to publish it outside the repository. ?Jesse would like to
>> see it included in the Python Documentation, and Georg has suggested the
>> possibility of creating a separate, sphinx-based, automatically-uploaded
>> document collection in the repository to contain this and related
>> information (Misc/devdocs?).
>
> For those who aren't on stdlib-sig, I'd like to elaborate a bit on that:
> There are quite a few resources for and about Python core development,
> but they aren't very accessible. ?For example, there's Misc/developers.txt
> and the upcoming maintainers.txt. ?Then there's the dev FAQ, but it's not
> maintained where developers usually look, but on the website. ?Etc.
> So the plan would be to consolidate these into another set of rst docs,
> having them in the repo, editable by every committer, as well as published
> somewhere on python.org (devdocs.python.org or somesuch).
>
> cheers,
> Georg
>

+1 to getting them in The One True Format (ReST) and up and online,
and then porting all the dev docs into The One True Format.

From turnbull at sk.tsukuba.ac.jp  Thu Sep 17 21:56:43 2009
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Fri, 18 Sep 2009 04:56:43 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090917071516.217655AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<20090917032126.66F896006DE@longblack.object-craft.com.au>
	<8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>
	<20090917041745.C331B6006DE@longblack.object-craft.com.au>
	<87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20090917071516.217655AC0D6@longblack.object-craft.com.au>
Message-ID: <878wgd1h1g.fsf@uwakimon.sk.tsukuba.ac.jp>

Andrew McNamara writes:

 > Conceptually, you sometimes need a bare address, and other times,
 > you need an address with an associated network (host interface
 > configs, router configs, etc). By AddressWithMask, I really mean
 > AddressWithEnoughInformationToDeriveNetworkWhenNeeded. Conveniently,
 > IPv4 and IPv6 addressing allows us to derive the network from the
 > host

That's precisely the claim I deny.  Yes, you can derive the network
*address* from a generic address and a mask.  You cannot derive the
*network* that way!  Eg, suppose in a routing application you have

class IPv4NetworkWithGateway(IPv4Network):

... oops.  You want *the* IPv4Network instance that has all the
gateway info, not just *any* IPv4Network instance that happens to have
the same network address and mask.  Address + mask is *not* sufficient
to derive the desired IPv4Network instance.

I grant that sometimes all you really care about is that the network
address and the mask match; constructing on the fly would be
sufficient.  However, many of the applications I can think of attach
additional attributes to IPv4Networks, and really do want "the real
thing" vs. a constructed imitation.

From python at rcn.com  Thu Sep 17 22:13:51 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 17 Sep 2009 13:13:51 -0700
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <bbaeab100909171059j58d336d1vc32b2591b0058895@mail.gmail.com>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
	<h8tsb9$bkq$1@ger.gmane.org>
	<bbaeab100909171059j58d336d1vc32b2591b0058895@mail.gmail.com>
Message-ID: <91ECB2DF-7C76-4896-B547-D73D5E71A0B5@rcn.com>


On Sep 17, 2009, at 10:59 AM, Brett Cannon wrote:

> On Thu, Sep 17, 2009 at 10:38, Georg Brandl <g.brandl at gmx.net> wrote:
>> R. David Murray schrieb:
>>> I floated a proposal on stdlib-sig to create a file named
>>> Misc/maintainers.rst.  The purpose of this file is to collect  
>>> knowledge
>>> about who knows which modules well enough to make decision about  
>>> issues
>>> in the tracker when the participants in the issue aren't sure, and  
>>> to
>>> write down the community knowledge about who has special interest  
>>> and
>>> expertise in specific topic areas.
>>>
>>> This proposal was met with approval and, after a couple of small
>>> modifications of the proposal, no dissent; so I've created the  
>>> skeleton
>>> of the file.  I've filled in some of the blanks where I had personal
>>> knowledge, was told info on IRC, from the two referenced PEPs, and
>>> from PEP 11.
>>>
>>> Feel free to respond with comments about the header text, but more
>>> importantly let me know what you know about who (especially if it  
>>> is you)
>>> should be listed for each module, platform, and topic, and feel  
>>> free to
>>> suggest additional topics.  My goal is to record the community  
>>> knowledge.
>>
>> One thing I'd like to see in the list are real names of committers,
>> not tracker names.  Of course, people looking for people to assign  
>> a bug
>> to should not have to search for the tracker name, so I'd like to  
>> make
>> another request (that Brett already made when we switched trackers):
>>
>>  Could we *please* have tracker names that match the committer names?
>>
>> (This doesn't even need to be done by the individual users, I would
>> volunteer to rename all committer accounts and notify them by email  
>> that
>> their name changed.  This will also be coordinated with the new names
>> used for Mercurial commits, if a change will be made.)
>
> +1 from me. It would be nice to not have to look up what tracker name
> someone uses since I know everyone's committer name.
>

+1 from me also.  This has long been a source of irritation.


Raymond

From g.brandl at gmx.net  Thu Sep 17 23:32:25 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 17 Sep 2009 23:32:25 +0200
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <20090917184410.GB13389@amk-desktop.matrixgroup.net>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>	<h8tsb9$bkq$1@ger.gmane.org>
	<20090917184410.GB13389@amk-desktop.matrixgroup.net>
Message-ID: <h8ua18$o6s$1@ger.gmane.org>

A.M. Kuchling schrieb:
> On Thu, Sep 17, 2009 at 07:38:50PM +0200, Georg Brandl wrote:
>> So the plan would be to consolidate these into another set of rst docs,
>> having them in the repo, editable by every committer, as well as published
>> somewhere on python.org (devdocs.python.org or somesuch).
> 
> +1.
> 
> Should we do something similar with the FAQs at
> http://www.python.org/doc/faq/ ?

They would then go into the main docs, I guess?

OK, so there seems to be some consensus on both the committer name and the
dev docs proposal.  I'll start working on both by the end of next week,
when I have more contiguous time to spare.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From ncoghlan at gmail.com  Thu Sep 17 23:45:14 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Sep 2009 07:45:14 +1000
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <Pine.LNX.4.64.0909171346550.17020@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
	<Pine.LNX.4.64.0909171329450.17020@kimball.webabinitio.net>	<8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com>
	<Pine.LNX.4.64.0909171346550.17020@kimball.webabinitio.net>
Message-ID: <4AB2ADEA.40601@gmail.com>

R. David Murray wrote:
> I would have IPv4Address itself be strict, and thus the new constructors
> would compute the network address and call the regular IPv4Address
> constructor.(*)

s/Address/Network/ in this paragraph :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Sep 17 23:48:09 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Sep 2009 07:48:09 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB24DB9.9050605@trueblade.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>	
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>	
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>	
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>	
	<4AB1A39B.7090003@canterbury.ac.nz>	
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>	
	<loom.20090917T134809-168@post.gmane.org>
	<4AB22E4F.40509@gmail.com>	 <4AB238D9.1020204@trueblade.com>
	<8517e9350909170748g7ec8b1a0u926e26de6907b494@mail.gmail.com>
	<4AB24DB9.9050605@trueblade.com>
Message-ID: <4AB2AE99.8070006@gmail.com>

Eric Smith wrote:
> Peter Moody wrote:
> 
>> indexing is plenty efficient, but the problem is that these names for
>> these attributes are common to the point of causing confusion if
>> they're omitted.
> 
> After thinking about it some more, I'm okay with names for [-1] and [0].
> I like .broadcast, and I can live with .network (although it's
> confusing, since in my view it should return an address, not a network).

And in fact *does* return an Address - the same address that my_net[0]
returns.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Sep 17 23:58:56 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Sep 2009 07:58:56 +1000
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <79990c6b0909171211q697442f3n1948b946ffe31708@mail.gmail.com>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
	<79990c6b0909171211q697442f3n1948b946ffe31708@mail.gmail.com>
Message-ID: <4AB2B120.5060202@gmail.com>

Paul Moore wrote:
> runpy - Nick Coghlan

And for the general interest table, the import system and the AST
compiler are the two main areas that I know better than most.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From greg.ewing at canterbury.ac.nz  Fri Sep 18 00:27:32 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 18 Sep 2009 10:27:32 +1200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
Message-ID: <4AB2B7D4.8090107@canterbury.ac.nz>

Peter Moody wrote:

> the address with all of the hosts bits masked to zero is most commonly
> referred to as the network address.

Then call the attribute 'network_address', not just 'network'.

-- 
Greg

From janssen at parc.com  Fri Sep 18 00:33:30 2009
From: janssen at parc.com (Bill Janssen)
Date: Thu, 17 Sep 2009 15:33:30 PDT
Subject: [Python-Dev] how to debug httplib slowness
In-Reply-To: <20090904153241.GB7342@phd.pp.ru>
References: <4A82A20A.1070704@simplistix.co.uk>
	<ca471dc20908120807t4fe500efjef2a135a36cbbb3d@mail.gmail.com>
	<4A82E120.4030105@simplistix.co.uk>
	<ca471dc20908120910y7e850fbew126a7e09afd83c0f@mail.gmail.com>
	<4AA0F5F2.6030903@simplistix.co.uk>
	<fb73205e0909040428w3dcb22e7g141b12dd4a36d9b5@mail.gmail.com>
	<4AA12C0F.1090305@simplistix.co.uk>
	<20090904153241.GB7342@phd.pp.ru>
Message-ID: <3746.1253226810@parc.com>

Oleg Broytmann <phd at phd.pp.ru> wrote:

> On Fri, Sep 04, 2009 at 04:02:39PM +0100, Chris Withers wrote:
> > So, httplib does this:
> >
> > GET /<blah> HTTP/1.1
> [skip]
> > While wget does this:
> >
> > GET /<blah> HTTP/1.0
> [skip]
> > - Apache responds with a chunked response only to httplib. Why is that?
> 
>    Probably because wget uses HTTP/1.0?

Yep.  Not allowed to respond chunked to 1.0 requests.

Bill

From rdmurray at bitdance.com  Fri Sep 18 00:41:16 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 18:41:16 -0400 (EDT)
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <4AB2ADEA.40601@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au> 
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> 
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au> 
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org> 
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> 
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> 
	<Pine.LNX.4.64.0909171329450.17020@kimball.webabinitio.net>
	<8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com>
	<Pine.LNX.4.64.0909171346550.17020@kimball.webabinitio.net>
	<4AB2ADEA.40601@gmail.com>
Message-ID: <Pine.LNX.4.64.0909171840450.17020@kimball.webabinitio.net>

On Fri, 18 Sep 2009 at 07:45, Nick Coghlan wrote:
> R. David Murray wrote:
>> I would have IPv4Address itself be strict, and thus the new constructors
>> would compute the network address and call the regular IPv4Address
>> constructor.(*)
>
> s/Address/Network/ in this paragraph :)

Ah, yes, sorry for the slip, and thanks for catching it.

--David

From peter at hda3.com  Fri Sep 18 00:50:16 2009
From: peter at hda3.com (Peter Moody)
Date: Thu, 17 Sep 2009 15:50:16 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB2B7D4.8090107@canterbury.ac.nz>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<4AB2B7D4.8090107@canterbury.ac.nz>
Message-ID: <8517e9350909171550y4033bbc4l6beaae10104cf7d7@mail.gmail.com>

On Thu, Sep 17, 2009 at 3:27 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Peter Moody wrote:
>
>> the address with all of the hosts bits masked to zero is most commonly
>> referred to as the network address.
>
> Then call the attribute 'network_address', not just 'network'.

from an earlier email:

Again, the same error-catching functionality can be obtained through
an option to the constructor. network and broadcast attributes can be
renamed to .\1_address to alleviate confusion as well.

> --
> Greg
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From amk at amk.ca  Fri Sep 18 01:26:52 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 17 Sep 2009 19:26:52 -0400
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <h8ua18$o6s$1@ger.gmane.org>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
	<h8tsb9$bkq$1@ger.gmane.org>
	<20090917184410.GB13389@amk-desktop.matrixgroup.net>
	<h8ua18$o6s$1@ger.gmane.org>
Message-ID: <20090917232652.GA1730@andrew-kuchlings-macbook.local>

On Thu, Sep 17, 2009 at 11:32:25PM +0200, Georg Brandl wrote:
> > Should we do something similar with the FAQs at
> > http://www.python.org/doc/faq/ ?
> 
> They would then go into the main docs, I guess?

That would make sense.  However, we already have seven different FAQs
there, so I don't think they should all be added as peers of the
Tutorial, Library Reference, etc.  Maybe there should be a single
'FAQ' document, with the existing faqs as subsections.

Another option might be to put them on an entirely new site such as
faq.python.org, and then configure www.python.org/doc/faq/ to proxy
for it.

Should I look at implementing the python.org as an output style for
Sphinx?

--amk



From rdmurray at bitdance.com  Fri Sep 18 01:27:23 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 19:27:23 -0400 (EDT)
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <E02AC97B-3ADB-4073-B173-A4C5ACC27A29@acm.org>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
	<h8tsb9$bkq$1@ger.gmane.org>
	<bbaeab100909171056l4a7c1845yd2a73bf01a423a49@mail.gmail.com>
	<E02AC97B-3ADB-4073-B173-A4C5ACC27A29@acm.org>
Message-ID: <Pine.LNX.4.64.0909171924520.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 14:19, Fred Drake wrote:
> One of the reasons www.python.org/doc/ was considered less discoverable was 
> the about of only-sometimes-interesting information there; docs.python.org 
> contains only "current" docs (for some vague notion of current and only, 
> given that dev builds and both Python 2 and Python 3 versions).  Some claimed 
> that having "docs" at the front of the URL helped, though I don't recall why.

It's anecdotal, but I can tell you that currently I often type
'http://dev.python.org', get a not found, smack my head, and retype
the url.  I don't have that problem with docs, obviously :)

--David

From srittau at jroger.in-berlin.de  Fri Sep 18 02:24:23 2009
From: srittau at jroger.in-berlin.de (Sebastian Rittau)
Date: Fri, 18 Sep 2009 02:24:23 +0200
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <Pine.LNX.4.64.0909171346550.17020@kimball.webabinitio.net>
References: <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
	<Pine.LNX.4.64.0909171329450.17020@kimball.webabinitio.net>
	<8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com>
	<Pine.LNX.4.64.0909171346550.17020@kimball.webabinitio.net>
Message-ID: <20090918002423.GB7534@jroger.in-berlin.de>

On Thu, Sep 17, 2009 at 02:04:11PM -0400, R. David Murray wrote:

> I mean, eg, IPv4Network.fromHostIP('192.168.1.1/24').

I'd actually suggest to use

  >>> net, host = parse_network_and_host("192.168.111.33/24")
  (IPv4Network('192.168.111.0/24'), IPv4Address('192.168.111.33'))
  >>> 

I think this helps the use case of the short network+gateway notation,
while keeping the concepts of network and host address cleanly separate.

> I would have IPv4Address itself be strict, and thus the new constructors
> would compute the network address and call the regular IPv4Address
> constructor.(*)

Then the regular IPv4Network constructor could be strict and can be used
for validation.

 - Sebastian

From andrewm at object-craft.com.au  Fri Sep 18 02:49:49 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Fri, 18 Sep 2009 10:49:49 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909180447.36475.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>
	<20090917004138.2177C5AC0D6@longblack.object-craft.com.au>
	<200909180447.36475.steve@pearwood.info>
Message-ID: <20090918004949.8DE1D5AC0D6@longblack.object-craft.com.au>

>On Thu, 17 Sep 2009 10:41:37 am Andrew McNamara wrote:
>> In the olden days, the mask was spelled out in octets (eg
>> 255.255.255.0). But we've moved to a more compact and logical
>> notation where the number of leading significant bits is specified
>> (eg /24).
>
>I hope you're not suggesting the older notation be unsupported? I would 
>expect to be able to use a mask like 255.255.255.192 without having to 
>count bits myself.

No, of course not - I was just explaining the relationship between the
two notations for people who may not have been aware.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From andrewm at object-craft.com.au  Fri Sep 18 02:57:41 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Fri, 18 Sep 2009 10:57:41 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909180426.41808.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20090917071516.217655AC0D6@longblack.object-craft.com.au>
	<200909180426.41808.steve@pearwood.info>
Message-ID: <20090918005742.0B2A95AC0D6@longblack.object-craft.com.au>

>> Conceptually, you sometimes need a bare address, and other times,
>> you need an address with an associated network (host interface
>> configs, router configs, etc). By AddressWithMask, I really mean 
>> AddressWithEnoughInformationToDeriveNetworkWhenNeeded. Conveniently,
>> IPv4 and IPv6 addressing allows us to derive the network from the
>> host address combined with the netmask - in other words, we don't
>> have to attach a real Network object to Address objects until the
>> user tries to access it, and then we derive it from the address and
>> mask.
>
>To clarify: when you say "derive the network" are you talking about the 
>network (which is a container) or the network address = host_address & 
>netmask (which is not a container)? I think you're referring to the 
>later.

I mean a Network object which is a container (which, by definition,
has a network address + mask).

>If there's need for address+netmask, does it need to be a separate 
>class? Perhaps Address objects could simply have a netmask property, 
>defaulting to None. If you need an "address with mask" object, you 
>create an Address and set the mask:
>
>addr = Address(...)
>addr.netmask = "255.255.255.0"

Greg Ewing suggested this yesterday - I'm neutral on whether it's done this
way or as a separate class. The implementation may be somewhat cleaner if
it's a separate class, however.

>> If you have a .network attribute on an address object, checking if an
>> address is in the same network as another address becomes:
>>
>>     addr_a in addr_b.network
>>
>> As the module stands, you write that as:
>>
>>     addr_a in addr_b
>>
>> I don't think the intent is as clear with the later.
>
>I would find the later completely unclear and disturbing -- how can one 
>address contain another address?

Yes - that's how it works now, and I can only see it resulting in
confusion and bugs for no advantage.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From andrewm at object-craft.com.au  Fri Sep 18 03:04:46 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Fri, 18 Sep 2009 11:04:46 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909180344.31794.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org>
	<4AB22E4F.40509@gmail.com> <200909180344.31794.steve@pearwood.info>
Message-ID: <20090918010446.50EB15AC0D6@longblack.object-craft.com.au>

>To a non-specialist, "the network address" is ambiguous. There are many 
>addresses in a network, and none of them are the entire network. It's 
>like saying, given a list [2, 4, 8, 12], what's "the list item"?

A "network address" is an IP address and mask, but I understand your
confusion - we're mixing terminology from disperate domains. In my
postings, I have tried to refer to Network (a containter) and Address
(an item).

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From jnoller at gmail.com  Fri Sep 18 03:13:17 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 17 Sep 2009 21:13:17 -0400
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <20090917232652.GA1730@andrew-kuchlings-macbook.local>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
	<h8tsb9$bkq$1@ger.gmane.org>
	<20090917184410.GB13389@amk-desktop.matrixgroup.net>
	<h8ua18$o6s$1@ger.gmane.org>
	<20090917232652.GA1730@andrew-kuchlings-macbook.local>
Message-ID: <4222a8490909171813i37ed0ee0i72e42223f3610130@mail.gmail.com>

On Thu, Sep 17, 2009 at 7:26 PM, A.M. Kuchling <amk at amk.ca> wrote:
> On Thu, Sep 17, 2009 at 11:32:25PM +0200, Georg Brandl wrote:
>> > Should we do something similar with the FAQs at
>> > http://www.python.org/doc/faq/ ?
>>
>> They would then go into the main docs, I guess?
>
> That would make sense. ?However, we already have seven different FAQs
> there, so I don't think they should all be added as peers of the
> Tutorial, Library Reference, etc. ?Maybe there should be a single
> 'FAQ' document, with the existing faqs as subsections.
>
> Another option might be to put them on an entirely new site such as
> faq.python.org, and then configure www.python.org/doc/faq/ to proxy
> for it.
>
> Should I look at implementing the python.org as an output style for
> Sphinx?
>
> --amk
>

I'm definitely pro dev.python.org or python.org/dev

From andrewm at object-craft.com.au  Fri Sep 18 03:17:25 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Fri, 18 Sep 2009 11:17:25 +1000
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
Message-ID: <20090918011725.604DE5AC0D6@longblack.object-craft.com.au>

>off to patch the pep and implement some of the non controversial changes.

It might be a good idea to add some use-cases to the PEP.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From andrewm at object-craft.com.au  Fri Sep 18 03:41:32 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Fri, 18 Sep 2009 11:41:32 +1000
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
Message-ID: <20090918014132.1D2985AC0D6@longblack.object-craft.com.au>

>Again, the same error-catching functionality can be obtained through
>an option to the constructor. network and broadcast attributes can be
>renamed to .\1_address to alleviate confusion as well.
>
>I mentioned before that IPy's insistence on receiving masked out
>networks was one of the main reasons I wrote ipaddr to begin with.
>Having ipaddr mimic this behavior would make it significantly less
>useful. Removing functionality in the name of avoiding confusion
>doesn't make sense when the same confusion can be alleviated w/o the
>loss.

The issue is bigger than error checking - I'm maintaining that a
distinction between an Address (singleton, item) and a Network (Container)
is useful and should be embraced. The current implementation has already
partially gone this route, but hasn't completed the transition, and
this does not give the structure to users that it could - there's an
obligation on modules in the standard library to provide leadership and
clarity without being dictatorial. They are essentially silent mentors.

So, while I am not suggesting we build a bondage and discipline machine,
I am suggesting that partitioning the functionality differently will
result in a better outcome all round.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From rdmurray at bitdance.com  Fri Sep 18 04:25:24 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 22:25:24 -0400 (EDT)
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <20090918002423.GB7534@jroger.in-berlin.de>
References: <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
	<Pine.LNX.4.64.0909171329450.17020@kimball.webabinitio.net>
	<8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com>
	<Pine.LNX.4.64.0909171346550.17020@kimball.webabinitio.net>
	<20090918002423.GB7534@jroger.in-berlin.de>
Message-ID: <Pine.LNX.4.64.0909172222010.17020@kimball.webabinitio.net>

On Fri, 18 Sep 2009 at 02:24, Sebastian Rittau wrote:
> On Thu, Sep 17, 2009 at 02:04:11PM -0400, R. David Murray wrote:
>
>> I mean, eg, IPv4Network.fromHostIP('192.168.1.1/24').
>
> I'd actually suggest to use
>
>  >>> net, host = parse_network_and_host("192.168.111.33/24")
>  (IPv4Network('192.168.111.0/24'), IPv4Address('192.168.111.33'))
>  >>>
>
> I think this helps the use case of the short network+gateway notation,
> while keeping the concepts of network and host address cleanly separate.
>
> Then the regular IPv4Network constructor could be strict and can be used
> for validation.

I'd be happy with this solution, which was also suggested earlier
by someone else (Antoine?).

--David

From rdmurray at bitdance.com  Fri Sep 18 04:30:24 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 17 Sep 2009 22:30:24 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090918010446.50EB15AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org> <4AB22E4F.40509@gmail.com>
	<200909180344.31794.steve@pearwood.info>
	<20090918010446.50EB15AC0D6@longblack.object-craft.com.au>
Message-ID: <Pine.LNX.4.64.0909172225340.17020@kimball.webabinitio.net>

On Fri, 18 Sep 2009 at 11:04, Andrew McNamara wrote:
>[attribution lost; apparently Steven D'Aprano given the CC]
>> To a non-specialist, "the network address" is ambiguous. There are many
>> addresses in a network, and none of them are the entire network. It's
>> like saying, given a list [2, 4, 8, 12], what's "the list item"?
>
> A "network address" is an IP address and mask, but I understand your
> confusion - we're mixing terminology from disperate domains. In my
> postings, I have tried to refer to Network (a containter) and Address
> (an item).

Apparently not, in many people's vocabulary.  The 'network address'
is used to designate the IP address whose bits corresponding to the one
bits in the mask have been set to zero (ie: the first IP address in the
network range).

It is interesting how this item seems to lead to the greatest amount of
confusion on everyone's part, and I'm guessing it is because the common
terminology and usage blurs the line between addresses and networks.
And that's what we are trying to make clear(er) through a well structured
API.

--David

From steve at pearwood.info  Fri Sep 18 05:18:00 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 18 Sep 2009 13:18:00 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090918010446.50EB15AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909180344.31794.steve@pearwood.info>
	<20090918010446.50EB15AC0D6@longblack.object-craft.com.au>
Message-ID: <200909181318.00733.steve@pearwood.info>

On Fri, 18 Sep 2009 11:04:46 am Andrew McNamara wrote:
> >To a non-specialist, "the network address" is ambiguous. There are
> > many addresses in a network, and none of them are the entire
> > network. It's like saying, given a list [2, 4, 8, 12], what's "the
> > list item"?
>
> A "network address" is an IP address and mask, but I understand your
> confusion - we're mixing terminology from disperate domains. In my
> postings, I have tried to refer to Network (a containter) and Address
> (an item).


So to clarify, how many different things which need to be handled are 
there?

Items:
1 IP address  -- a 32 bit (IPv4) or 128 bit (IPv6) number
2 Netmask -- a bit mask of the form 111..100..0
3 Network address -- the lowest address in a network, and equal 
  to (defined by?) the bitwise-AND of any address in the network 
  with the network's netmask
4 Host address -- the part of the IP address that is not masked 
  by the netmask
5 Broadcast address -- the highest address in a IPv4 network

Containers:
6 Network -- a range of IP address

Have I missed anything or got anything wrong?



-- 
Steven D'Aprano

From steve at pearwood.info  Fri Sep 18 05:19:44 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 18 Sep 2009 13:19:44 +1000
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <20090918014132.1D2985AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
	<20090918014132.1D2985AC0D6@longblack.object-craft.com.au>
Message-ID: <200909181319.45083.steve@pearwood.info>

On Fri, 18 Sep 2009 11:41:32 am Andrew McNamara wrote:

> The issue is bigger than error checking - I'm maintaining that a
> distinction between an Address (singleton, item) and a Network
> (Container) is useful and should be embraced. The current
> implementation has already partially gone this route, but hasn't
> completed the transition, and this does not give the structure to
> users that it could - there's an obligation on modules in the
> standard library to provide leadership and clarity without being
> dictatorial. They are essentially silent mentors.
>
> So, while I am not suggesting we build a bondage and discipline
> machine, I am suggesting that partitioning the functionality
> differently will result in a better outcome all round.

As one of those non-expert users, can I give a big THANK YOU to Andrew 
for this.


-- 
Steven D'Aprano

From peter at hda3.com  Fri Sep 18 05:29:21 2009
From: peter at hda3.com (Peter Moody)
Date: Thu, 17 Sep 2009 20:29:21 -0700
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <20090918011725.604DE5AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
	<20090918011725.604DE5AC0D6@longblack.object-craft.com.au>
Message-ID: <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com>

On Thu, Sep 17, 2009 at 6:17 PM, Andrew McNamara
<andrewm at object-craft.com.au> wrote:
>>off to patch the pep and implement some of the non controversial changes.
>
> It might be a good idea to add some use-cases to the PEP.

There are several use-cases in the PEP already.

The problem is, for every use-case where one can show that the
existing implementation is confusing, I can come up with a use-case
showing where the existing implementation makes more sense than
anything proposed.

Cheers,
/peter

> --
> Andrew McNamara, Senior Developer, Object Craft
> http://www.object-craft.com.au/
>

From andrewm at object-craft.com.au  Fri Sep 18 05:42:03 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Fri, 18 Sep 2009 13:42:03 +1000
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
	<20090918011725.604DE5AC0D6@longblack.object-craft.com.au>
	<8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com>
Message-ID: <20090918034203.E16915AC0D6@longblack.object-craft.com.au>

>> It might be a good idea to add some use-cases to the PEP.
>
>There are several use-cases in the PEP already.

Maybe the use-cases deserve their own section in the PEP, or better yet,
be pulled up into the Motivation section.

>The problem is, for every use-case where one can show that the
>existing implementation is confusing, I can come up with a use-case
>showing where the existing implementation makes more sense than
>anything proposed.

Uh, I don't think that is the intention of use-cases - they're there
to inform the design, rather than to show how a specific implementation
can be used.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From peter at hda3.com  Fri Sep 18 05:42:27 2009
From: peter at hda3.com (Peter Moody)
Date: Thu, 17 Sep 2009 20:42:27 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <fbe2e2100909171110s523a3707w9d7d2196614326e3@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<ca471dc20909140954m4579654mfd75deed3697b9c1@mail.gmail.com>
	<77c780b40909151016n4675292craab781d115723383@mail.gmail.com>
	<8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com>
	<77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com>
	<8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com>
	<fbe2e2100909151334k62686719veac45c75c644858e@mail.gmail.com>
	<8517e9350909151343q4eb419d1v35ad5468bc2798a8@mail.gmail.com>
	<fbe2e2100909171110s523a3707w9d7d2196614326e3@mail.gmail.com>
Message-ID: <8517e9350909172042h21166d4bka8e1eb009ac1bf6d@mail.gmail.com>

On Thu, Sep 17, 2009 at 11:10 AM, Daniel Fetchinson
<fetchinson at googlemail.com> wrote:
>>>>>> 188 (check that, 190) people have downloaded the 2.0 release in the
>>>>>> last week (numbers publicly available from the code.google.com). I
>>>>>> can't tell you how many (if any) have downloaded it via svn.
>>>>>
>>>>> Downloading and using are not the same thing.
>>>>
>>>> Correct, but there is a strong positive correlation between the two.
>>>> If you have a better method for determining what you would consider an
>>>> appropriate level of usage, I'm all ears.
>>>
>>> A good way of determining the level of usage would be pointing to open
>>> source projects that are popular in the python community and which
>>> incorporate your module.
>>
>> well, the 2.0 release is still new. codesearch.google.com shows some
>> projects using the 1.x release; hopefully some of those 200
>> downloaders will put up some publicly indexable python code at some
>> point.
>
> I think one first needs to wait until this happens, I meana large user
> base is formed, before a meaningful discussion can be done on whether
> to include it in the stdlib or not. The long and largely academic
> thread here I think illustrates this point. Without a large user base
> it's up to anybody's gut feelings what is 'right' and what 'feels
> wrong'.

I like the sound of this.

I'm going to continue to work on the PEP and the reference
implementation, but as Martin said earlier, there's no reason to
hurry.

any discussion on ipaddr and/or PEP 3144 can continue on
ipaddr-py-dev at googlegroups.com in the meantime (past spam
unfortunately means that you have to subscribe in order to post,
apologies, but all are welcome).  Hopefully, the continued work will
help iron any remaining confusion of the API.

Cheers,
/peter

> Cheers,
> Daniel
>
>
> --
> Psss, psss, put it down! - http://www.cafepress.com/putitdown
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From steve at pearwood.info  Fri Sep 18 05:45:11 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 18 Sep 2009 13:45:11 +1000
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090918011725.604DE5AC0D6@longblack.object-craft.com.au>
	<8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com>
Message-ID: <200909181345.11638.steve@pearwood.info>

On Fri, 18 Sep 2009 01:29:21 pm Peter Moody wrote:
> On Thu, Sep 17, 2009 at 6:17 PM, Andrew McNamara
>
> <andrewm at object-craft.com.au> wrote:
> >>off to patch the pep and implement some of the non controversial
> >> changes.
> >
> > It might be a good idea to add some use-cases to the PEP.
>
> There are several use-cases in the PEP already.
>
> The problem is, for every use-case where one can show that the
> existing implementation is confusing, I can come up with a use-case
> showing where the existing implementation makes more sense than
> anything proposed.

Please excuse my skepticism, but I find that hard to believe.

Given the current API, to test if an address is in the same network as 
another address, you write:

addr_a in addr_b

Can you please give a use-case where that makes more sense than the 
suggested alternative:

addr_a in addr_b.network



-- 
Steven D'Aprano

From andrewm at object-craft.com.au  Fri Sep 18 06:26:19 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Fri, 18 Sep 2009 14:26:19 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909181318.00733.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909180344.31794.steve@pearwood.info>
	<20090918010446.50EB15AC0D6@longblack.object-craft.com.au>
	<200909181318.00733.steve@pearwood.info>
Message-ID: <20090918042620.0D3B85AC0D6@longblack.object-craft.com.au>

>On Fri, 18 Sep 2009 11:04:46 am Andrew McNamara wrote:
>> >To a non-specialist, "the network address" is ambiguous. There are
>> > many addresses in a network, and none of them are the entire
>> > network. It's like saying, given a list [2, 4, 8, 12], what's "the
>> > list item"?
>>
>> A "network address" is an IP address and mask, but I understand your
>> confusion - we're mixing terminology from disperate domains. In my
>> postings, I have tried to refer to Network (a containter) and Address
>> (an item).
>
>So to clarify, how many different things which need to be handled are 
>there?
>
>Items:
>1 IP address  -- a 32 bit (IPv4) or 128 bit (IPv6) number

Yes.

>2 Netmask -- a bit mask of the form 111..100..0

I don't think there's much to be gained by exposing a Netmask object,
although other objects might have a .netmask property returning an
IPAddress instance. Where we expose a netmask, it should be as an
Address instance (or maybe a subclass with additional restrictions).

>3 Network address -- the lowest address in a network, and equal 
>  to (defined by?) the bitwise-AND of any address in the network 
>  with the network's netmask

This idea of a "network address" being simply an IP address is in error - 
a network address was always an address and a mask, however in the
days prior to CIDR, the mask was implicitly specified by the class of
the network.

>4 Host address -- the part of the IP address that is not masked 
>  by the netmask

Well, yes, but I don't think we need an entity representing that.

>5 Broadcast address -- the highest address in a IPv4 network

Yes, but again, we don't need an entity - as with the netmask, when
exposed, it should just be an Address instance (or subclass thereof).

>Containers:
>6 Network -- a range of IP address

Yes, although not an arbitrary or discontinuous range of address.

Really, I think we just need two entities (per protocol):

 Address (& maybe AddressWithMask)

   * If no mask is specified, this is just the IP address.
   * If a mask is specified, then it gains a .network property returning a
     Network instance. It probably should also have a .netmask property
     containing an Address instance.

 Network

   * Has an IP address with netmask 
   * for consistency's sake, masked address bits are not allowed
   * behaves like a read-only container wrt Addresses

So, you want to represent an interface on your host:

  >>> if_addr = IPv4Address('10.0.0.1/24')

from this, you could get:

  >>> if_addr.address
  IPv4Address('10.0.0.1')
  >>> if_addr.netmask
  IPv4Address('255.255.255.0')
  >>> if_addr.broadcast
  IPv4Address('10.0.0.255')
  >>> if_addr.network
  IPV4Network('10.0.0.0/24')

you might also have an address for the default gateway:

  >>> router_addr = IPv4Address('10.0.0.254/24')
  >>> router_addr in if_addr.network
  True

or:

  >>> router_addr = IPv4Address('10.0.0.254')
  >>> router_addr in if_addr.network
  True

Or maybe you've subneted your LAN:

  >>> IPV4Network('10.0.0.0/24') in IPv4Network('10.0.0.0/8')
  True
  >>> IPV4Network('10.0.1.0/24') in IPv4Network('10.0.0.0/8')
  True

but:

  >>> IPV4Network('10.0.0.0/8') in IPv4Network('10.0.0.0/24')
  False

This suggests the natural behaviour if the Address mask doesn't fit in the
network:

  >>> IPv4Address('10.0.0.254/8') in IPv4Network('10.0.0.0/24')
  False

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From ncoghlan at gmail.com  Fri Sep 18 11:11:21 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 18 Sep 2009 19:11:21 +1000
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <E02AC97B-3ADB-4073-B173-A4C5ACC27A29@acm.org>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>	<h8tsb9$bkq$1@ger.gmane.org>	<bbaeab100909171056l4a7c1845yd2a73bf01a423a49@mail.gmail.com>
	<E02AC97B-3ADB-4073-B173-A4C5ACC27A29@acm.org>
Message-ID: <4AB34EB9.4080306@gmail.com>

Fred Drake wrote:
> On Thu, Sep 17, 2009 at 10:38, Georg Brandl <g.brandl at gmx.net> wrote:
>> So the plan would be to consolidate these into another set of rst docs,
>> having them in the repo, editable by every committer, as well as
>> published
>> somewhere on python.org (devdocs.python.org or somesuch).
> 
> On Sep 17, 2009, at 1:56 PM, Brett Cannon wrote:
>> dev.python.org would be nice to have, from which we can simply
>> redirect www.python.org/dev/ to dev.python.org. www.python.org/dev/
>> can then get cleaned up be made simpler to navigate and more obvious
>> for how people can get started.
> 
> Many years ago, we decided to add docs.python.org with the "current"
> version of the documentation, so people would be able to find the docs
> more easily.  Since then, we've had problems with keeping
> docs.python.org and www.python.org/doc/ in sync, and with different
> styles being applied to the sites.

We can avoid that problem by having the URLs just be alternative names
for the same thing (as Brett suggested), rather than having them be
actually different sites (as is the case for docs.python.org and
www.python.org/doc).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From p.f.moore at gmail.com  Fri Sep 18 12:37:32 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 18 Sep 2009 11:37:32 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909172225340.17020@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org> <4AB22E4F.40509@gmail.com>
	<200909180344.31794.steve@pearwood.info>
	<20090918010446.50EB15AC0D6@longblack.object-craft.com.au>
	<Pine.LNX.4.64.0909172225340.17020@kimball.webabinitio.net>
Message-ID: <79990c6b0909180337x536b0be2xa183bb3cb56f8d3d@mail.gmail.com>

2009/9/18 R. David Murray <rdmurray at bitdance.com>:
> On Fri, 18 Sep 2009 at 11:04, Andrew McNamara wrote:
>>
>> [attribution lost; apparently Steven D'Aprano given the CC]
>>>
>>> To a non-specialist, "the network address" is ambiguous. There are many
>>> addresses in a network, and none of them are the entire network. It's
>>> like saying, given a list [2, 4, 8, 12], what's "the list item"?
>>
>> A "network address" is an IP address and mask, but I understand your
>> confusion - we're mixing terminology from disperate domains. In my
>> postings, I have tried to refer to Network (a containter) and Address
>> (an item).
>
> Apparently not, in many people's vocabulary. ?The 'network address'
> is used to designate the IP address whose bits corresponding to the one
> bits in the mask have been set to zero (ie: the first IP address in the
> network range).
>
> It is interesting how this item seems to lead to the greatest amount of
> confusion on everyone's part, and I'm guessing it is because the common
> terminology and usage blurs the line between addresses and networks.
> And that's what we are trying to make clear(er) through a well structured
> API.

How non-expert do you want to get? My immediate reaction is that the
network address of my PC is 157.215.187.94 - it's what comes up ad "IP
address" when I type ipconfig. I understand why that's "wrong", and I
see why the definitions above are "better", but that doesn't affect my
instinct.

Suggestion - just don't use the term "network address" anywhere in the
library, the PEP, or its documentation. The term seems too loaded with
contradictory meanings to be useful. If the concept behind it (one of
the many concepts - among our concepts...) is useful, then maybe coin
a new term specific to the module.

Paul.

From rdmurray at bitdance.com  Fri Sep 18 15:24:32 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 18 Sep 2009 09:24:32 -0400 (EDT)
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
	<20090918011725.604DE5AC0D6@longblack.object-craft.com.au>
	<8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909180922050.17020@kimball.webabinitio.net>

On Thu, 17 Sep 2009 at 20:29, Peter Moody wrote:
> On Thu, Sep 17, 2009 at 6:17 PM, Andrew McNamara
> <andrewm at object-craft.com.au> wrote:
>>> off to patch the pep and implement some of the non controversial changes.
>>
>> It might be a good idea to add some use-cases to the PEP.
>
> There are several use-cases in the PEP already.
>
> The problem is, for every use-case where one can show that the
> existing implementation is confusing, I can come up with a use-case
> showing where the existing implementation makes more sense than
> anything proposed.

Then all of those use cases should be in the PEP, so we can try to find
the best refinement we can of the API for handling all of them.

--David

From status at bugs.python.org  Fri Sep 18 18:07:12 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 18 Sep 2009 18:07:12 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090918160712.81C0F7863C@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (09/11/09 - 09/18/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2396 open (+27) / 16366 closed (+29) / 18762 total (+56)

Open issues with patches:   953

Average duration of open issues: 664 days.
Median duration of open issues: 420 days.

Open Issues Breakdown
   open  2363 (+27)
pending    32 ( +0)

Issues Created Or Reopened (56)
_______________________________

pdb documentation shows running as script using "python" not "py 09/11/09
CLOSED http://bugs.python.org/issue6885    created  MLModel                       
                                                                               

cgi.py runs python, not python3                                  09/11/09
       http://bugs.python.org/issue6886    created  MLModel                       
                                                                               

executables in lib use /usr/bin/env python, not python3          09/11/09
       http://bugs.python.org/issue6887    created  MLModel                       
                                                                               

pdb alias command with no arguments is broken                    09/11/09
CLOSED http://bugs.python.org/issue6888    created  MLModel                       
                                                                               

??Python???s default encoding is the ???ascii??? encoding.??     09/11/09
CLOSED http://bugs.python.org/issue6889    created  flox                          
                                                                               

IOError has no __unicode__ method - and loses information        09/11/09
CLOSED http://bugs.python.org/issue6890    created  michael.foord                 
       26backport                                                              

small typo in ??unicode.rst??: ??the Uniode APIs should be used? 09/11/09
CLOSED http://bugs.python.org/issue6891    created  flox                          
                                                                               

optparser example in optparse documentation is broken            09/12/09
CLOSED http://bugs.python.org/issue6892    created  MLModel                       
                                                                               

defaultdict example in py3 doc should mention the new Counter cl 09/12/09
       http://bugs.python.org/issue6893    created  ezio.melotti                  
                                                                               

urllib2 doesn't respect "no_proxy" environment  (python2.6.2)    09/12/09
       http://bugs.python.org/issue6894    created  xelnor                        
       patch                                                                   

locale._parse_localename fails when localename does not contain  09/12/09
       http://bugs.python.org/issue6895    created  santhosh.thottingal           
       patch                                                                   

Intermittent failures in test_mailbox                            09/12/09
       http://bugs.python.org/issue6896    created  ezio.melotti                  
                                                                               

imaplib fails during login                                       09/12/09
       http://bugs.python.org/issue6897    created  debatem1                      
                                                                               

Unicode  Error                                                   09/13/09
CLOSED http://bugs.python.org/issue6898    created  vitvn                         
                                                                               

Base.replace breaks tree                                         09/13/09
       http://bugs.python.org/issue6899    created  loewis                        
       patch                                                                   

Sub-optimal "Locate" button behaviour in Windows CHM file        09/13/09
       http://bugs.python.org/issue6900    created  tom_seddon                    
                                                                               

Runaway programs often become unresponsive to CTRL-C             09/13/09
CLOSED http://bugs.python.org/issue6901    created  bernie                        
                                                                               

Built-in types format incorrectly with 0 padding.                09/13/09
       http://bugs.python.org/issue6902    created  eric.smith                    
                                                                               

pdb - print in for iteration prints None after printing          09/14/09
CLOSED http://bugs.python.org/issue6903    created  MLModel                       
                                                                               

Broken URL for pyqt                                              09/14/09
CLOSED http://bugs.python.org/issue6904    created  thijs                         
                                                                               

inspect.getargspec(print) fails                                  09/14/09
CLOSED http://bugs.python.org/issue6905    created  ubershmekel                   
                                                                               

Tkinter sets an unicode environment variable on win32            09/14/09
       http://bugs.python.org/issue6906    created  dotz                          
                                                                               

xmlrpclib.make_connection() is not thread safe                   09/14/09
CLOSED http://bugs.python.org/issue6907    created  haypo                         
                                                                               

Minor markup error in hashlib docs                               09/14/09
CLOSED http://bugs.python.org/issue6908    created  lehmannro                     
       patch                                                                   

python 3.1 - filecmp.cmp exception based on file name            09/14/09
CLOSED http://bugs.python.org/issue6909    created  jdeifik                       
                                                                               

1-char typo in language reference doc of import                  09/14/09
CLOSED http://bugs.python.org/issue6910    created  MLModel                       
                                                                               

Document changes in asynchat                                     09/14/09
       http://bugs.python.org/issue6911    created  lehmannro                     
       patch                                                                   

Add 'with' block support to Tools/Scripts/pindent.py             09/14/09
CLOSED http://bugs.python.org/issue6912    created  tjreedy                       
                                                                               

Py_SetPythonHome incorrectly documented py3.x (rev 74792)        09/15/09
CLOSED http://bugs.python.org/issue6913    created  ideasman42                    
       patch                                                                   

Py_SetPythonHome, undocumented behavoir                          09/15/09
CLOSED http://bugs.python.org/issue6914    created  ideasman42                    
                                                                               

os.listdir inconsistenly releases the GIL on win32               09/15/09
       http://bugs.python.org/issue6915    created  rfk                           
       patch                                                                   

Remove deprecated items from asynchat                            09/15/09
       http://bugs.python.org/issue6916    created  lehmannro                     
       patch                                                                   

".et_folder" instead of "get_folder" in mailbox documentation    09/15/09
CLOSED http://bugs.python.org/issue6917    created  dauerbaustelle                
                                                                               

ctypes compilation error on SnowLeopard                          09/15/09
       http://bugs.python.org/issue6918    created  ronaldoussoren                
                                                                               

Link CRT Statically                                              09/16/09
CLOSED http://bugs.python.org/issue6919    created  hankdane                      
                                                                               

IDLE 3 crashes on Ctrl+Space with "'utf8' codec can't decode byt 09/16/09
CLOSED http://bugs.python.org/issue6920    created  Yinon                         
                                                                               

recursion wierdness ... related to variable scoping?             09/16/09
CLOSED http://bugs.python.org/issue6921    created  mrdiskodave                   
                                                                               

Interpreter hangs up while trying to decode invalid utf32 stream 09/16/09
CLOSED http://bugs.python.org/issue6922    created  mwizard                       
       patch                                                                   

Need pthread_atfork-like functionality in CPython                09/16/09
       http://bugs.python.org/issue6923    created  twouters                      
                                                                               

struct.unpack weird behavior with "bi" (byte then integer)       09/16/09
CLOSED http://bugs.python.org/issue6924    created  Manux                         
                                                                               

Doc for locals and vars                                          09/16/09
       http://bugs.python.org/issue6925    created  tjreedy                       
                                                                               

socket module missing IPPROTO_IPV6, IPPROTO_IPV4                 09/16/09
       http://bugs.python.org/issue6926    created  jaraco                        
                                                                               

Metaclass doc (Ref 3.3.3) errors                                 09/16/09
       http://bugs.python.org/issue6927    created  tjreedy                       
                                                                               

Doc:  Class statements and metaclass = xxx                       09/16/09
       http://bugs.python.org/issue6928    created  tjreedy                       
                                                                               

Confusion between "write" method of rowiobase and rawfileio      09/17/09
CLOSED http://bugs.python.org/issue6929    created  pakal                         
                                                                               

[PATCH] PyUnicode_DecodeUTF16 docs wrong (byteorder param)       09/17/09
       http://bugs.python.org/issue6930    created  JonFoster                     
       patch                                                                   

dreadful performance in difflib: ndiff and HtmlDiff              09/17/09
       http://bugs.python.org/issue6931    created  heidar.rafn                   
                                                                               

Open shelves fail when Python exits                              09/17/09
CLOSED http://bugs.python.org/issue6932    created  lehmannro                     
       patch                                                                   

Threading issue with Tkinter Frame.insert                        09/17/09
       http://bugs.python.org/issue6933    created  moraygrieve                   
                                                                               

[PATCH] postflight.framework script (from the Mac OS X .dmg inst 09/17/09
       http://bugs.python.org/issue6934    created  slavi                         
       patch                                                                   

Version updates needed in tutorial                               09/17/09
CLOSED http://bugs.python.org/issue6935    created  MLModel                       
                                                                               

Import needed to quit Python?                                    09/17/09
CLOSED http://bugs.python.org/issue6936    created  MLModel                       
                                                                               

multiprocessing lock on OS X Snow Leopard dumps core             09/18/09
       http://bugs.python.org/issue6937    created  aaditya                       
                                                                               

Incorrect multiprocessing log string in managers.py (python 2.6. 09/18/09
CLOSED http://bugs.python.org/issue6938    created  ftofficer                     
       patch                                                                   

shadows around the io truncate() semantics                       09/18/09
       http://bugs.python.org/issue6939    created  pakal                         
                                                                               

cannot compute sizeof wchar_t on OS X 10.5.7, Python 2.6.2       09/18/09
       http://bugs.python.org/issue6940    created  rajarshi                      
                                                                               



Issues Now Closed (60)
______________________

xmlrpclib closes connection after each call                       580 days
       http://bugs.python.org/issue2076    haypo                         
                                                                               

Error when printing an exception containing a Unicode string      535 days
       http://bugs.python.org/issue2517    ezio.melotti                  
       patch                                                                   

Windows installer should add Python and Scripts directories to t  396 days
       http://bugs.python.org/issue3561    srid                          
                                                                               

Issue with RotatingFileHandler logging handler on Windows          99 days
       http://bugs.python.org/issue4749    gagenellina                   
                                                                               

Add description of special case to "Assignment statements" secti  169 days
       http://bugs.python.org/issue5621    georg.brandl                  
       patch                                                                   

test_(zipfile|zipimport|gzip|distutils|sqlite) fail if zlib is n  121 days
       http://bugs.python.org/issue6026    ezio.melotti                  
                                                                               

Add "intel" universal architecture on OSX                          99 days
       http://bugs.python.org/issue6245    ronaldoussoren                
                                                                               

Bug in hashlib                                                     93 days
       http://bugs.python.org/issue6281    Eloff                         
                                                                               

Error in tutorial section 7.2                                      88 days
       http://bugs.python.org/issue6301    michael.foord                 
                                                                               

Tkinter cannot find *64 bit* Tcl/Tk on Mac OS X                    69 days
       http://bugs.python.org/issue6441    ronaldoussoren                
       patch                                                                   

Make inf be almost equal to inf                                    50 days
       http://bugs.python.org/issue6567    michael.foord                 
                                                                               

unittest test discovery improvements                               51 days
       http://bugs.python.org/issue6568    michael.foord                 
                                                                               

List the __future__ features in a table                            50 days
       http://bugs.python.org/issue6574    lehmannro                     
       patch, easy                                                             

Profiler doesn't print usage (indexError instead)                  41 days
       http://bugs.python.org/issue6635    doko                          
       patch                                                                   

sys._getframe is not available on all Python implementations       28 days
       http://bugs.python.org/issue6712    michael.foord                 
                                                                               

Python as zip package                                              25 days
       http://bugs.python.org/issue6767    senthil.kumaran               
                                                                               

2to3 assignment division conversion                                 9 days
       http://bugs.python.org/issue6831    benjamin.peterson             
                                                                               

BaseException DeprecationError raises inappropriately              11 days
       http://bugs.python.org/issue6844    georg.brandl                  
       patch                                                                   

Tutorial changes                                                    7 days
       http://bugs.python.org/issue6849    michael.foord                 
       easy                                                                    

allow setting uid and gid when creating tar files                   5 days
       http://bugs.python.org/issue6856    lars.gustaebel                
       patch                                                                   

TestLoader.loadTestsFromName does not use suiteClass attribute t    5 days
       http://bugs.python.org/issue6866    michael.foord                 
       patch                                                                   

sequence method  .count() and .index() shoud be in immutable seq    2 days
       http://bugs.python.org/issue6874    tjreedy                       
                                                                               

add os.close() suggestion to mkstemp documentation                  6 days
       http://bugs.python.org/issue6875    georg.brandl                  
       patch                                                                   

readline documentation example error                                6 days
       http://bugs.python.org/issue6876    georg.brandl                  
                                                                               

misstatement in example explanation using raise                     5 days
       http://bugs.python.org/issue6879    ezio.melotti                  
                                                                               

class needs forward reference                                       5 days
       http://bugs.python.org/issue6880    georg.brandl                  
                                                                               

pdb documentation shows running as script using "python" not "py    5 days
       http://bugs.python.org/issue6885    georg.brandl                  
                                                                               

pdb alias command with no arguments is broken                       0 days
       http://bugs.python.org/issue6888    benjamin.peterson             
                                                                               

??Python???s default encoding is the ???ascii??? encoding.??        0 days
       http://bugs.python.org/issue6889    benjamin.peterson             
                                                                               

IOError has no __unicode__ method - and loses information           5 days
       http://bugs.python.org/issue6890    georg.brandl                  
       26backport                                                              

small typo in ??unicode.rst??: ??the Uniode APIs should be used?    5 days
       http://bugs.python.org/issue6891    georg.brandl                  
                                                                               

optparser example in optparse documentation is broken               4 days
       http://bugs.python.org/issue6892    georg.brandl                  
                                                                               

Unicode  Error                                                      0 days
       http://bugs.python.org/issue6898    ezio.melotti                  
                                                                               

Runaway programs often become unresponsive to CTRL-C                0 days
       http://bugs.python.org/issue6901    bernie                        
                                                                               

pdb - print in for iteration prints None after printing             3 days
       http://bugs.python.org/issue6903    georg.brandl                  
                                                                               

Broken URL for pyqt                                                 0 days
       http://bugs.python.org/issue6904    ezio.melotti                  
                                                                               

inspect.getargspec(print) fails                                     4 days
       http://bugs.python.org/issue6905    georg.brandl                  
                                                                               

xmlrpclib.make_connection() is not thread safe                      1 days
       http://bugs.python.org/issue6907    krisvale                      
                                                                               

Minor markup error in hashlib docs                                  0 days
       http://bugs.python.org/issue6908    georg.brandl                  
       patch                                                                   

python 3.1 - filecmp.cmp exception based on file name               0 days
       http://bugs.python.org/issue6909    r.david.murray                
                                                                               

1-char typo in language reference doc of import                     1 days
       http://bugs.python.org/issue6910    ezio.melotti                  
                                                                               

Add 'with' block support to Tools/Scripts/pindent.py                3 days
       http://bugs.python.org/issue6912    georg.brandl                  
                                                                               

Py_SetPythonHome incorrectly documented py3.x (rev 74792)           0 days
       http://bugs.python.org/issue6913    benjamin.peterson             
       patch                                                                   

Py_SetPythonHome, undocumented behavoir                             0 days
       http://bugs.python.org/issue6914    benjamin.peterson             
                                                                               

".et_folder" instead of "get_folder" in mailbox documentation       0 days
       http://bugs.python.org/issue6917    ezio.melotti                  
                                                                               

Link CRT Statically                                                 2 days
       http://bugs.python.org/issue6919    hankdane                      
                                                                               

IDLE 3 crashes on Ctrl+Space with "'utf8' codec can't decode byt    0 days
       http://bugs.python.org/issue6920    ezio.melotti                  
                                                                               

recursion wierdness ... related to variable scoping?                0 days
       http://bugs.python.org/issue6921    mrdiskodave                   
                                                                               

Interpreter hangs up while trying to decode invalid utf32 stream    1 days
       http://bugs.python.org/issue6922    georg.brandl                  
       patch                                                                   

struct.unpack weird behavior with "bi" (byte then integer)          0 days
       http://bugs.python.org/issue6924    mark.dickinson                
                                                                               

Confusion between "write" method of rowiobase and rawfileio         0 days
       http://bugs.python.org/issue6929    pitrou                        
                                                                               

Open shelves fail when Python exits                                 0 days
       http://bugs.python.org/issue6932    georg.brandl                  
       patch                                                                   

Version updates needed in tutorial                                  0 days
       http://bugs.python.org/issue6935    georg.brandl                  
                                                                               

Import needed to quit Python?                                       0 days
       http://bugs.python.org/issue6936    georg.brandl                  
                                                                               

Incorrect multiprocessing log string in managers.py (python 2.6.    0 days
       http://bugs.python.org/issue6938    georg.brandl                  
       patch                                                                   

distutils ignores configure's --includedir                       1840 days
       http://bugs.python.org/issue1019715 senthil.kumaran               
                                                                               

os.path.splitext don't handle unix hidden file correctly         1684 days
       http://bugs.python.org/issue1115886 senthil.kumaran               
                                                                               

Function-level import in os triggering an threaded import deadlo  104 days
       http://bugs.python.org/issue1590864 twouters                      
       patch                                                                   

xmlrpclib ServerProxy uses old httplib interface                 1010 days
       http://bugs.python.org/issue1613573 techtonik                     
       patch                                                                   

Make xmlrpc use HTTP/1.1 and keepalive                            772 days
       http://bugs.python.org/issue1767370 haypo                         
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 14 should we include argparse                                         8 days
open    http://bugs.python.org/issue6247   

 10 BaseException DeprecationError raises inappropriately             11 days
closed  http://bugs.python.org/issue6844   

  9 uuid creates zombies                                               7 days
open    http://bugs.python.org/issue6882   

  9 xmlrpclib ServerProxy uses old httplib interface                1010 days
closed  http://bugs.python.org/issue1613573

  7 multiprocessing lock on OS X Snow Leopard dumps core               0 days
open    http://bugs.python.org/issue6937   

  7 expose setresuid                                                  63 days
open    http://bugs.python.org/issue6508   

  7 encode and decode should accept 'errors' as a keyword argument    93 days
open    http://bugs.python.org/issue6300   

  5 Confusion between "write" method of rowiobase and rawfileio        0 days
closed  http://bugs.python.org/issue6929   

  5 Link CRT Statically                                                2 days
closed  http://bugs.python.org/issue6919   

  5 IDLE 2.6.1 locks up on Mac OS 10.6                                10 days
open    http://bugs.python.org/issue6864   




From theller at ctypes.org  Fri Sep 18 19:13:36 2009
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 18 Sep 2009 19:13:36 +0200
Subject: [Python-Dev] OS X buildbot
Message-ID: <h90f44$9ta$1@ger.gmane.org>

I have updated the OS X buildbot to Snow Leopard.

-- 
Thomas


From chambon.pascal at gmail.com  Fri Sep 18 21:17:29 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Fri, 18 Sep 2009 21:17:29 +0200
Subject: [Python-Dev] Fuzziness in io module specs
Message-ID: <4AB3DCC9.5050209@gmail.com>

Hello everyone

I'm currently working on a reimplementation of io.FileIO, which would 
allow cross-platform file range locking and all kinds of other safety 
features ; however I'm slightly stuck due to some specification 
fuzziness in the IO docs.
CF     http://bugs.python.org/issue6939

The main points that annoy me at the moment :
- it is unclear what truncate() methods do with the file pointer, and 
even if the current implementation simply moves it to the truncation 
point, it's very contrary to the standard way of doing under unix, where 
the file pointer is normally left unchanged. Shouldn't we specify that 
the file pointer remains unmoved, and fix the _fileio module accordingly ?
- exceptions are not always specified, and even if most of them are 
IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if 
we try to wrap a wrong file descriptor when instanciating a new FileIO). 
This might lead to bad program crashes if some people don't "refuse the 
temptation to guess" and only get prepared to catch IOErrors
- the doc sometimes says that when we receive an empty string from a 
read() operation, without exceptions, it means the file is empty. 
However, with the current implementation, if we call file.read(0), we 
simply receive "", even though it doesn't mean that we're at EOF. 
Shouldn't we avoid this (rare, I admit) ambiguity on the return value, 
by preventing read(0) ? Or at least, note in the doc that (we receive an 
empty string) <-> (the file is at EOF OR we called read with 0 as 
parameter) ?

Are there some arguments that I don't know, which lead to this or that 
particular implementation choice ?
I'd strongly advocate very detailled specifications, letting no room for 
cross-platform subtilities (that's also a strong goal of my 
reimplemntation), since that new IO system (which saved me a lot of 
coding time, by the way) should become the base of many programs.

So wouldn't it be a godo idea to write some kind of mini-pep, just to 
fix the corner cases of the current IO documentation ? I might handle 
it, if no more-knowledgeable people feels like it.

Regards,
Pascal

From guido at python.org  Fri Sep 18 21:32:14 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 18 Sep 2009 12:32:14 -0700
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <4AB3DCC9.5050209@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>
Message-ID: <ca471dc20909181232t661add58o4eeae8d5c98996b4@mail.gmail.com>

Why not propose an update to the existing PEP to clarify the vaguenesses?

On Fri, Sep 18, 2009 at 12:17 PM, Pascal Chambon
<chambon.pascal at gmail.com> wrote:
> Hello everyone
>
> I'm currently working on a reimplementation of io.FileIO, which would allow
> cross-platform file range locking and all kinds of other safety features ;
> however I'm slightly stuck due to some specification fuzziness in the IO
> docs.
> CF ? ? http://bugs.python.org/issue6939
>
> The main points that annoy me at the moment :
> - it is unclear what truncate() methods do with the file pointer, and even
> if the current implementation simply moves it to the truncation point, it's
> very contrary to the standard way of doing under unix, where the file
> pointer is normally left unchanged. Shouldn't we specify that the file
> pointer remains unmoved, and fix the _fileio module accordingly ?
> - exceptions are not always specified, and even if most of them are
> IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if we
> try to wrap a wrong file descriptor when instanciating a new FileIO). This
> might lead to bad program crashes if some people don't "refuse the
> temptation to guess" and only get prepared to catch IOErrors
> - the doc sometimes says that when we receive an empty string from a read()
> operation, without exceptions, it means the file is empty. However, with the
> current implementation, if we call file.read(0), we simply receive "", even
> though it doesn't mean that we're at EOF. Shouldn't we avoid this (rare, I
> admit) ambiguity on the return value, by preventing read(0) ? Or at least,
> note in the doc that (we receive an empty string) <-> (the file is at EOF OR
> we called read with 0 as parameter) ?
>
> Are there some arguments that I don't know, which lead to this or that
> particular implementation choice ?
> I'd strongly advocate very detailled specifications, letting no room for
> cross-platform subtilities (that's also a strong goal of my
> reimplemntation), since that new IO system (which saved me a lot of coding
> time, by the way) should become the base of many programs.
>
> So wouldn't it be a godo idea to write some kind of mini-pep, just to fix
> the corner cases of the current IO documentation ? I might handle it, if no
> more-knowledgeable people feels like it.
>
> Regards,
> Pascal
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at mrabarnett.plus.com  Fri Sep 18 21:55:00 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 18 Sep 2009 20:55:00 +0100
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <4AB3DCC9.5050209@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>
Message-ID: <4AB3E594.5090907@mrabarnett.plus.com>

[Oops! Hit Send to soon]

Pascal Chambon wrote:
> Hello everyone
> 
> I'm currently working on a reimplementation of io.FileIO, which would 
> allow cross-platform file range locking and all kinds of other safety 
> features ; however I'm slightly stuck due to some specification 
> fuzziness in the IO docs.
> CF     http://bugs.python.org/issue6939
> 
> The main points that annoy me at the moment :
> - it is unclear what truncate() methods do with the file pointer, and 
> even if the current implementation simply moves it to the truncation 
> point, it's very contrary to the standard way of doing under unix, where 
> the file pointer is normally left unchanged. Shouldn't we specify that 
> the file pointer remains unmoved, and fix the _fileio module accordingly ?

I think that this should be an invariant:

     0 <= file pointer <= file size

so the file pointer might sometimes have to be moved.

> - exceptions are not always specified, and even if most of them are 
> IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if 
> we try to wrap a wrong file descriptor when instanciating a new FileIO). 
> This might lead to bad program crashes if some people don't "refuse the 
> temptation to guess" and only get prepared to catch IOErrors
> - the doc sometimes says that when we receive an empty string from a 
> read() operation, without exceptions, it means the file is empty. 
> However, with the current implementation, if we call file.read(0), we 
> simply receive "", even though it doesn't mean that we're at EOF. 
> Shouldn't we avoid this (rare, I admit) ambiguity on the return value, 
> by preventing read(0) ? Or at least, note in the doc that (we receive an 
> empty string) <-> (the file is at EOF OR we called read with 0 as 
> parameter) ?
> 
If you ask for 0 bytes then you get 0 bytes.

> Are there some arguments that I don't know, which lead to this or that 
> particular implementation choice ?
> I'd strongly advocate very detailled specifications, letting no room for 
> cross-platform subtilities (that's also a strong goal of my 
> reimplemntation), since that new IO system (which saved me a lot of 
> coding time, by the way) should become the base of many programs.
> 
> So wouldn't it be a godo idea to write some kind of mini-pep, just to 
> fix the corner cases of the current IO documentation ? I might handle 
> it, if no more-knowledgeable people feels like it.
> 
As for the question of whether 'truncate' should be able to lengthen a
file, the method name suggests no; if the method name were 'resize', for
example, then maybe yes, zeroing the new bytes for security.

From mrabarnett at mrabarnett.plus.com  Fri Sep 18 21:50:33 2009
From: mrabarnett at mrabarnett.plus.com (Matthew Barnett)
Date: Fri, 18 Sep 2009 20:50:33 +0100
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <4AB3DCC9.5050209@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>
Message-ID: <4AB3E489.3010506@mrabarnett.plus.com>

Pascal Chambon wrote:
> Hello everyone
> 
> I'm currently working on a reimplementation of io.FileIO, which would 
> allow cross-platform file range locking and all kinds of other safety 
> features ; however I'm slightly stuck due to some specification 
> fuzziness in the IO docs.
> CF     http://bugs.python.org/issue6939
> 
> The main points that annoy me at the moment :
> - it is unclear what truncate() methods do with the file pointer, and 
> even if the current implementation simply moves it to the truncation 
> point, it's very contrary to the standard way of doing under unix, where 
> the file pointer is normally left unchanged. Shouldn't we specify that 
> the file pointer remains unmoved, and fix the _fileio module accordingly ?

I think that this should be an invariant:

     0 <= file pointer <= file size

so the file pointer might sometimes have to be moved.

> - exceptions are not always specified, and even if most of them are 
> IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if 
> we try to wrap a wrong file descriptor when instanciating a new FileIO). 
> This might lead to bad program crashes if some people don't "refuse the 
> temptation to guess" and only get prepared to catch IOErrors
> - the doc sometimes says that when we receive an empty string from a 
> read() operation, without exceptions, it means the file is empty. 
> However, with the current implementation, if we call file.read(0), we 
> simply receive "", even though it doesn't mean that we're at EOF. 
> Shouldn't we avoid this (rare, I admit) ambiguity on the return value, 
> by preventing read(0) ? Or at least, note in the doc that (we receive an 
> empty string) <-> (the file is at EOF OR we called read with 0 as 
> parameter) ?
> 
If you ask for 0 bytes then you get 0 bytes.

> Are there some arguments that I don't know, which lead to this or that 
> particular implementation choice ?
> I'd strongly advocate very detailled specifications, letting no room for 
> cross-platform subtilities (that's also a strong goal of my 
> reimplemntation), since that new IO system (which saved me a lot of 
> coding time, by the way) should become the base of many programs.
> 
> So wouldn't it be a godo idea to write some kind of mini-pep, just to 
> fix the corner cases of the current IO documentation ? I might handle 
> it, if no more-knowledgeable people feels like it.
> 


From foom at fuhm.net  Fri Sep 18 22:08:23 2009
From: foom at fuhm.net (James Y Knight)
Date: Fri, 18 Sep 2009 16:08:23 -0400
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <4AB3E594.5090907@mrabarnett.plus.com>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
Message-ID: <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>


On Sep 18, 2009, at 3:55 PM, MRAB wrote:

> I think that this should be an invariant:
>
>    0 <= file pointer <= file size
>
> so the file pointer might sometimes have to be moved.

> As for the question of whether 'truncate' should be able to lengthen a
> file, the method name suggests no; if the method name were 'resize',  
> for
> example, then maybe yes, zeroing the new bytes for security.


Why are you just making things up? There is a *vast* amount of  
precedent for how file operations should work. Python should follow  
that precedent and do like POSIX unless there's a compelling reason  
not to. Quoting:

        If  fildes  refers  to  a  regular  file,  the ftruncate()  
function shall cause the size of the file to be truncated to
        length. If the size of the file previously exceeded length,  
the extra data shall no longer be available to reads on the
        file.  If  the  file  previously  was smaller than this size,  
ftruncate() shall either increase the size of the file or
        fail.   XSI-conformant systems shall increase the size of the  
file.  If the file size is increased, the  extended  area
        shall appear as if it were zero-filled. The value of the seek  
pointer shall not be modified by a call to ftruncate().

James

From daniel at stutzbachenterprises.com  Fri Sep 18 22:15:28 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Fri, 18 Sep 2009 15:15:28 -0500
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <4AB3DCC9.5050209@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>
Message-ID: <eae285400909181315s5758ca5dua789d159391c528f@mail.gmail.com>

On Fri, Sep 18, 2009 at 2:17 PM, Pascal Chambon <chambon.pascal at gmail.com>wrote:

> - it is unclear what truncate() methods do with the file pointer, and even
> if the current implementation simply moves it to the truncation point, it's
> very contrary to the standard way of doing under unix, where the file
> pointer is normally left unchanged. Shouldn't we specify that the file
> pointer remains unmoved, and fix the _fileio module accordingly ?
>

+1 on having consistent, documented behavior (regardless of what that
behavior is :) ).


> - exceptions are not always specified, and even if most of them are
> IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if we
> try to wrap a wrong file descriptor when instanciating a new FileIO). This
> might lead to bad program crashes if some people don't "refuse the
> temptation to guess" and only get prepared to catch IOErrors
>

I'd wager that you may also get a WindowsError in some cases, on Windows
systems.  Part of the reason for having several different, but similar,
exceptions is that they may contain operating system specific error codes
and the type of exception helps the programmer figure out how to interpret
those codes.  I'm not really clear on when IOError is preferred over
OSError, but I know that WindowsError necessarily uses a completely
different error numbering system.

The careful programmer should catch EnvironmentError, which is the base
class of all of these different kinds of related errors.

+1 on documenting that the methods may raise an EnvironmentError.


> - the doc sometimes says that when we receive an empty string from a read()
> operation, without exceptions, it means the file is empty. However, with the
> current implementation, if we call file.read(0), we simply receive "", even
> though it doesn't mean that we're at EOF. Shouldn't we avoid this (rare, I
> admit) ambiguity on the return value, by preventing read(0) ? Or at least,
> note in the doc that (we receive an empty string) <-> (the file is at EOF OR
> we called read with 0 as parameter) ?
>

Some programs may rely on read(0) and the behavior matches the behavior of
POSIX, so I'm -1 on changing the behavior.  +1 on documenting the exception
to the rule.


> Are there some arguments that I don't know, which lead to this or that
> particular implementation choice ?
>

The original I/O PEP and some of the code was put together during a sprint
at PyCon 2007.  Our primary goal was to get a decent first cut of the new
I/O system put together quickly.  For nitty-gritty details and corner-cases
like these, there's a good chance that the current undocumented behavior is
simply an accident of implementation.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090918/4634ade5/attachment.htm>

From python at mrabarnett.plus.com  Fri Sep 18 22:24:52 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 18 Sep 2009 21:24:52 +0100
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
Message-ID: <4AB3EC94.3020606@mrabarnett.plus.com>

James Y Knight wrote:
> 
> On Sep 18, 2009, at 3:55 PM, MRAB wrote:
> 
>> I think that this should be an invariant:
>>
>>    0 <= file pointer <= file size
>>
>> so the file pointer might sometimes have to be moved.
> 
>> As for the question of whether 'truncate' should be able to lengthen a
>> file, the method name suggests no; if the method name were 'resize', for
>> example, then maybe yes, zeroing the new bytes for security.
> 
> 
> Why are you just making things up? There is a *vast* amount of precedent 
> for how file operations should work. Python should follow that precedent 
> and do like POSIX unless there's a compelling reason not to. Quoting:
> 
>        If  fildes  refers  to  a  regular  file,  the ftruncate() 
> function shall cause the size of the file to be truncated to
>        length. If the size of the file previously exceeded length, the 
> extra data shall no longer be available to reads on the
>        file.  If  the  file  previously  was smaller than this size, 
> ftruncate() shall either increase the size of the file or
>        fail.   XSI-conformant systems shall increase the size of the 
> file.  If the file size is increased, the  extended  area
>        shall appear as if it were zero-filled. The value of the seek 
> pointer shall not be modified by a call to ftruncate().
> 
"making things up"? I'm just expressing an opinion!

From solipsis at pitrou.net  Fri Sep 18 22:38:51 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 18 Sep 2009 20:38:51 +0000 (UTC)
Subject: [Python-Dev] Fuzziness in io module specs
References: <4AB3DCC9.5050209@gmail.com>
Message-ID: <h90r4r$h9u$1@ger.gmane.org>

Le Fri, 18 Sep 2009 21:17:29 +0200, Pascal Chambon a ?crit?:


Hello,

First, thanks for experimenting with this.
(as a sidenote, we lack real-world testing of non-blocking features, 
perhaps you want to take a look)

> I'm currently working on a reimplementation of io.FileIO, which would 
> allow cross-platform file range locking and all kinds of other safety 
> features ;

Hmm, do you *have* to reimplement it in order to add these new features, 
or is it just a personal preference?

> - it is unclear what truncate() methods do with the file pointer, and 
> even if the current implementation simply moves it to the truncation 
> point, it's very contrary to the standard way of doing under unix, 
where 
> the file pointer is normally left unchanged. Shouldn't we specify that 
> the file pointer remains unmoved, and fix the _fileio module 
accordingly ?

Well, first Python and its IO library are cross-platform, so the 
behaviour is not always identical to POSIX behaviour, especially where 
Windows and Unix have different specs.

Second, now that 3.1 is in the wild, we should be reluctant to change the
behaviour just to make it more conformant to what POSIX people can 
expect. What might be convincing would be an actual use case where POSIX-
like behaviour would be significantly more useful than the current on.

> - exceptions are not always specified, and even if most of them are 
> IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if 
> we try to wrap a wrong file descriptor when instanciating a new 
FileIO). 

This is not different than with 2.x here. If you want to trap both 
OSError and IOError, use EnvironmentError (which is the common base class 
for both).

I agree it is slightly annoying and not well-defined, however. Also, 
Python can hardly determine by itself whether an error is caused by IO 
problems or an OS-level malfunction, so the distinction is a bit 
fallacious.

> However, with the current implementation, if we call file.read(0), we 
> simply receive "", even though it doesn't mean that we're at EOF. 
> Shouldn't we avoid this (rare, I admit) ambiguity on the return value, 
> by preventing read(0) ?

Well, if you are asking for 0 bytes, it returns 0 bytes. It's not that
ambiguous, and it helps avoid special-casing the 0 case :)

> So wouldn't it be a godo idea to write some kind of mini-pep, just to 
> fix the corner cases of the current IO documentation ?

Improvements, either to the docs or to the implementation, are always 
welcome. I think you already know where to post them!

Regards

Antoine.



From solipsis at pitrou.net  Fri Sep 18 22:40:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 18 Sep 2009 20:40:14 +0000 (UTC)
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
Message-ID: <loom.20090918T223653-896@post.gmane.org>

James Y Knight <foom <at> fuhm.net> writes:
> 
> Why are you just making things up? There is a *vast* amount of  
> precedent for how file operations should work. Python should follow  
> that precedent and do like POSIX unless there's a compelling reason  
> not to.

Actually, Python is cross-platform and therefore does not necessarily follow
POSIX behaviour, especially when it is desired to hide inconsistencies between
different platform.

(I do agree, of course, that the IO APIs are quite heavily inspired by the POSIX
APIs)

Regards

Antoine.



From stephen at xemacs.org  Sat Sep 19 02:19:53 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 19 Sep 2009 09:19:53 +0900
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <loom.20090918T223653-896@post.gmane.org>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<loom.20090918T223653-896@post.gmane.org>
Message-ID: <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:
 > James Y Knight <foom <at> fuhm.net> writes:
 > > 
 > > Why are you just making things up? There is a *vast* amount of  
 > > precedent for how file operations should work. Python should follow  
 > > that precedent and do like POSIX unless there's a compelling reason  
 > > not to.
 > 
 > Actually, Python is cross-platform and therefore does not
 > necessarily follow POSIX behaviour, especially when it is desired
 > to hide inconsistencies between different platform.

That's what *James* said, except that I prefer his standard, because I
believe POSIX documentation to be more accessible to a variety of
Python developers than other system's, and it's better documented:
rationales are included, history is available, etc.

From solipsis at pitrou.net  Sat Sep 19 02:58:08 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 19 Sep 2009 00:58:08 +0000 (UTC)
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<loom.20090918T223653-896@post.gmane.org>
	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <h91ab0$lf7$1@ger.gmane.org>

Le Sat, 19 Sep 2009 09:19:53 +0900, Stephen J. Turnbull a ?crit?:
> Antoine Pitrou writes:
>  > James Y Knight <foom <at> fuhm.net> writes:
>  > > 
>  > > Why are you just making things up? There is a *vast* amount of
>  > > precedent for how file operations should work. Python should follow
>  > > that precedent and do like POSIX unless there's a compelling reason
>  > > not to.
>  > 
>  > Actually, Python is cross-platform and therefore does not necessarily
>  > follow POSIX behaviour, especially when it is desired to hide
>  > inconsistencies between different platform.
> 
> That's what *James* said, except that I prefer his standard,

I don't believe that POSIX compliance is a sufficient argument to ask 
someone to shut up in the discussion of a cross-platform API. Which is 
more or less what James' answer was trying to do. So, no, not exactly the 
same thing that I said.

> I
> believe POSIX documentation to be more accessible to a variety of Python
> developers than other system's, and it's better documented: rationales
> are included, history is available, etc.

I'm not sure that's true. Various Unix/Linux man pages are readily 
available on the Internet, but they regard specific implementations, 
which often depart from the spec in one way or another. POSIX specs 
themselves don't seem to be easily reachable; you might even have to pay 
for them.

cheers

Antoine.


From foom at fuhm.net  Sat Sep 19 03:51:11 2009
From: foom at fuhm.net (James Y Knight)
Date: Fri, 18 Sep 2009 21:51:11 -0400
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <h91ab0$lf7$1@ger.gmane.org>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<loom.20090918T223653-896@post.gmane.org>
	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>
	<h91ab0$lf7$1@ger.gmane.org>
Message-ID: <D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>

On Sep 18, 2009, at 8:58 PM, Antoine Pitrou wrote:
> I'm not sure that's true. Various Unix/Linux man pages are readily
> available on the Internet, but they regard specific implementations,
> which often depart from the spec in one way or another. POSIX specs
> themselves don't seem to be easily reachable; you might even have to  
> pay
> for them.


The POSIX specs are quite easily accessible, without payment.

I got my quote by doing:
     man 3p ftruncate

I had previously done:
     apt-get install manpages-posix-dev
to install the posix manpages. That package contains the POSIX  
standard as of 2003. Which is good enough for most uses. It seems to  
be available here, if you don't have a debian system:
     http://www.kernel.org/pub/linux/docs/man-pages/man-pages-posix/

There's also a webpage, containing the official POSIX 2008 standard:
    http://www.opengroup.org/onlinepubs/9699919799/

And to navigate to ftruncate from there, click "System Interfaces" in  
the left pane, "System Interfaces" in the bottom pane, and then  
"ftruncate" in the bottom pane.

James

From greg.ewing at canterbury.ac.nz  Sat Sep 19 07:35:48 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 19 Sep 2009 17:35:48 +1200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB238D9.1020204@trueblade.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>
	<4AB17B29.8050708@canterbury.ac.nz>
	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>
	<4AB1A39B.7090003@canterbury.ac.nz>
	<8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com>
	<loom.20090917T134809-168@post.gmane.org> <4AB22E4F.40509@gmail.com>
	<4AB238D9.1020204@trueblade.com>
Message-ID: <4AB46DB4.2010902@canterbury.ac.nz>

Eric Smith wrote:

> My only concern with this is a possible performance issue with v6 
> networks. Would this be implemented such that [-1] doesn't need to 
> iterate through the (possibly large) address space of a v6 network?

I'm not familiar with v6, but if netmasks work the same
way as they do in v4, then there's no need to iterate
over anything -- it's just a matter of turning on all
the low bits of the address.

-- 
Greg

From stephen at xemacs.org  Sat Sep 19 09:53:00 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 19 Sep 2009 16:53:00 +0900
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <h91ab0$lf7$1@ger.gmane.org>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<loom.20090918T223653-896@post.gmane.org>
	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>
	<h91ab0$lf7$1@ger.gmane.org>
Message-ID: <87ocp7z7z7.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > I don't believe that POSIX compliance is a sufficient argument to ask 
 > someone to shut up in the discussion of a cross-platform API. Which is 
 > more or less what James' answer was trying to do.

No, as I read it, James said, "when there's precedent, even a
standard, don't make stuff up".  He then referred to the POSIX
standard, which I assume means that's a standard he likes.  But he
didn't say it had to be POSIX, or else shut up.  He said, "respect
precedent" (and implied that ignorance is no excuse, I guess, which is
a little harsh on the Internet<wink>).

And I think he'd agree to weaken his dictum to add "and if there's
good reason to vary from the standard, either quote other standards or
give rationale referring to why you want to vary from the standard."

 > I'm not sure that's true [that POSIX is readily available]. [...] 
 > POSIX specs themselves don't seem to be easily reachable; you might
 > even have to pay for them.

Five minutes with Google gives

http://standards.ieee.org/catalog/olis/arch_posix.html (IEEE members only)
http://www.unix.org/version3/ (registration but no fee required)
http://www.opengroup.org/onlinepubs/000095399/toc.htm (title page for above,
    lets you sneak past the registration, and has higher Googlefu)

Older versions also seem to be readily available.

From chambon.pascal at gmail.com  Sat Sep 19 09:46:58 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Sat, 19 Sep 2009 09:46:58 +0200
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <4AB489C9.7000908@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<loom.20090918T223653-896@post.gmane.org>	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>	<h91ab0$lf7$1@ger.gmane.org>
	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>
	<4AB489C9.7000908@gmail.com>
Message-ID: <4AB48C72.8060603@gmail.com>


@pitrou: non-blocking IO in python ? which ones are you thinking about ?
I have currently no plan to work on asynchronous IO like win32's 
readFileEx() etc. (too many troubles for the benefit), however I'd be 
interested by getting non-blocking operations on IPC pipes (I've crossed 
several people in trouble with that, having a process never end on some 
OSes because they couldn't stop threads blocked on pipes).
This reimplementation is actually necessary to get file locking, because 
advanced win32 operations only work on real file handles, not the 
handles that are underlying the C API layer. Furthermore, some 
interesting features (like O_EXCL | O_CREAT) are not possible with the 
current io implementations. So well, reimplementation required ^^

Else, allright, I'll try to summarize the various points in a PEP-update.

Concerning the "truncate" method however, on second thought I feel we 
might take distance from Posix API for naming, precisely since it's 
anyway too "platform-specific" (windows knows nothing about Posix, and 
even common unix-like systems modify it in a way or another - several 
systems don't zero-fill files when extending them).
When seeing "truncate", in my opinion, most people will think "it's only 
to reduce file size" (for beginners), or will immediately get in mind 
all the tips of posix-like systems (for more experienced developers). 
Shouldn't we, like other cross-platform APIs, use a more unambiguous 
notion, like "setLength" (java) or "resize" (Qt) ? And let the 
filepointer untouched, simply because there are no reasons to move it, 
especially when extending the file (yep, on windows we're forced to move 
the pointer, but it's easy to fix) ?
If it's too late to modify the IO API, too bad, but I don't feel 
comfortable with the "truncate" word. And I don't like the fact that we 
move the filepointer to prevent it from exceeding the file size, whereas 
on the other hand we can seek() anywhere without getting exceptions (and 
so, set the filepointer past the end of file). Having 0 <= filepointer 
<= EOF is OK to me, but then we have to enforce it for all functions, 
not just truncate.

Concerning exceptions, which one is raised is not so important to me, as 
long as it's well documented and not tricky (eg. WindowsErrors are OK to 
me, because they subclass OSError, so most cross-platform programs wont 
even have to know about them).
I had the feeling that IOErrors were for operations on file streams 
(opening, writing/reading, closing...), whereas OSErrors were for 
manipulations on filesystems (renaming, linking, stating...) and 
processes. This semantic would be perfect for me, and it's already 95% 
here, we would just have to fix some unwelcomed OSErrors exceptions in 
the iomodule. Isn't that worth it ? It'd simplify programmers' job a 
lot, and allow a more subtle treatment of exceptions (if everyone just 
catches Environment errors, without being sure of which subcless is 
actually raised, we miss the point of IOError and OSError).

Regards,
Pascal



>
>
>
> James Y Knight a ?crit :
>>
>> On Sep 18, 2009, at 8:58 PM, Antoine Pitrou wrote:
>>> I'm not sure that's true. Various Unix/Linux man pages are readily
>>> available on the Internet, but they regard specific implementations,
>>> which often depart from the spec in one way or another. POSIX specs
>>> themselves don't seem to be easily reachable; you might even have to 
>>> pay
>>> for them.
>>
>>
>> The POSIX specs are quite easily accessible, without payment.
>>
>> I got my quote by doing:
>>     man 3p ftruncate
>>
>> I had previously done:
>>     apt-get install manpages-posix-dev
>> to install the posix manpages. That package contains the POSIX 
>> standard as of 2003. Which is good enough for most uses. It seems to 
>> be available here, if you don't have a debian system:
>>     http://www.kernel.org/pub/linux/docs/man-pages/man-pages-posix/
>>
>> There's also a webpage, containing the official POSIX 2008 standard:
>>    http://www.opengroup.org/onlinepubs/9699919799/
>>
>> And to navigate to ftruncate from there, click "System Interfaces" in 
>> the left pane, "System Interfaces" in the bottom pane, and then 
>> "ftruncate" in the bottom pane.
>>
>> James
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/firephoenix%40wanadoo.fr 
>>
>>
>>
>
>


From dave.l.harrison at gmail.com  Sat Sep 19 09:59:34 2009
From: dave.l.harrison at gmail.com (David Harrison)
Date: Sat, 19 Sep 2009 17:59:34 +1000
Subject: [Python-Dev] conceptual clarity
In-Reply-To: <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>
	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>
	<4AB19F44.8010102@canterbury.ac.nz> <h8t88h$9q$1@ger.gmane.org>
	<4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com>
	<8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com>
	<20090918011725.604DE5AC0D6@longblack.object-craft.com.au>
	<8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com>
Message-ID: <a1581f70909190059m486a8d2di4025c53b748f77a4@mail.gmail.com>

2009/9/18 Peter Moody <peter at hda3.com>:
> On Thu, Sep 17, 2009 at 6:17 PM, Andrew McNamara
> <andrewm at object-craft.com.au> wrote:
>>>off to patch the pep and implement some of the non controversial changes.
>>
>> It might be a good idea to add some use-cases to the PEP.
>
> There are several use-cases in the PEP already.
>
> The problem is, for every use-case where one can show that the
> existing implementation is confusing, I can come up with a use-case
> showing where the existing implementation makes more sense than
> anything proposed.
>
> Cheers,
> /peter

For any given use-case there is bound to be an opposing one, but the
goal is to satisfy the most common or relevant use-cases that we all
experience - can't please all the people all the time etc.

I think that alot of the confusion and disagreements surrounding the
API are due to the absence of use-cases in the PEP - we can all
probably agree an IP address module has intrinsic merit, but how we
expect to use it may vary significantly.  As a result the discussion
seems to be driven by the example implementation, instead of the by
the use-cases that make the module worthwhile being added - I would
normally think it would be the other way round.  Without a clear set
of expectations and use-cases, it would also be hard to propose an
alternate implementation.

From daniel at stutzbachenterprises.com  Sat Sep 19 11:24:29 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sat, 19 Sep 2009 04:24:29 -0500
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <4AB48C72.8060603@gmail.com>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<loom.20090918T223653-896@post.gmane.org>
	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>
	<h91ab0$lf7$1@ger.gmane.org>
	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>
	<4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com>
Message-ID: <eae285400909190224g17186238w965dcaa9c8f378c@mail.gmail.com>

On Sat, Sep 19, 2009 at 2:46 AM, Pascal Chambon <chambon.pascal at gmail.com>wrote:

> This reimplementation is actually necessary to get file locking, because
> advanced win32 operations only work on real file handles, not the handles
> that are underlying the C API layer. Furthermore, some interesting features
> (like O_EXCL | O_CREAT) are not possible with the current io
> implementations. So well, reimplementation required ^^
>


> Concerning exceptions, which one is raised is not so important to me, as
> long as it's well documented and not tricky (eg. WindowsErrors are OK to me,
> because they subclass OSError, so most cross-platform programs wont even
> have to know about them).
>

If you use real Windows file handles (instead of the POSIX-ish Windows API),
won't you need to return WindowsErrors?


> I had the feeling that IOErrors were for operations on file streams
> (opening, writing/reading, closing...), whereas OSErrors were for
> manipulations on filesystems (renaming, linking, stating...) and processes.
>

If that were documented and a firm rule, that would certainly be great.
It's not too hard to find counterexamples in the current codebase.  Also,
I'm not sure how one could avoid needing to raise WindowsError in some
cases.

Maybe someone with more knowledge of the history of IOError vs. OSError
could chime in.

Python 2.6:

>>> os.write(f.fileno(), 'blah')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 9] Bad file descriptor
>>> f.write('blah')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 9] Bad file descriptor

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090919/82eef81a/attachment.htm>

From chambon.pascal at gmail.com  Sat Sep 19 12:31:15 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Sat, 19 Sep 2009 12:31:15 +0200
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <eae285400909190224g17186238w965dcaa9c8f378c@mail.gmail.com>
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<loom.20090918T223653-896@post.gmane.org>	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>	<h91ab0$lf7$1@ger.gmane.org>	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>	<4AB489C9.7000908@gmail.com>
	<4AB48C72.8060603@gmail.com>
	<eae285400909190224g17186238w965dcaa9c8f378c@mail.gmail.com>
Message-ID: <4AB4B2F3.8060207@gmail.com>

Good example with "os.write(f.fileno(), 'blah')" - and you obtain the 
same error if you try to open an io.FileIo by providing a file 
descriptor instead of a file name as first argument. This would really 
deserve an unification.

Actually, since Windows Error Codes concern any possible error (IO, file 
permissions, memory problems...), I thought the best would be to convert 
them to the most appropriate python standard exception, only defaulting 
to WindowsError (i.e, OSError's hierarchy) when no other exception type 
matches. So at the moment, I use a decorator to automatically convert 
all errors on stream operations into IOErrors. Error codes are not the 
same as unix ones indeed, but I don't know if it's really important 
(imo, most people just want to know if the operation was successful, I 
don't know if many developers scan error codes to act accordingly). For 
IOError types that really matter (eg. file already locked, buffer full), 
the easiest is actually to use subclasses of IOError (the io module 
already does that, even though I'll maybe have to create new exceptions 
for errors like "file already exists" or "file already locked by another 
process")

Regards,
Pascal

Daniel Stutzbach a ?crit :
> On Sat, Sep 19, 2009 at 2:46 AM, Pascal Chambon 
> <chambon.pascal at gmail.com <mailto:chambon.pascal at gmail.com>> wrote:
>
>     This reimplementation is actually necessary to get file locking,
>     because advanced win32 operations only work on real file handles,
>     not the handles that are underlying the C API layer. Furthermore,
>     some interesting features (like O_EXCL | O_CREAT) are not possible
>     with the current io implementations. So well, reimplementation
>     required ^^
>
>  
>
>     Concerning exceptions, which one is raised is not so important to
>     me, as long as it's well documented and not tricky (eg.
>     WindowsErrors are OK to me, because they subclass OSError, so most
>     cross-platform programs wont even have to know about them).
>
>
> If you use real Windows file handles (instead of the POSIX-ish Windows 
> API), won't you need to return WindowsErrors?
>  
>
>     I had the feeling that IOErrors were for operations on file
>     streams (opening, writing/reading, closing...), whereas OSErrors
>     were for manipulations on filesystems (renaming, linking,
>     stating...) and processes.
>
>
> If that were documented and a firm rule, that would certainly be 
> great.  It's not too hard to find counterexamples in the current 
> codebase.  Also, I'm not sure how one could avoid needing to raise 
> WindowsError in some cases.
>
> Maybe someone with more knowledge of the history of IOError vs. 
> OSError could chime in.
>
> Python 2.6:
>
> >>> os.write(f.fileno(), 'blah')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> OSError: [Errno 9] Bad file descriptor
> >>> f.write('blah')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> IOError: [Errno 9] Bad file descriptor
>
>     --
>     Daniel Stutzbach, Ph.D.
>     President, Stutzbach Enterprises, LLC
>     <http://stutzbachenterprises.com> 
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/firephoenix%40wanadoo.fr
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090919/e4f09df6/attachment.htm>

From daniel at stutzbachenterprises.com  Sat Sep 19 12:37:07 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sat, 19 Sep 2009 05:37:07 -0500
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <4AB4B2F3.8060207@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<loom.20090918T223653-896@post.gmane.org>
	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>
	<h91ab0$lf7$1@ger.gmane.org>
	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>
	<4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com>
	<eae285400909190224g17186238w965dcaa9c8f378c@mail.gmail.com>
	<4AB4B2F3.8060207@gmail.com>
Message-ID: <eae285400909190337n3d37df33lcfd7518f58e3f4b9@mail.gmail.com>

On Sat, Sep 19, 2009 at 5:31 AM, Pascal Chambon <chambon.pascal at gmail.com>wrote:

>  Actually, since Windows Error Codes concern any possible error (IO, file
> permissions, memory problems...), I thought the best would be to convert
> them to the most appropriate python standard exception, only defaulting to
> WindowsError (i.e, OSError's hierarchy) when no other exception type
> matches. So at the moment, I use a decorator to automatically convert all
> errors on stream operations into IOErrors. Error codes are not the same as
> unix ones indeed, but I don't know if it's really important (imo, most
> people just want to know if the operation was successful, I don't know if
> many developers scan error codes to act accordingly).
>

I don't often need to check the error code at runtime but seeing the
corresponding message is often critical for debugging.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090919/f68e6350/attachment.htm>

From cs at zip.com.au  Sat Sep 19 12:52:28 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 19 Sep 2009 20:52:28 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AB2AE99.8070006@gmail.com>
References: <4AB2AE99.8070006@gmail.com>
Message-ID: <20090919105228.GA10341@cskk.homeip.net>

On 18Sep2009 07:48, Nick Coghlan <ncoghlan at gmail.com> wrote:
| Eric Smith wrote:
| > Peter Moody wrote:
| >> indexing is plenty efficient, but the problem is that these names for
| >> these attributes are common to the point of causing confusion if
| >> they're omitted.
| > 
| > After thinking about it some more, I'm okay with names for [-1] and [0].
| > I like .broadcast, and I can live with .network (although it's
| > confusing, since in my view it should return an address, not a network).
| 
| And in fact *does* return an Address - the same address that my_net[0]
| returns.

Yes, I think Eric was complaining about the name being "network", since
we have Network objects and this isn't one.
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

From eric at trueblade.com  Sat Sep 19 12:54:36 2009
From: eric at trueblade.com (Eric Smith)
Date: Sat, 19 Sep 2009 06:54:36 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090919105228.GA10341@cskk.homeip.net>
References: <4AB2AE99.8070006@gmail.com>
	<20090919105228.GA10341@cskk.homeip.net>
Message-ID: <4AB4B86C.3010308@trueblade.com>

Cameron Simpson wrote:
> On 18Sep2009 07:48, Nick Coghlan <ncoghlan at gmail.com> wrote:
> | Eric Smith wrote:
> | > Peter Moody wrote:
> | >> indexing is plenty efficient, but the problem is that these names for
> | >> these attributes are common to the point of causing confusion if
> | >> they're omitted.
> | > 
> | > After thinking about it some more, I'm okay with names for [-1] and [0].
> | > I like .broadcast, and I can live with .network (although it's
> | > confusing, since in my view it should return an address, not a network).
> | 
> | And in fact *does* return an Address - the same address that my_net[0]
> | returns.
> 
> Yes, I think Eric was complaining about the name being "network", since
> we have Network objects and this isn't one.

Right. thing_commonly_referred_to_as_network_but_is_an_address.


From pythoniks at gmail.com  Sat Sep 19 09:35:37 2009
From: pythoniks at gmail.com (Pascal Chambon)
Date: Sat, 19 Sep 2009 09:35:37 +0200
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<loom.20090918T223653-896@post.gmane.org>	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>	<h91ab0$lf7$1@ger.gmane.org>
	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>
Message-ID: <4AB489C9.7000908@gmail.com>


@pitrou: non-blocking IO in python ? which ones are you thinking about ?
I have currently no plan to work on asynchronous IO like win32's 
readFileEx() etc. (too many troubles for the benefit), however I'd be 
interested by getting non-blocking operations on IPC pipes (I've crossed 
several people in trouble with that, having a process never end on some 
OSes because they couldn't stop threads blocked on pipes).
This reimplementation is actually necessary to get file locking, because 
advanced win32 operations only work on real file handles, not the 
handles that are underlying the C API layer. Furthermore, some 
interesting features (like O_EXCL | O_CREAT) are not possible with the 
current io implementations. So well, reimplementation required ^^

Else, allright, I'll try to summarize the various points in a PEP-update.

Concerning the "truncate" method however, on second thought I feel we 
might take distance from Posix API for naming, precisely since it's 
anyway too "platform-specific" (windows knows nothing about Posix, and 
even common unix-like systems modify it in a way or another - several 
systems don't zero-fill files when extending them).
When seeing "truncate", in my opinion, most people will think "it's only 
to reduce file size" (for beginners), or will immediately get in mind 
all the tips of posix-like systems (for more experienced developers). 
Shouldn't we, like other cross-platform APIs, use a more unambiguous 
notion, like "setLength" (java) or "resize" (Qt) ? And let the 
filepointer untouched, simply because there are no reasons to move it, 
especially when extending the file (yep, on windows we're forced to move 
the pointer, but it's easy to fix) ?
If it's too late to modify the IO API, too bad, but I don't feel 
comfortable with the "truncate" word. And I don't like the fact that we 
move the filepointer to prevent it from exceeding the file size, whereas 
on the other hand we can seek() anywhere without getting exceptions (and 
so, set the filepointer past the end of file). Having 0 <= filepointer 
<= EOF is OK to me, but then we have to enforce it for all functions, 
not just truncate.

Concerning exceptions, which one is raised is not so important to me, as 
long as it's well documented and not tricky (eg. WindowsErrors are OK to 
me, because they subclass OSError, so most cross-platform programs wont 
even have to know about them).
I had the feeling that IOErrors were for operations on file streams 
(opening, writing/reading, closing...), whereas OSErrors were for 
manipulations on filesystems (renaming, linking, stating...) and 
processes. This semantic would be perfect for me, and it's already 95% 
here, we would just have to fix some unwelcomed OSErrors exceptions in 
the iomodule. Isn't that worth it ? It'd simplify programmers' job a 
lot, and allow a more subtle treatment of exceptions (if everyone just 
catches Environment errors, without being sure of which subcless is 
actually raised, we miss the point of IOError and OSError).

Regards,
Pascal

James Y Knight a ?crit :
>
> On Sep 18, 2009, at 8:58 PM, Antoine Pitrou wrote:
>> I'm not sure that's true. Various Unix/Linux man pages are readily
>> available on the Internet, but they regard specific implementations,
>> which often depart from the spec in one way or another. POSIX specs
>> themselves don't seem to be easily reachable; you might even have to pay
>> for them.
>
>
> The POSIX specs are quite easily accessible, without payment.
>
> I got my quote by doing:
>     man 3p ftruncate
>
> I had previously done:
>     apt-get install manpages-posix-dev
> to install the posix manpages. That package contains the POSIX 
> standard as of 2003. Which is good enough for most uses. It seems to 
> be available here, if you don't have a debian system:
>     http://www.kernel.org/pub/linux/docs/man-pages/man-pages-posix/
>
> There's also a webpage, containing the official POSIX 2008 standard:
>    http://www.opengroup.org/onlinepubs/9699919799/
>
> And to navigate to ftruncate from there, click "System Interfaces" in 
> the left pane, "System Interfaces" in the bottom pane, and then 
> "ftruncate" in the bottom pane.
>
> James
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/firephoenix%40wanadoo.fr 
>
>
>


From martin at v.loewis.de  Sat Sep 19 13:09:40 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 19 Sep 2009 13:09:40 +0200
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <h91ab0$lf7$1@ger.gmane.org>
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<loom.20090918T223653-896@post.gmane.org>	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>
	<h91ab0$lf7$1@ger.gmane.org>
Message-ID: <4AB4BBF4.4070509@v.loewis.de>

>> I
>> believe POSIX documentation to be more accessible to a variety of Python
>> developers than other system's, and it's better documented: rationales
>> are included, history is available, etc.
> 
> I'm not sure that's true. Various Unix/Linux man pages are readily 
> available on the Internet, but they regard specific implementations, 
> which often depart from the spec in one way or another. POSIX specs 
> themselves don't seem to be easily reachable; you might even have to pay 
> for them.

Then please rest assured that it actually *is* true:
- the Linux man pages are often a literal copy of the POSIX man pages,
  so when you look at a Linux man page, there is a good chance that it
  either is the wording of the POSIX spec, or points out what details
  are specific to POSIX.
- The Open Group publishes POSIX free of charge, and these days also
  free of registration.
- these days, specific implementations typically do *not* deviate from
  POSIX. Some provide additional features, but in a way that does not
  harm compatibility.

Regards,
Martin

From rdmurray at bitdance.com  Sat Sep 19 14:47:57 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 19 Sep 2009 08:47:57 -0400 (EDT)
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <4AB4B2F3.8060207@gmail.com>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<loom.20090918T223653-896@post.gmane.org>
	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <h91ab0$lf7$1@ger.gmane.org>
	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>
	<4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com>
	<eae285400909190224g17186238w965dcaa9c8f378c@mail.gmail.com>
	<4AB4B2F3.8060207@gmail.com>
Message-ID: <Pine.LNX.4.64.0909190841581.17020@kimball.webabinitio.net>

On Sat, 19 Sep 2009 at 12:31, Pascal Chambon wrote:
> stream operations into IOErrors. Error codes are not the same as unix ones 
> indeed, but I don't know if it's really important (imo, most people just want 
> to know if the operation was successful, I don't know if many developers scan 
> error codes to act accordingly). For IOError types that really matter (eg.

Doesn't matter if it isn't very many, I think, just that it can be done.
But I suspect it is fairly common.  I know I have inspected OSError codes
(though I can't remember if I've done it for file operations).

--David

From ncoghlan at gmail.com  Sat Sep 19 15:01:23 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Sep 2009 23:01:23 +1000
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <4AB4B2F3.8060207@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<loom.20090918T223653-896@post.gmane.org>	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>	<h91ab0$lf7$1@ger.gmane.org>	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>	<4AB489C9.7000908@gmail.com>	<4AB48C72.8060603@gmail.com>	<eae285400909190224g17186238w965dcaa9c8f378c@mail.gmail.com>
	<4AB4B2F3.8060207@gmail.com>
Message-ID: <4AB4D623.8050405@gmail.com>

Pascal Chambon wrote:
> (imo, most people just want to know if the operation was successful, I
> don't know if many developers scan error codes to act accordingly).

And as a user of applications written by those developers, it is a
practice I detest with a passion. Debugging environmental problems is
painful enough without further encouraging applications that lie to
their users as to what has actually gone wrong.

For example, a file not existing, a file being locked by another
process, and the user not having write permissions to the file are
problems that demand very different responses from the user.
Applications that give the same error message for all three problems are
far too common.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Sat Sep 19 19:04:13 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 19 Sep 2009 17:04:13 +0000 (UTC)
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<loom.20090918T223653-896@post.gmane.org>	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>	<h91ab0$lf7$1@ger.gmane.org>
	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>
	<4AB489C9.7000908@gmail.com>
Message-ID: <loom.20090919T185833-755@post.gmane.org>


Hello,

Pascal Chambon <pythoniks <at> gmail.com> writes:
> @pitrou: non-blocking IO in python ? which ones are you thinking about ?

I was talking about the existing support for non-blocking IO in the FileIO class
(look up EAGAIN in fileio.c), as well as in the Buffered* objects.

> If it's too late to modify the IO API, too bad, but I don't feel 
> comfortable with the "truncate" word.

It's certainly too late to modify the IO API only for naming purposes.

> And I don't like the fact that we 
> move the filepointer to prevent it from exceeding the file size,

I don't see what you mean:

>>> with open('foobar', 'wb') as f:
...   f.truncate(0)
... 
0
>>> os.stat('foobar').st_size
0
>>> with open('foobar', 'wb') as f:
...   f.truncate(16)
...   f.tell()
... 
16
16
>>> os.stat('foobar').st_size
16

> I had the feeling that IOErrors were for operations on file streams 
> (opening, writing/reading, closing...), whereas OSErrors were for 
> manipulations on filesystems (renaming, linking, stating...) and 
> processes.

Ok, but the distinction is certainly fuzzy in many cases. I have no problem with
trying to change the corner cases you mention, though.

Regards

Antoine.



From chambon.pascal at gmail.com  Sat Sep 19 19:43:31 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Sat, 19 Sep 2009 19:43:31 +0200
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <loom.20090919T185833-755@post.gmane.org>
References: <4AB3DCC9.5050209@gmail.com>	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<loom.20090918T223653-896@post.gmane.org>	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>	<h91ab0$lf7$1@ger.gmane.org>	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>	<4AB489C9.7000908@gmail.com>
	<loom.20090919T185833-755@post.gmane.org>
Message-ID: <4AB51843.1030603@gmail.com>




Antoine Pitrou a ?crit :
> Hello,
>
> Pascal Chambon <pythoniks <at> gmail.com> writes:
>   
>> @pitrou: non-blocking IO in python ? which ones are you thinking about ?
>>     
>
> I was talking about the existing support for non-blocking IO in the FileIO class
> (look up EAGAIN in fileio.c), as well as in the Buffered* objects.
>
>   

Allright, I'll check that EAGAIN stuff, that I hadn't even noticed  :)



>> And I don't like the fact that we 
>> move the filepointer to prevent it from exceeding the file size,
>>     
>
> I don't see what you mean:
>
>   

Well the sample code you showed is not shocking, but I'd like to have a 
coherency the with file.seek(), because if truncate() prevents 
out-of-bound file pointer, other methods should do the same as well (and 
raise IOError when seeking out of file bounds).



>> I had the feeling that IOErrors were for operations on file streams 
>> (opening, writing/reading, closing...), whereas OSErrors were for 
>> manipulations on filesystems (renaming, linking, stating...) and 
>> processes.
>>     
>
> Ok, but the distinction is certainly fuzzy in many cases. I have no problem with
> trying to change the corner cases you mention, though.
>   
The case which could be problematic there is the file opening, because 
it can involve problems at all levels of the OS (filesystem not 
existing, permission problems, file locking...), so we should keep it in 
the "EnvironmentError" area.
But as soon as a file is open, I guess only IOErrors can be involved (no 
space left, range locked etc), so enforcing all this to raise IOError 
would be OK I think.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090919/9589902d/attachment.htm>

From tmm at fastmail.fm  Sat Sep 19 20:51:33 2009
From: tmm at fastmail.fm (qwavel)
Date: Sat, 19 Sep 2009 11:51:33 -0700 (PDT)
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-C736AE.09230710092009@ger.gmane.org>
	<F10025B7-8AEA-4FD2-88C4-231F8BD77458@mac.com>
	<B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>
Message-ID: <25524412.post@talk.nabble.com>



Barry Warsaw wrote:
> 
> On Sep 15, 2009, at 9:53 AM, Ronald Oussoren wrote:
> 
>> The IDLE issue is IMHO a release blocker, as is issue 6851.
> 
> So that leaves 4 release blockers for 2.6.3.
> 

What about this bug:
http://bugs.python.org/issue3890
It appears to me that the SSL module is broken in the 2.6.x line on all
platforms in one of its most common uses (non-blocking).  It also seems that
the problem and solution are well understood, so the solution would make a
good candidate for 2.6.3?
Tom.




-- 
View this message in context: http://www.nabble.com/Python-2.6.3-tp25364517p25524412.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From janssen at parc.com  Sat Sep 19 22:40:59 2009
From: janssen at parc.com (Bill Janssen)
Date: Sat, 19 Sep 2009 13:40:59 PDT
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <25524412.post@talk.nabble.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-C736AE.09230710092009@ger.gmane.org>
	<F10025B7-8AEA-4FD2-88C4-231F8BD77458@mac.com>
	<B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>
	<25524412.post@talk.nabble.com>
Message-ID: <18464.1253392859@parc.com>

qwavel <tmm at fastmail.fm> wrote:

> What about this bug:
> http://bugs.python.org/issue3890
> It appears to me that the SSL module is broken in the 2.6.x line on all
> platforms in one of its most common uses (non-blocking).  It also seems that
> the problem and solution are well understood, so the solution would make a
> good candidate for 2.6.3?
> Tom.

I might quibble about "most common uses", but I agree this would be a good idea.

Bill

From steve at pearwood.info  Sat Sep 19 23:55:03 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 20 Sep 2009 07:55:03 +1000
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <4AB4B2F3.8060207@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>
	<eae285400909190224g17186238w965dcaa9c8f378c@mail.gmail.com>
	<4AB4B2F3.8060207@gmail.com>
Message-ID: <200909200755.04172.steve@pearwood.info>

On Sat, 19 Sep 2009 08:31:15 pm Pascal Chambon wrote:
> Error codes are not the
> same as unix ones indeed, but I don't know if it's really important
> (imo, most people just want to know if the operation was successful,
> I don't know if many developers scan error codes to act accordingly).

I do. Please don't throw away useful information.


-- 
Steven D'Aprano

From greg.ewing at canterbury.ac.nz  Sun Sep 20 01:43:13 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 20 Sep 2009 11:43:13 +1200
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <4AB48C72.8060603@gmail.com>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<loom.20090918T223653-896@post.gmane.org>
	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <h91ab0$lf7$1@ger.gmane.org>
	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>
	<4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com>
Message-ID: <4AB56C91.2080005@canterbury.ac.nz>

Pascal Chambon wrote:
> 
> And let the 
> filepointer untouched, simply because there are no reasons to move it,

On some systems it may not be possible for the file pointer to
be positioned past the end of the file (I think Classic MacOS
was like that).

> I had the feeling that IOErrors were for operations on file streams 
> (opening, writing/reading, closing...), whereas OSErrors were for 
> manipulations on filesystems (renaming, linking, stating...)

I always thought the distinction was that IOError contains
a C library errno value, whereas OSError contains an
OS-specific error code. So system calls that aren't part
of the C stdlib need to use OSError, at least on some
platforms.

I don't see that file errors vs. everything else is
a very useful distinction to make when catching exceptions.
I almost always end up catching EnvironmentError to make
sure I get both, particularly when working cross-platform.

What we could do with is better platform-independent
ways of distinguishing particular error conditions,
such as file not found, out of space, etc., either
using subclasses of IOError or mapping error codes
to a set of platform-independent ones.

-- 
Greg

From steve at pearwood.info  Sun Sep 20 02:07:11 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 20 Sep 2009 10:07:11 +1000
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
Message-ID: <200909201007.11923.steve@pearwood.info>

On Sat, 19 Sep 2009 06:08:23 am James Y Knight wrote:
> On Sep 18, 2009, at 3:55 PM, MRAB wrote:
> > I think that this should be an invariant:
> >
> >    0 <= file pointer <= file size
> >
> > so the file pointer might sometimes have to be moved.
> >
> > As for the question of whether 'truncate' should be able to
> > lengthen a file, the method name suggests no; if the method name
> > were 'resize', for
> > example, then maybe yes, zeroing the new bytes for security.
>
> Why are you just making things up? 

Well, why not? The POSIX standard wasn't derived from the physical laws 
of the universe, somebody -- or more likely, some committee -- made 
them up. It's not like we are forced to accept MRAB's suggestions, but 
surely he's allowed to make them?

> There is a *vast* amount of 
> precedent for how file operations should work. Python should follow
> that precedent and do like POSIX unless there's a compelling reason
> not to. Quoting:
>
>         If  fildes  refers  to  a  regular  file,  the ftruncate()
> function shall cause the size of the file to be truncated to
>         length. If the size of the file previously exceeded length,
> the extra data shall no longer be available to reads on the
>         file.  If  the  file  previously  was smaller than this size,
> ftruncate() shall either increase the size of the file or
>         fail.   XSI-conformant systems shall increase the size of the
> file.  If the file size is increased, the  extended  area
>         shall appear as if it were zero-filled. The value of the seek
> pointer shall not be modified by a call to ftruncate().

Standard or not, it has a seriously misleading name. Truncate means "to 
cut off", and is a standard English word and mathematical term. In 
English and mathematics, truncating something can NEVER lead it to 
increase in size, the object being truncated can only decrease or 
remain the same size.

Using "truncate" to mean "increase in size" makes about as much sense as 
having a list method called "remove" used to insert items. I can't 
imagine what the committee who approved this were thinking.

Java's FileChannel object uses truncate to only shrink a file, never 
increase it. To increase it, you have to explicitly write to the file.

http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/FileChannel.html#truncate(long)

In any case, some platforms don't allow the file pointer to exceed the 
size of the file, so it's not clear that the POSIX standard is 
appropriate for Python.



-- 
Steven D'Aprano

From greg.ewing at canterbury.ac.nz  Sun Sep 20 02:23:20 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 20 Sep 2009 12:23:20 +1200
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <4AB4D623.8050405@gmail.com>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<loom.20090918T223653-896@post.gmane.org>
	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <h91ab0$lf7$1@ger.gmane.org>
	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>
	<4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com>
	<eae285400909190224g17186238w965dcaa9c8f378c@mail.gmail.com>
	<4AB4B2F3.8060207@gmail.com> <4AB4D623.8050405@gmail.com>
Message-ID: <4AB575F8.9040107@canterbury.ac.nz>

Nick Coghlan wrote:

> For example, a file not existing, a file being locked by another
> process, and the user not having write permissions to the file are
> problems that demand very different responses from the user.

You can display an error-specific message without having
to inspect the error code, e.g.

   try:
     something_with_file(path)
   except EnvironmentError, e:
     report_error("Couldn't do that with %s: %s" % (path, e))

This is a pattern I use a lot, and it seems to work
pretty well.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sun Sep 20 03:11:53 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 20 Sep 2009 13:11:53 +1200
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <200909201007.11923.steve@pearwood.info>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<200909201007.11923.steve@pearwood.info>
Message-ID: <4AB58159.3040303@canterbury.ac.nz>

Steven D'Aprano wrote:

> Using "truncate" to mean "increase in size" makes about as much sense as 
> having a list method called "remove" used to insert items. I can't 
> imagine what the committee who approved this were thinking.

I expect the reason is historical. Some time back in the
early days of Unix, someone wanted a way of chopping back
files, so they added a truncate() system call. Then someone
else noticed that it would happily accept an argument greater
than the existing length, and it seemed like that could
be useful behaviour, so they documented it and left it that way.

Then the POSIX committee came along and incorporated it into
the standard so as to be compatible with existing practice.

-- 
Greg

From chambon.pascal at gmail.com  Sun Sep 20 09:16:27 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Sun, 20 Sep 2009 09:16:27 +0200
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <4AB58159.3040303@canterbury.ac.nz>
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<200909201007.11923.steve@pearwood.info>
	<4AB58159.3040303@canterbury.ac.nz>
Message-ID: <4AB5D6CB.6050604@gmail.com>

Well, system compatibility argues strongl in favor of not letting 
filepointer > EOF.
However, is that really necessary to move the pointer to EOF in ANY case 
? I mean, if I extend the file, or if I reduce it without going lower 
than my current filepointer, I really don't expect at all the io system 
to move my pointer to the end of file, "just for fun". In these 
patterns, people would have to remember their current filepointer, to 
come back to where they were, and that's not pretty imo...

If we agree on the simple mandatory expression 0 <= filepointer <= EOF 
(for cross-platform safety), then we just have to enforce it when the 
rule is broken : reducing the size lower than the filepointer, and 
seeking past the end of file. All other conditions should leav the 
filepointer where the user put it. Shouldnt it be so ?

   
Concerning the naming of truncate(), would it be possible to deprecate 
it and alias it to "resize()" ? It's not very gratifying to have 
duplicated methods at the beginning of a major release, but I feel too 
that "truncate" is a misleading term, that had better be replaced asap.

Regards,
Pascal

From chambon.pascal at gmail.com  Sun Sep 20 09:24:16 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Sun, 20 Sep 2009 09:24:16 +0200
Subject: [Python-Dev] POSIX [Fuzziness in io module specs]
In-Reply-To: <4AB56C91.2080005@canterbury.ac.nz>
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<loom.20090918T223653-896@post.gmane.org>	<87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp>
	<h91ab0$lf7$1@ger.gmane.org>	<D6F6BC27-8EBC-4FC0-B76A-AC45B273FB74@fuhm.net>	<4AB489C9.7000908@gmail.com>
	<4AB48C72.8060603@gmail.com> <4AB56C91.2080005@canterbury.ac.nz>
Message-ID: <4AB5D8A0.1060202@gmail.com>


>
> What we could do with is better platform-independent
> ways of distinguishing particular error conditions,
> such as file not found, out of space, etc., either
> using subclasses of IOError or mapping error codes
> to a set of platform-independent ones.
>

Well, mapping all errors (including C ones and windows-specific ones) to 
a common set would be extremely useful for developers indeed.
I guess some advanced windows errors will never have equivalents 
elsewhere, but does anyone know an error code set which would be 
relevant to cover all memorty, filesystem, io and locking aspects ?


Regards,
Pascal

From greg.ewing at canterbury.ac.nz  Sun Sep 20 10:33:33 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 20 Sep 2009 20:33:33 +1200
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <4AB5D6CB.6050604@gmail.com>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<200909201007.11923.steve@pearwood.info>
	<4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com>
Message-ID: <4AB5E8DD.6050204@canterbury.ac.nz>

Pascal Chambon wrote:

>   Concerning the naming of truncate(), would it be possible to deprecate 
> it and alias it to "resize()" ? It's not very gratifying to have 
> duplicated methods at the beginning of a major release, but I feel too 
> that "truncate" is a misleading term, that had better be replaced asap.

There's something to be said for that, but there's also
something to be said for following established conventions,
and there's a long-established precedent from the C library
for having a function called truncate() that behaves this
way.

-- 
Greg

From chambon.pascal at gmail.com  Sun Sep 20 11:48:49 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Sun, 20 Sep 2009 11:48:49 +0200
Subject: [Python-Dev] Fuzziness in io module specs - PEP update
	proposition
In-Reply-To: <4AB5E8DD.6050204@canterbury.ac.nz>
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<200909201007.11923.steve@pearwood.info>	<4AB58159.3040303@canterbury.ac.nz>
	<4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz>
Message-ID: <4AB5FA81.5060408@gmail.com>

Hello

After weighing up here and that, here is what I have come with. Comments 
and issue notifications more than welcome, of course. The exception 
thingy is not yet addressed.

Regards,
Pascal


*Truncate and file pointer semantics*

Rationale :

The current implementation of truncate() always move the file pointer to 
the new end of file.

This behaviour is interesting for compatibility, if the file has been 
reduced and the file pointer is now past its end, since some platforms 
might require 0 <= filepointer <= filesize.

However, there are several arguments against this semantic:

    * Most common standards (posix, win32...) allow the file pointer to
      be past the end of file, and define the behaviour of other stream
      methods in this case
    * In many cases, moving the filepointer when truncating has no
      reasons to happen (if we're extending the file, or reducing it
      without going beneath the file pointer)
    * Making 0 <= filepointer <= filesize a global rule of the python IO
      module doesn't seems possible, since it would require
      modifications of the semantic of other methods (eg. seek() should
      raise exceptions or silently disobey when asked to move the
      filepointer past the end of file), and lead to incoherent
      situations when concurrently accessing files without locking (what
      if another process truncates to 0 bytes the file you're writing ?)

So here is the proposed semantic, which matches established conventions:

*RawIOBase.truncate(n: int = None) -> int*

*(same for BufferedIOBase.truncate(pos: int = None) -> int)*

Resizes the file to the size specified by the positive integer n, or by 
the current filepointer position if n is None.

The file must be opened with write permissions.

If the file was previously larger than n, the extra data is discarded. 
If the file was previously shorter than n, its size is increased, and 
the extended area appears as if it were zero-filled.

In any case, the file pointer is left unchanged, and may point beyond 
the end of file.

Note: trying to read past the end of file returns an empty string, and 
trying to write past the end of file extends it by zero-ing the gap. On 
rare platforms which don't support file pointers to be beyond the end of 
file, all these behaviours shall be faked thanks to internal storage of 
the "wanted" file pointer position (silently extending the file, if 
necessary, when a write operation occurs).

 

*Proposition of doc update*

*RawIOBase*.read(n: int) -> bytes

Read up to n bytes from the object and return them. Fewer than n bytes 
may be returned if the operating system call returns fewer than n bytes. 
If 0 bytes are returned, and n was not 0, this indicates end of file. If 
the object is in non-blocking mode and no bytes are available, the call 
returns None.

*RawIOBase*.readinto(b: bytes) -> int

Read up to len(b) bytes from the object and stores them in b, returning 
the number of bytes read. Like .read, fewer than len(b) bytes may be 
read, and 0 indicates end of file if b is not 0. None is returned if a 
non-blocking object has no bytes available. The length of b is never 
changed.

 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090920/91f4c305/attachment.htm>

From python at mrabarnett.plus.com  Sun Sep 20 14:57:45 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 20 Sep 2009 13:57:45 +0100
Subject: [Python-Dev] Fuzziness in io module specs - PEP
	update	proposition
In-Reply-To: <4AB5FA81.5060408@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<200909201007.11923.steve@pearwood.info>	<4AB58159.3040303@canterbury.ac.nz>	<4AB5D6CB.6050604@gmail.com>
	<4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com>
Message-ID: <4AB626C9.1010304@mrabarnett.plus.com>

Pascal Chambon wrote:
> Hello
> 
> After weighing up here and that, here is what I have come with. Comments 
> and issue notifications more than welcome, of course. The exception 
> thingy is not yet addressed.
> 
> Regards,
> Pascal
> 
> 
> *Truncate and file pointer semantics*
> 
> Rationale :
> 
> The current implementation of truncate() always move the file pointer to 
> the new end of file.
> 
> This behaviour is interesting for compatibility, if the file has been 
> reduced and the file pointer is now past its end, since some platforms 
> might require 0 <= filepointer <= filesize.
> 
> However, there are several arguments against this semantic:
> 
>     * Most common standards (posix, win32?) allow the file pointer to be
>       past the end of file, and define the behaviour of other stream
>       methods in this case
>     * In many cases, moving the filepointer when truncating has no
>       reasons to happen (if we?re extending the file, or reducing it
>       without going beneath the file pointer)
>     * Making 0 <= filepointer <= filesize a global rule of the python IO
>       module doesn?t seems possible, since it would require
>       modifications of the semantic of other methods (eg. seek() should
>       raise exceptions or silently disobey when asked to move the
>       filepointer past the end of file), and lead to incoherent
>       situations when concurrently accessing files without locking (what
>       if another process truncates to 0 bytes the file you?re writing ?)
> 
> So here is the proposed semantic, which matches established conventions:
> 
> *RawIOBase.truncate(n: int = None) -> int*
> 
> *(same for BufferedIOBase.truncate(pos: int = None) -> int)*
> 
> Resizes the file to the size specified by the positive integer n, or by 
> the current filepointer position if n is None.
> 
The new size could be positive or zero.

> The file must be opened with write permissions.
> 
> If the file was previously larger than n, the extra data is discarded. 
> If the file was previously shorter than n, its size is increased, and 
> the extended area appears as if it were zero-filled.
> 
> In any case, the file pointer is left unchanged, and may point beyond 
> the end of file.
> 
> Note: trying to read past the end of file returns an empty string, and 
> trying to write past the end of file extends it by zero-ing the gap. On 
> rare platforms which don?t support file pointers to be beyond the end of 
> file, all these behaviours shall be faked thanks to internal storage of 
> the ?wanted? file pointer position (silently extending the file, if 
> necessary, when a write operation occurs).
> 
>  
> 
> *Proposition of doc update*
> 
> *RawIOBase*.read(n: int) -> bytes
> 
> Read up to n bytes from the object and return them. Fewer than n bytes 
> may be returned if the operating system call returns fewer than n bytes. 
> If 0 bytes are returned, and n was not 0, this indicates end of file. If 
> the object is in non-blocking mode and no bytes are available, the call 
> returns None.
> 
> *RawIOBase*.readinto(b: bytes) -> int
> 
> Read up to len(b) bytes from the object and stores them in b, returning 
> the number of bytes read. Like .read, fewer than len(b) bytes may be 
> read, and 0 indicates end of file if b is not 0. None is returned if a 
> non-blocking object has no bytes available. The length of b is never 
> changed.
> 
I thought 'bytes' was immutable!

If you're going to read into a list or array, it would be nice to also
be able to give the start index and either the end index or the count
(start defaults to 0, end defaults to len).

From python at mrabarnett.plus.com  Sun Sep 20 14:59:20 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 20 Sep 2009 13:59:20 +0100
Subject: [Python-Dev] Fuzziness in io module specs
In-Reply-To: <4AB5E8DD.6050204@canterbury.ac.nz>
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<200909201007.11923.steve@pearwood.info>	<4AB58159.3040303@canterbury.ac.nz>
	<4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz>
Message-ID: <4AB62728.5080408@mrabarnett.plus.com>

Greg Ewing wrote:
> Pascal Chambon wrote:
> 
>>   Concerning the naming of truncate(), would it be possible to 
>> deprecate it and alias it to "resize()" ? It's not very gratifying to 
>> have duplicated methods at the beginning of a major release, but I 
>> feel too that "truncate" is a misleading term, that had better be 
>> replaced asap.
> 
> There's something to be said for that, but there's also
> something to be said for following established conventions,
> and there's a long-established precedent from the C library
> for having a function called truncate() that behaves this
> way.
> 
But Python isn't C. :-)


From solipsis at pitrou.net  Sun Sep 20 15:08:07 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 20 Sep 2009 13:08:07 +0000 (UTC)
Subject: [Python-Dev] Fuzziness in io module specs - PEP
	update	proposition
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<200909201007.11923.steve@pearwood.info>
	<4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com>
	<4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com>
Message-ID: <h959fn$66p$1@ger.gmane.org>


Hello,

> *Truncate and file pointer semantics*
[snip]

The major problem here is that you are changing the current semantics.
I don't think the argument you are making for it is strong enough to 
counter-balance the backwards compatibility issue. The situation would be 
different if 3.0 hadn't been released yet.

Besides, we already broke compatibility with 3.0/3.1, let's not give 
users the impression that we don't care about compatibility anymore.

Regards

Antoine.


From zookog at gmail.com  Sun Sep 20 16:02:09 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Sun, 20 Sep 2009 08:02:09 -0600
Subject: [Python-Dev] please consider changing --enable-unicode default to
	ucs4
Message-ID: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>

Dear Pythonistas:

This issue causes serious problems.  Users occasionally get binaries built for a
compatible Linux and Python version but with a different UCS2-vs-UCS4 setting,
and those users get mysterious memory corruption errors which are hard to
diagnose.  It is possible that these situations also open up security
vulnerabilities.  A couple such instances are documented on
http://bugs.python.org/setuptools/issue78, but you can find more by googling.
I would like to get this problem fixed!

In order to help address this issue I sampled what UCS size is used by python
executables in the wild.  I instrumented a few buildslaves that are
contributed by
various people to the Tahoe-LAFS project to print out their platform,
python version,
and sys.maxunicode.  The full results are appended below.  maxunicode: 1114111
means that python executable was configured with --enable-unicode=ucs4, and
maxunicode: 65535 means that python executable was configured with
--enable-unicode=ucs2 or just with --enable-unicode .  The only
incompatibilities
that I found are because some packagers have deliberately set UCS4
configuration and other packagers have left the default setting.

In the three cases where someone configured python with UCS2, one of the three
is certainly an accident (a custom-built python executable on an Ubuntu server)
and the other two just use the default instead of specifically configuring ucs2
in their configurations of Python and I suspect that they don't know the
difference and that it was an accident that they built a Python incompatible
with other distributions of their operating system.

In sum, while it would be good to add the unicode setting to the platform's ABI
(as discussed in setuptools ticket #78), it would also be good to make
the default
value be UCS4 instead of UCS2.  This would fix all three of the potential
incompatibilities that I found (listed below), and once we have proper inclusion
of the unicode setting in the ABI in order to prevent the memory corruption,
defaulting to UCS4 would increase the likelihood that a binary built on one
distribution would be usable on another.

I'm sure that someone can come up with a reason why UCS2 is better than UCS4,
but I'm also sure that the benefits of compatibility outweigh any benefits of
UCS2 encoding, and that the widespread use of UCS4 demonstrates that there is
nothing fatally wrong with it, and that people who really value UCS2 encoding
more than compatibility can choose that for themselves by explicitly
setting UCS2.

Let me restate that I am not suggesting taking away anyone's options, only
making the setting for people who don't specify default to the
compatible option.
Hm, I guess that means that it should default to UCS2 on Windows and Mac and
to UCS4 on Linux and Solaris.

Regards,

Zooko

Ubuntu 6.10 "edgy" i386: python: 2.4.4c1 (#2, Mar  7 2008, 03:03:38)  [GCC 4.1.2
20060928 (prerelease) (Ubuntu 4.1.1-13ubuntu5)], maxunicode: 1114111
Ubuntu 7.04 "feisty": python: 2.5.1 (r251:54863, Jul 31 2008, 22:53:39)  [GCC
4.1.2 (Ubuntu 4.1.2-0ubuntu4)], maxunicode: 1114111
Ubuntu 7.10 "gutsy" i386: python: 2.5.1 (r251:54863, Jul 31 2008, 23:17:40)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)], maxunicode: 1114111
Ubuntu 8.04 "hardy" amd64: python: 2.5.2 (r252:60911, Jul 22 2009, 15:33:10)
[GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)], maxunicode: 1114111
Ubuntu 8.04 "hardy" i386: *custom* python: 2.6 (r26:66714, Oct  2 2008,
13:40:28)  [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)], maxunicode: 65535
Ubuntu 8.04 "hardy" i386: python: 2.5.2 (r252:60911, Jul 22 2009, 15:35:03)
[GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)], maxunicode: 1114111
Ubuntu 9.04 "jaunty" amd64: *custom* python: 2.6.2 (release26-maint, Apr 19
2009, 01:58:18)  [GCC 4.3.3], maxunicode: 1114111

Debian 4.0 "etch" i386: python: 2.4.4 (#2, Oct 22 2008, 19:52:44)  [GCC 4.1.2
20061115 (prerelease) (Debian 4.1.1-21)], maxunicode: 1114111
Debian 5.0 "lenny" i386: python: 2.5.2 (r252:60911, Jan  4 2009, 17:40:26)  [GCC
4.3.2], maxunicode: 1114111
Debian 5.0 "lenny" amd64: python: 2.5.2 (r252:60911, Jan  4 2009, 21:59:32)
[GCC 4.3.2], maxunicode: 1114111
Debian 5.0 "lenny" armv5tel: python: 2.5.2 (r252:60911, Jan  5 2009, 02:00:00)
[GCC 4.3.2], maxunicode: 1114111
Debian unstable "squeeze/sid" i386: python: 2.5.4 (r254:67916, Feb 17 2009,
20:16:45)  [GCC 4.3.3], maxunicode: 1114111

Fedora 11 "leonidas" amd64: python: 2.6 (r26:66714, Jul  4 2009, 17:37:13)  [GCC
4.4.0 20090506 (Red Hat 4.4.0-4)], maxunicode: 1114111

ArchLinux: python: 2.6.2 (r262:71600, Jul 20 2009, 02:23:30)  [GCC 4.4.0
20090630 (prerelease)], maxunicode: 65535

NetBSD 4: python: 2.5.2 (r252:60911, Mar 20 2009, 14:00:07)  [GCC 4.1.2 20060628
prerelease (NetBSD nb2 20060711)], maxunicode: 65535

OpenSolaris SunOS-5.11-i86pc-i386-32bit: python: 2.4.4 (#1, Mar 10 2009,
09:35:36) [C], maxunicode: 65535
Nexenta NCP1 SunOS-5.11-i86pc-i386-32bit: python: 2.4.3 (#2, May  3 2006,
19:12:42)  [GCC 4.0.3 (GNU_OpenSolaris 4.0.3-1nexenta4)], maxunicode: 1114111

Mac OS 10.6 "snow leopard" i386: python: 2.6.1 (r261:67515, Jul  7 2009,
23:51:51)  [GCC 4.2.1 (Apple Inc. build 5646)], maxunicode: 65535
Mac OS 10.5 "leopard" i386: python: 2.5.1 (r251:54863, Feb  6 2009, 19:02:12)
[GCC 4.0.1 (Apple Inc. build 5465)], maxunicode: 65535
Mac OS 10.4 "tiger" *custom* python: 2.5.4 (release25-maint:72153M, Apr 30 2009,
12:28:20)  [GCC 4.0.1 (Apple Computer, Inc. build 5367)], maxunicode: 65535

Cygwin CYGWIN_NT-5.1-1.5.25-0.156-4-2-i686-32bit-WindowsPE: python: 2.5.2
(r252:60911, Dec  2 2008, 09:26:14)  [GCC 3.4.4 (cygming special, gdc 0.12,
using dmd 0.125)], maxunicode: 65535

Windows: python: 2.6.2 (r262:71600, Apr 21 2009, 15:05:37) [MSC v.1500 32 bit
(Intel)], maxunicode: 65535
Windows: python: 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
(Intel)], maxunicode: 65535

From benjamin at python.org  Sun Sep 20 16:06:53 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 20 Sep 2009 09:06:53 -0500
Subject: [Python-Dev] please consider changing --enable-unicode default
	to ucs4
In-Reply-To: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
Message-ID: <1afaf6160909200706p4b2145f6qb21f8a4d6074a2c3@mail.gmail.com>

2009/9/20 Zooko O'Whielacronx <zookog at gmail.com>:
> Dear Pythonistas:
>
> This issue causes serious problems. ?Users occasionally get binaries built for a
> compatible Linux and Python version but with a different UCS2-vs-UCS4 setting,
> and those users get mysterious memory corruption errors which are hard to
> diagnose. ?It is possible that these situations also open up security
> vulnerabilities. ?A couple such instances are documented on
> http://bugs.python.org/setuptools/issue78, but you can find more by googling.
> I would like to get this problem fixed!

You may want to have a look at the archives of the last time this was
extensively discussed:
http://mail.python.org/pipermail/python-dev/2008-July/080886.html


-- 
Regards,
Benjamin

From zookog at gmail.com  Sun Sep 20 16:16:01 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Sun, 20 Sep 2009 08:16:01 -0600
Subject: [Python-Dev] please consider changing --enable-unicode default
	to ucs4
In-Reply-To: <1afaf6160909200706p4b2145f6qb21f8a4d6074a2c3@mail.gmail.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<1afaf6160909200706p4b2145f6qb21f8a4d6074a2c3@mail.gmail.com>
Message-ID: <cd6401a0909200716g16097566g6dea6778f352fc81@mail.gmail.com>

I'm sorry, I should have mentioned that I did read those archives
before I posted my letter.  That discussion was all about whether UCS2
or UCS4 is better.  I consider that question to be mostly irrelevant
to this issue, which is about compatibility for people who don't
choose to configure that setting themselves.  Platforms or people who
prefer UCS2 will continue to use it as appropriate.  UCS4 is clearly
good enough for the vast majority of Linux users, and having fewer
mysterious segfaults and potential security vulnerabilities would be
an important improvement to the user experience of Python on Linux.

I should mention that the reason I'm spending time on this right now
is that it is currently blocking me from being able to distribute
binaries of Python packages which will work for all of my Linux users.

Regards,

Zooko

From solipsis at pitrou.net  Sun Sep 20 16:27:18 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 20 Sep 2009 14:27:18 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?please_consider_changing_--enable-unicode_?=
	=?utf-8?q?default_to=09ucs4?=
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
Message-ID: <loom.20090920T162358-891@post.gmane.org>

Zooko O'Whielacronx <zookog <at> gmail.com> writes:
> 
> Users occasionally get binaries built for a
> compatible Linux and Python version but with a different UCS2-vs-UCS4 setting,
> and those users get mysterious memory corruption errors which are hard to
> diagnose.

What "binaries" are you talking about?
AFAIK, C extensions should fail loading when they have the wrong UCS2/4 setting.
That's the reason we have all those #define's in unicodeobject.h: the actual
function names end up being different and, therefore, are not found when linking.

> In order to help address this issue I sampled what UCS size is used by python
> executables in the wild.

For information, all Mandriva versions I've used until now have had their
Python's built with UCS2 (maxunicode == 65535).

Regards

Antoine.



From daniel at stutzbachenterprises.com  Sun Sep 20 16:43:10 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sun, 20 Sep 2009 09:43:10 -0500
Subject: [Python-Dev] Fuzziness in io module specs - PEP update
	proposition
In-Reply-To: <4AB5FA81.5060408@gmail.com>
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<200909201007.11923.steve@pearwood.info>
	<4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com>
	<4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com>
Message-ID: <eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com>

On Sun, Sep 20, 2009 at 4:48 AM, Pascal Chambon <chambon.pascal at gmail.com>wrote:

> *RawIOBase*.readinto(b: bytes) -> int
>

"bytes" are immutable.  The signature is:

*RawIOBase*.readinto(b: bytearray) -> int

Your efforts in working on clarifying these important corner cases is
appreciated. :-)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090920/35e32290/attachment.htm>

From zookog at gmail.com  Sun Sep 20 18:17:45 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Sun, 20 Sep 2009 10:17:45 -0600
Subject: [Python-Dev] please consider changing --enable-unicode default
	to ucs4
In-Reply-To: <loom.20090920T162358-891@post.gmane.org>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
Message-ID: <cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>

On Sun, Sep 20, 2009 at 8:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> What "binaries" are you talking about?

I mean extension modules with native code, which means .so shared
library files on unix.

> AFAIK, C extensions should fail loading when they have the wrong UCS2/4 setting.

That would be an improvement!  Unfortunately we instead get mysterious
misbehavior of the module, e.g.:

http://bugs.python.org/setuptools/msg309
http://allmydata.org/trac/tahoe/ticket/704#comment:5

> For information, all Mandriva versions I've used until now have had their
> Python's built with UCS2 (maxunicode == 65535).

Thank you for the data point.  This means that binary extension
modules built on Mandriva can't be ported to Ubuntu or vice versa.
However, is this an argument for or against changing the default
setting to UCS4?  Changing the default setting wouldn't interfere with
Mandriva's decision, right?

Regards,

Zooko

From chambon.pascal at gmail.com  Sun Sep 20 18:32:14 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Sun, 20 Sep 2009 18:32:14 +0200
Subject: [Python-Dev] Fuzziness in io module specs - PEP
	update	proposition
In-Reply-To: <eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com>
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<200909201007.11923.steve@pearwood.info>	<4AB58159.3040303@canterbury.ac.nz>
	<4AB5D6CB.6050604@gmail.com>	<4AB5E8DD.6050204@canterbury.ac.nz>
	<4AB5FA81.5060408@gmail.com>
	<eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com>
Message-ID: <4AB6590E.2070800@gmail.com>

Daniel Stutzbach a ?crit :
> On Sun, Sep 20, 2009 at 4:48 AM, Pascal Chambon 
> <chambon.pascal at gmail.com <mailto:chambon.pascal at gmail.com>> wrote:
>
>     *RawIOBase*.readinto(b: bytes) -> int
>
>
> "bytes" are immutable.  The signature is:
>
> *RawIOBase*.readinto(b: bytearray) -> int
>
> Your efforts in working on clarifying these important corner cases is 
> appreciated. :-)
>
You're welcome B-)

Indeed my copy/paste of the current pep was an epic fail - you'll all 
have recognized readinto actually dealt with bytearrays, contrarily to 
what the current pep tells
-> http://www.python.org/dev/peps/pep-3116/

RawIOBase.read(int) takes a positive-or-zero integer indeed (I am used 
to understanding this, as opposed to "strictly positive")

Does MRAb's suggestion of providing beginning and end offsets for the 
bytearray meets people's expectations ? Personnaly, I feel readinto is a 
very low-level method, mostly used by read() to get a result from 
low-level native functions (fread, readfile), and read() always provides 
a buffer with the proper size... are there cases in which these two 
additional arguments would provide some real gain ?


Concerning the "backward compatibility" problem, I agree we should not 
break specifications, but breaking impelmentation details is another 
thing for me. It's a golden rule in programmers' world : thou shalt 
NEVER rely on implementation details. Programs that count on these (eg. 
thinking that listdir() will always returns "." and ".." as first 
item0... until it doesnt anymore) encounter huge problems when changing 
of platform or API version. When programming with the current 
truncate(), I would always have moved the file pointer after truncating 
the file, simply because I have no idea of what might happen to it 
(nothing was documented on this at the moment, and looking at the 
sources is really not a sustainable behaviour).
So well, it's a pity if some early 3.1 users relied on it, but if we 
stick to the current semantic we still have a real coherency problem - 
seek() is not limited in range, and some experienced programmers might 
be trapped by this non-conventionnal truncate() if they rely on posix or 
previous python versions... I really dislike the idea that truncate() 
might move my file offset even when there are no reasons for it.

Regards,
Pascal




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090920/2fd8b208/attachment-0001.htm>

From zookog at gmail.com  Sun Sep 20 18:33:23 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Sun, 20 Sep 2009 10:33:23 -0600
Subject: [Python-Dev] please consider changing --enable-unicode default
	to ucs4
In-Reply-To: <loom.20090920T162358-891@post.gmane.org>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
Message-ID: <cd6401a0909200933t26ec258em21289fa8f22e81fc@mail.gmail.com>

On Sun, Sep 20, 2009 at 8:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> For information, all Mandriva versions I've used until now have had their
> Python's built with UCS2 (maxunicode == 65535).

By the way, I was investigating this, and discovered an issue on the
Mandriva tracker which suggests that they intend to switch to UCS4 in
the next release in order to avoid compatibility problems like these.
(Not because they think that UCS4 is better than UCS2.)

https://qa.mandriva.com/show_bug.cgi?id=48570

Regards,

Zooko

From solipsis at pitrou.net  Sun Sep 20 18:36:03 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 20 Sep 2009 16:36:03 +0000 (UTC)
Subject: [Python-Dev] please consider changing --enable-unicode
	default	to ucs4
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
Message-ID: <h95llj$d57$1@ger.gmane.org>

Le Sun, 20 Sep 2009 10:17:45 -0600, Zooko O'Whielacronx a ?crit?:
> 
>> AFAIK, C extensions should fail loading when they have the wrong UCS2/4
>> setting.
> 
> That would be an improvement!  Unfortunately we instead get mysterious
> misbehavior of the module, e.g.:
> 
> http://bugs.python.org/setuptools/msg309
> http://allmydata.org/trac/tahoe/ticket/704#comment:5

The bug reports in themselves aren't very explicit, and they don't seem 
to be related to any native extension. So I'm not sure why you're talking 
about "mysterious memory corruption errors" in your original mail, 
because there doesn't seem to be such a thing happening at all.

Please note that there's a bug related to a non-portable peephole 
optimization of some unicode constants, perhaps it may explain the 
aforementioned problems (perhaps not) :
http://bugs.python.org/issue5057

I expect the solution to this bug to be rather easy (just disable the 
optimization, since it isn't really useful), but someone has to care 
enough to produce a patch.

>> For information, all Mandriva versions I've used until now have had
>> their Python's built with UCS2 (maxunicode == 65535).
> 
> Thank you for the data point.  This means that binary extension modules
> built on Mandriva can't be ported to Ubuntu or vice versa.

"Ported" they can certainly be, you just have to recompile.

> However, is
> this an argument for or against changing the default setting to UCS4? 
> Changing the default setting wouldn't interfere with Mandriva's
> decision, right?

Well, let's put it this way:
- either you expect the default setting to be observed by everyone, and 
it *will* interfere with someone's current decision
- or you don't expect the default setting to be observed by everyone, and 
then there's no point in changing it because it won't stop your problems

Either way, my mentioning of Mandriva was just meant as an additional 
data point to those you already provided ;-)

Regards

Antoine.


From solipsis at pitrou.net  Sun Sep 20 18:42:31 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 20 Sep 2009 16:42:31 +0000 (UTC)
Subject: [Python-Dev] please consider changing --enable-unicode
	default	to ucs4
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200933t26ec258em21289fa8f22e81fc@mail.gmail.com>
Message-ID: <h95m1n$d57$2@ger.gmane.org>

Le Sun, 20 Sep 2009 10:33:23 -0600, Zooko O'Whielacronx a ?crit?:
> 
> By the way, I was investigating this, and discovered an issue on the
> Mandriva tracker which suggests that they intend to switch to UCS4 in
> the next release in order to avoid compatibility problems like these.

Trying to use a Fedora or Suse RPM under Mandriva (or the other way 
round) isn't reasonable and is certainly not supported.
I don't understand why this so-called "compatibility problem" should be 
taken seriously by anyone.

Regards

Antoine.



From mal at egenix.com  Sun Sep 20 20:28:00 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Sun, 20 Sep 2009 20:28:00 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to ucs4
In-Reply-To: <cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
Message-ID: <4AB67430.2060306@egenix.com>

Zooko O'Whielacronx wrote:
> On Sun, Sep 20, 2009 at 8:27 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> What "binaries" are you talking about?
> 
> I mean extension modules with native code, which means .so shared
> library files on unix.

Those will not load unless they are for the right UCS-version of
Python. The extensions will give an ImportError if they are
using any Unicode APIs - we go through great lengths in the
Unicode API to make sure that you cannot mix UCS2 and UCS4 APIs.

I'm not exactly sure what you are trying to achieve by making
UCS4 the default... if you build extensions using the system
Python version, distutils will automatically build the right
UCS-version for you.

>> AFAIK, C extensions should fail loading when they have the wrong UCS2/4 setting.
> 
> That would be an improvement!  Unfortunately we instead get mysterious
> misbehavior of the module, e.g.:
> 
> http://bugs.python.org/setuptools/msg309
> http://allmydata.org/trac/tahoe/ticket/704#comment:5

Those don't appear to be related to UCS2 vs. UCS4 but rather
some problem with the UTF-8 data those users are trying to load.

The fact that setuptools completely ignores the fact
that Python UCS2 and UCS4 are two different Python builds, is
not really a Python Unicode problem, but one of the setuptools design,
so you should probably complain there.

>> For information, all Mandriva versions I've used until now have had their
>> Python's built with UCS2 (maxunicode == 65535).
> 
> Thank you for the data point.  This means that binary extension
> modules built on Mandriva can't be ported to Ubuntu or vice versa.
> However, is this an argument for or against changing the default
> setting to UCS4?  Changing the default setting wouldn't interfere with
> Mandriva's decision, right?

Depends on what you mean with "ported": of course you can port a
source RPM between UCS2 and UCS4 builds. This just requires a
recompile.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 20 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From python at mrabarnett.plus.com  Sun Sep 20 20:46:44 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 20 Sep 2009 19:46:44 +0100
Subject: [Python-Dev] Fuzziness in io module specs -
	PEP	update	proposition
In-Reply-To: <4AB6590E.2070800@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<200909201007.11923.steve@pearwood.info>	<4AB58159.3040303@canterbury.ac.nz>	<4AB5D6CB.6050604@gmail.com>	<4AB5E8DD.6050204@canterbury.ac.nz>	<4AB5FA81.5060408@gmail.com>	<eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com>
	<4AB6590E.2070800@gmail.com>
Message-ID: <4AB67894.5020903@mrabarnett.plus.com>

Pascal Chambon wrote:
> Daniel Stutzbach a ?crit :
>> On Sun, Sep 20, 2009 at 4:48 AM, Pascal Chambon 
>> <chambon.pascal at gmail.com <mailto:chambon.pascal at gmail.com>> wrote:
>>
>>     *RawIOBase*.readinto(b: bytes) -> int
>>
>>
>> "bytes" are immutable.  The signature is:
>>
>> *RawIOBase*.readinto(b: bytearray) -> int
>>
>> Your efforts in working on clarifying these important corner cases is 
>> appreciated. :-)
>>
> You're welcome B-)
> 
> Indeed my copy/paste of the current pep was an epic fail - you'll all 
> have recognized readinto actually dealt with bytearrays, contrarily to 
> what the current pep tells
> -> http://www.python.org/dev/peps/pep-3116/
> 
> RawIOBase.read(int) takes a positive-or-zero integer indeed (I am used 
> to understanding this, as opposed to "strictly positive")
> 
> Does MRAb's suggestion of providing beginning and end offsets for the 
> bytearray meets people's expectations ? Personnaly, I feel readinto is a 
> very low-level method, mostly used by read() to get a result from 
> low-level native functions (fread, readfile), and read() always provides 
> a buffer with the proper size... are there cases in which these two 
> additional arguments would provide some real gain ?
> 
It's useful if you want to fill the buffer but 'read' might return fewer
bytes than you asked for because it returns only what's available. That
might not happen for files, but it might for other forms of I/O. Other
languages, like Delphi and Java, which read into pre-existing arrays,
have or allow the extra parameters.
> 
> Concerning the "backward compatibility" problem, I agree we should not 
> break specifications, but breaking impelmentation details is another 
> thing for me. It's a golden rule in programmers' world : thou shalt 
> NEVER rely on implementation details. Programs that count on these (eg. 
> thinking that listdir() will always returns "." and ".." as first 
> item0... until it doesnt anymore) encounter huge problems when changing 
> of platform or API version. When programming with the current 
> truncate(), I would always have moved the file pointer after truncating 
> the file, simply because I have no idea of what might happen to it 
> (nothing was documented on this at the moment, and looking at the 
> sources is really not a sustainable behaviour).
> So well, it's a pity if some early 3.1 users relied on it, but if we 
> stick to the current semantic we still have a real coherency problem - 
> seek() is not limited in range, and some experienced programmers might 
> be trapped by this non-conventionnal truncate() if they rely on posix or 
> previous python versions... I really dislike the idea that truncate() 
> might move my file offset even when there are no reasons for it.
> 
Well, if it's consistent and documented (and not totally stupid), I
can't really complain. :-)

From martin at v.loewis.de  Sun Sep 20 21:27:06 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 20 Sep 2009 21:27:06 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to ucs4
In-Reply-To: <cd6401a0909200716g16097566g6dea6778f352fc81@mail.gmail.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<1afaf6160909200706p4b2145f6qb21f8a4d6074a2c3@mail.gmail.com>
	<cd6401a0909200716g16097566g6dea6778f352fc81@mail.gmail.com>
Message-ID: <4AB6820A.8060202@v.loewis.de>

Zooko O'Whielacronx wrote:
> I'm sorry, I should have mentioned that I did read those archives
> before I posted my letter.  That discussion was all about whether UCS2
> or UCS4 is better.  I consider that question to be mostly irrelevant
> to this issue, which is about compatibility for people who don't
> choose to configure that setting themselves. 

You surely must have missed the sentence

"For that reason I think it's also better that the configure script
continues to default to UTF-16 -- this will give the UTF-16 support
code the necessary exercise."

This is effectively a BDFL pronouncement. Nothing has changed the
validity of the premise of the statement, so the conclusion remains
valid, as well.

Regards,
Martin

From solipsis at pitrou.net  Mon Sep 21 23:30:58 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 21 Sep 2009 21:30:58 +0000 (UTC)
Subject: [Python-Dev] buildbots
Message-ID: <h98rai$b5u$1@ger.gmane.org>


Hello,

I realize I may have missed some kind of formal announcement, but why 
have all the buildbots been down for a number of days (weeks?) now?

Thanks

Antoine.


From ziade.tarek at gmail.com  Tue Sep 22 15:21:06 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 22 Sep 2009 15:21:06 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
Message-ID: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>

Hello

Here's a wrapup of the Distutils-SIG discussion
we had on the "static metadata" topic.

I realize that it's a good thing to send in.
python-dev such wrapup on distutils design
decisions, to keep everyone informed and get
more feedback when required.

I will try to do it for every upcoming changes
that are not going in a PEP process (when it's not
a 'big' change). The rate of such mails should
not be very high. (around 1 mail in python-dev
for +150 mails in distutils-SIG)

If you feel that what we are about to change in distutils
is wrong, you can go ahead and help us by participating
in Distutils-ML, so we keep one and only one media
for these discussions.

The four sentences summary for people in a hurry:

    Getting metadata of a distribution that is not.
    installed means running its setup.py. This means.
    downloading the whole archive, and running.
    third party code on your system.

    To avoid it, we are adding a [setup] section in.
    setup.cfg where you can express the package metadata
    statically.

    Conditional sections, specific to some system.
    can be added to add some specific fields in [setup].

    At the end, you will be able to get metadata fields
    without running any third-party code, and possibly
    get only the distribution setup.cfg for this.

Now the long version.


Rational
========

Today, if you want to list all the metadata (PEP 314) of a.
distribution that is not installed, you need to use it's.
setup.py CLI.

So, basically, you download it, and run::

    $ python setup.py --name
    Foo

    $ python setup.py --version
    1.2

Where `name` and `version` are metadata fields. That's OK but as.
soon as the developers add more code in setup.py, this feature
might break or (worse) might do unwanted things on the target.
system.

Why should we run third-party code just to get its metadata ?

So we worked on a way to express those metadata statically,
by pushing them in `setup.cfg`. That's not hard, since all.
the metadata fields are static values.

Adding a setup section in setup.cfg
===================================

So the first thing we want to introduce is a [setup] section,
that may contain any field from the metadata::

    [setup]
    name: Foo
    version: 1.2

Distutils will look for them, and will use them. Of course
setup.py is still required, and in case an option that is.
a metadata field is passed to setup(), it will override one.
located in setup.cfg.

PEP 341 is coming up
====================

Remember the last Pycon Summit ? We said that we would
introduce a new metadata field to describe requirements..
That's basically what PEP 341 is about, and we are still.
working on it.

Basically, we will be able to write things like::

    requires: Twisted == 8.2.0

What takes us so long is that adding requirements like
this in the metadata requires more things:

- the ability to compare versions (this was described in.
  PEP 386 but not finished yet)

- the ability to make these requirements vary depending on.
  the target system

And the later makes our "setup.cfg" new [setup] section.
obsolete as soon as this new metadata field is added in.
Python.

So we need more that what I've shown in the previous section

Context-dependant sections
==========================

The second change we are going to introduce is context-dependant
sections in setup.cfg.

A context dependant section is a section with a condition that is.
used only if the execution environment meets its condition.

Here's an example::

    [setup]
    name: Foo
    version: 1.3

    [setup:sys_platform == 'win32']
    requires: pywin32
    requires: bar > 1.0

    [setup:os_machine == '64bits']
    requires: some_package

    [setup:python_version == '2.4' or python_version == '2.5']
    requires: some_package


Every [setup:condition] section will be added in [setup]
only if the condition is met.

The micro-language behind this is the simplest possible:
it compares only strings, with usual string operators,
and with the ability to combine expressions. It makes it also
easy to understand to non-pythoneers (os packagers).

The pseudo-grammar is (I don't know how to write those but you'll
get it hopefully)::

    comp: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'
    comparison: expr (comp_op expr)*
    expr: STRING
    test: or_test
    or_test: and_test ('or' and_test)*
    and_test: not_test ('and' not_test)*
    not_test: 'not' not_test | comparison

where STRING belongs to any of those:

- python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])
- os_name = os.name
- sys_platform = sys.platform
- os_sysname = os.uname()[0].
- os_nodename = os.uname()[1]
- os_release = os.uname()[2].
- os_version = os.uname()[3]..
- os_machine = os.uname()[4].
- a free string, like '2.4', or 'win32'

Distutils will provide a function that is able to read the metadata
of a distribution, given a setup.cfg file, for the target environment::

    >>> from distutils.util import local_metadata
    >>> local_metadata('setup.cfg')
    <DistributionMetadata instance>

Meaning that a Vanilla Python will be able to read the metadata
of a package without running any code.

That's what distutils will use when setup.py is run.

That's what any package manager will be able to use to work.
with distributions.

Limitations
===========

If a distribution is unable for obscure reasons (or does not
wish) to set all metadata fields in setup.cfg, that's fine :.
the fields will be set to UNKNOWN when `local_metadata` is called.

When setup.py is run, options passed to setup() will complete
those.

When a package manager gets 'UNKOWN' values, it knows it might
need to do further processing by running setup.py.

Why this change is good
=====================

Once the requires metadata is added in PEP 341,
being able to query the metadata for a distribution can be done
without doing anything else than reading a static file and interpreting
its conditions in a restricted and secured manner.

So there's no need to run third-party code anymore.

Possible use cases:

- setup.cfg is published on PyPI. Packages managers like easy_install
  or pip will be able to list all the distributions needed and their
  versions for installing a distribution without having to download
  anything !

- OS packagers will be able to list the dependencies of a distribution
  without having to browse Python code.

And, again, this new [setup] section is backward-compatible, and will
be a good complement to the work done in PEP 376.

Thanks for reading so far. This code will be added in 2.7 and 3.2.

Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????!

From phd at phd.pp.ru  Tue Sep 22 15:24:15 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Tue, 22 Sep 2009 17:24:15 +0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
Message-ID: <20090922132415.GA15767@phd.pp.ru>

On Tue, Sep 22, 2009 at 03:21:06PM +0200, Tarek Ziad? wrote:
> I realize that it's a good thing to send in.
> python-dev such wrapup on distutils design
> decisions, to keep everyone informed and get
> more feedback when required.

   Sure it is.

> I will try to do it for every upcoming changes
> that are not going in a PEP process (when it's not
> a 'big' change).

   Thank you!

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From pje at telecommunity.com  Tue Sep 22 15:50:45 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 22 Sep 2009 09:50:45 -0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.co
 m>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
Message-ID: <20090922135049.583BB3A4079@sparrow.telecommunity.com>

At 03:21 PM 9/22/2009 +0200, Tarek Ziad? wrote:
>Hello Here's a wrapup of the Distutils-SIG discussion we had on the 
>"static metadata" topic. I realize that it's a good thing to send 
>in. python-dev such wrapup on distutils design decisions, to keep 
>everyone informed and get more feedback when required

The above implies that a consensus was reached on this topic; you 
should perhaps at least note that in this case it actually means that 
you just terminated the discussion and pronounced your choice, just 
as a simpler solution for dynamic dependencies (proposed by Sridhar) 
was beginning to gain some support.


From solipsis at pitrou.net  Tue Sep 22 15:53:21 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 22 Sep 2009 13:53:21 +0000 (UTC)
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
Message-ID: <loom.20090922T155226-326@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
>     [setup]
>     name: Foo
>     version: 1.2

I am probably missing something, but why doesn't it use the more common
ini-style:

    [setup]
    name = Foo
    version = 1.2

Regards

Antoine.



From fuzzyman at voidspace.org.uk  Tue Sep 22 15:55:31 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 22 Sep 2009 14:55:31 +0100
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <loom.20090922T155226-326@post.gmane.org>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<loom.20090922T155226-326@post.gmane.org>
Message-ID: <4AB8D753.3040800@voidspace.org.uk>

Antoine Pitrou wrote:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>   
>>     [setup]
>>     name: Foo
>>     version: 1.2
>>     
>
> I am probably missing something, but why doesn't it use the more common
> ini-style:
>
>     [setup]
>     name = Foo
>     version = 1.2
>   

ConfigParser recognises both, but other tools (*coff* ConfigObj) only 
recognises the latter (=).

Michael

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From chris at simplistix.co.uk  Tue Sep 22 16:02:56 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 22 Sep 2009 15:02:56 +0100
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
Message-ID: <4AB8D910.8050601@simplistix.co.uk>

Tarek Ziad? wrote:
> The four sentences summary for people in a hurry:

If you're going to promise sentences, please deliver sentences...

>     Getting metadata of a distribution that is not.
>     installed means running its setup.py. This means.
>     downloading the whole archive, and running.
>     third party code on your system.

This is not a sentence, and I have no idea what you're trying to convey...

> A context dependant section is a section with a condition that is.
> used only if the execution environment meets its condition.
> 
> Here's an example::
> 
>     [setup]
>     name: Foo
>     version: 1.3
> 
>     [setup:sys_platform == 'win32']
>     requires: pywin32
>     requires: bar > 1.0
> 
>     [setup:os_machine == '64bits']
>     requires: some_package
> 
>     [setup:python_version == '2.4' or python_version == '2.5']
>     requires: some_package

As was brought up on the distutils list, this seems unnecessarily 
complicated.

Other, simpler, solutions were proposed:

http://mail.python.org/pipermail/distutils-sig/2009-September/013173.html

http://mail.python.org/pipermail/distutils-sig/2009-September/013289.html

However, you pronounced yourself bdfl(!):

http://mail.python.org/pipermail/distutils-sig/2009-September/013318.html

...and decided to ignore them. Please don't do this over such an 
important matter.

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Tue Sep 22 16:06:42 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 22 Sep 2009 15:06:42 +0100
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both object have implementations
Message-ID: <4AB8D9F2.6050807@simplistix.co.uk>

Hi All,

I didn't see any docs on this:

http://docs.python.org/reference/datamodel.html?highlight=__eq__#object.__eq__

Where are the specifications on what happens if two objects are compared 
and both have implementations of __eq__? Which __eq__ is called? What 
happens if the first one called returns False? Is the second one called? 
What is one implements __eq__ and the other __ne__?

If I've missed something, please point me in the right direction.

To all those about to tell me to go read the source: that's not good 
enough here. I'm hoping there *are* "official" rules for how these 
interact and they just need better linking in, otherwise, I worry that 
IronPython could do one thing, Jython another and CPython a third...

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From jflatow at gmail.com  Tue Sep 22 16:20:04 2009
From: jflatow at gmail.com (Jared Flatow)
Date: Tue, 22 Sep 2009 07:20:04 -0700
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both object have implementations
In-Reply-To: <4AB8D9F2.6050807@simplistix.co.uk>
References: <4AB8D9F2.6050807@simplistix.co.uk>
Message-ID: <B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>

This might help:

http://mail.python.org/pipermail/python-dev/2008-June/080111.html

Here is the most relevant part (quoting Guido):

 > Does it help if I tell you that for "x <binop> y" we always try
 > x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the
 > case where y is an instance of a subclass of the class of x?

jared

On Sep 22, 2009, at 7:06 AM, Chris Withers wrote:

> Hi All,
>
> I didn't see any docs on this:
>
> http://docs.python.org/reference/datamodel.html?highlight=__eq__#object.__eq__
>
> Where are the specifications on what happens if two objects are  
> compared and both have implementations of __eq__? Which __eq__ is  
> called? What happens if the first one called returns False? Is the  
> second one called? What is one implements __eq__ and the other __ne__?
>
> If I've missed something, please point me in the right direction.
>
> To all those about to tell me to go read the source: that's not good  
> enough here. I'm hoping there *are* "official" rules for how these  
> interact and they just need better linking in, otherwise, I worry  
> that IronPython could do one thing, Jython another and CPython a  
> third...
>
> cheers,
>
> Chris
>
> -- 
> Simplistix - Content Management, Batch Processing & Python Consulting
>           - http://www.simplistix.co.uk
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jflatow%40gmail.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090922/5d2ffc0f/attachment.htm>

From chris at simplistix.co.uk  Tue Sep 22 16:37:42 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 22 Sep 2009 15:37:42 +0100
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
Message-ID: <4AB8E136.8040706@simplistix.co.uk>

Jared Flatow wrote:
> This might help:
> 
> http://mail.python.org/pipermail/python-dev/2008-June/080111.html
> 
> Here is the most relevant part (quoting Guido):
> 
>  > Does it help if I tell you that for "x <binop> y" we always try
>  > x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the
>  > case where y is an instance of a subclass of the class of x?

Okay, but does that count as a pronouncement that should go across all 
versions and platforms?

I believe there are differences between when __eq__ and __ne__ are 
called between Python 2 and 3, and I don't see any docs on the expected 
behaviour for python 2, dunno about 3.

I'm willing to write these docs, 
http://docs.python.org/reference/datamodel.html#object.__eq__ feels like 
the right place, is there anywhere else they should go or be linked to from?

Mark Dickinson gave some useful insights into this, and Milko Krachounov 
provided these useful comparisons on #python:

Python 2
http://pastebin.com/f8f19ab3

Python 3
http://pastebin.com/f55e44630

Do they cover it all or has anything been missed?

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From dickinsm at gmail.com  Tue Sep 22 17:12:32 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 22 Sep 2009 16:12:32 +0100
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <4AB8E136.8040706@simplistix.co.uk>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
Message-ID: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>

On Tue, Sep 22, 2009 at 3:37 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> Where are the specifications on what happens if two objects are compared and both have implementations of __eq__? Which __eq__ is called?
> What happens if the first one called returns False? Is the second one called? What is one implements __eq__ and the other __ne__?

I (still :-) think this is covered, for Python 2.x at least, by:

http://docs.python.org/reference/datamodel.html#coercion-rules

Specifically, the bits that say:

- For objects x and y, first x.__op__(y) is tried. If this is not
implemented or returns NotImplemented, y.__rop__(x) is tried. If this
is also not implemented or returns NotImplemented, a TypeError
exception is raised. But see the following exception:

- Exception to the previous item: if the left operand is an instance
of a built-in type or a new-style class, and the right operand is an
instance of a proper subclass of that type or class and overrides the
base?s __rop__() method, the right operand?s __rop__() method is tried
before the left operand?s __op__() method.

I agree that having these rules in a section called 'Coercion rules'
is a bit confusing.

> Python 2
> http://pastebin.com/f8f19ab3
>
> Python 3
> http://pastebin.com/f55e44630

The duplicate __eq__ calls in these pastes are disturbing:  in short,
if A() and B() are new-style classes defining __eq__, it seems that
A() == B() ends up calling A.__eq__ twice *and* B.__eq__ twice, in the
order A.__eq__, B.__eq__, B.__eq__, A.__eq__.

In 3.x, slot_tp_richcompare (in typeobject.c) makes two calls to
half_richcompare;  I think the second is redundant.  The coercion
rules are already taken care of in do_richcompare (in object.c).  I
tried removing the second call to half_richcompare, and the entire
test-suite still runs without errors.

In 2.x, it's possible that this call is necessary for some bizarre
combinations of __cmp__ and __eq__;  I haven't tried to get my head
around this yet.

I'll open an issue for the duplicate __eq__ calls.

Mark

From metawilm at gmail.com  Tue Sep 22 17:23:03 2009
From: metawilm at gmail.com (Willem Broekema)
Date: Tue, 22 Sep 2009 17:23:03 +0200
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
Message-ID: <f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>

On Tue, Sep 22, 2009 at 5:12 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> - Exception to the previous item: if the left operand is an instance
> of a built-in type or a new-style class, and the right operand is an
> instance of a proper subclass of that type or class AND overrides the
> base?s __rop__() method, the right operand?s __rop__() method is tried
> before the left operand?s __op__() method.

The AND above (which I uppercased) is subtle but important. In the "x
op y" case with y being of a subclass of the class of x, if there is
no class in between x and y (excluding x, including y) that overrides
the __rop__ method, then y,__rop__(x) is *not* tried before
x.__op__(y).

It's easy for other implementations to get this wrong. :)

- Willem

From dirkjan at ochtman.nl  Tue Sep 22 16:02:55 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Tue, 22 Sep 2009 16:02:55 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <4AB8D753.3040800@voidspace.org.uk>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<loom.20090922T155226-326@post.gmane.org>
	<4AB8D753.3040800@voidspace.org.uk>
Message-ID: <ea2499da0909220702o5c1156d2o6f8e4e4608b3cb6e@mail.gmail.com>

On Tue, Sep 22, 2009 at 15:55, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> ConfigParser recognises both, but other tools (*coff* ConfigObj) only
> recognises the latter (=).

So does Mercurial, BTW, because this aspect of ConfigParser was pretty
annoying for some of the stuff we try to do (and not at all obvious to
most users, who have never seen anything but = used in config-style
files).

Cheers,

Dirkjan

From dickinsm at gmail.com  Tue Sep 22 18:01:20 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 22 Sep 2009 17:01:20 +0100
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
Message-ID: <5c6f2a5d0909220901o36d709a9n8842cb58c00bc863@mail.gmail.com>

On Tue, Sep 22, 2009 at 4:12 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> I'll open an issue for the duplicate __eq__ calls.

http://bugs.python.org/issue6970

Mark

From ziade.tarek at gmail.com  Tue Sep 22 20:06:55 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 22 Sep 2009 20:06:55 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <4AB8D910.8050601@simplistix.co.uk>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
Message-ID: <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>

On Tue, Sep 22, 2009 at 4:02 PM, Chris Withers <chris at simplistix.co.uk> wrote:
>> ? ?[setup:python_version == '2.4' or python_version == '2.5']
>> ? ?requires: some_package
>
> As was brought up on the distutils list, this seems unnecessarily
> complicated.
>
> Other, simpler, solutions were proposed:
>
> http://mail.python.org/pipermail/distutils-sig/2009-September/013173.html
>
> http://mail.python.org/pipermail/distutils-sig/2009-September/013289.html
>
> However, you pronounced yourself bdfl(!):
>
> http://mail.python.org/pipermail/distutils-sig/2009-September/013318.html
>
> ...and decided to ignore them. Please don't do this over such an important
> matter.

Yes, for the syntax bikeshedding on how conditions should be expressed,
I have decided I'll be some kind of distutils dictator to end up the
endless discussions
on how they should be expressed so we can move on.

Tarek

From fuzzyman at voidspace.org.uk  Tue Sep 22 20:07:56 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 22 Sep 2009 19:07:56 +0100
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
Message-ID: <4AB9127C.5070102@voidspace.org.uk>

Tarek Ziad? wrote:
> On Tue, Sep 22, 2009 at 4:02 PM, Chris Withers <chris at simplistix.co.uk> wrote:
>   
>>>    [setup:python_version == '2.4' or python_version == '2.5']
>>>    requires: some_package
>>>       
>> As was brought up on the distutils list, this seems unnecessarily
>> complicated.
>>
>> Other, simpler, solutions were proposed:
>>
>> http://mail.python.org/pipermail/distutils-sig/2009-September/013173.html
>>
>> http://mail.python.org/pipermail/distutils-sig/2009-September/013289.html
>>
>> However, you pronounced yourself bdfl(!):
>>
>> http://mail.python.org/pipermail/distutils-sig/2009-September/013318.html
>>
>> ...and decided to ignore them. Please don't do this over such an important
>> matter.
>>     
>
> Yes, for the syntax bikeshedding on how conditions should be expressed,
> I have decided I'll be some kind of distutils dictator to end up the
> endless discussions
> on how they should be expressed so we can move on.
>   

+1

Michael

> Tarek
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ziade.tarek at gmail.com  Tue Sep 22 20:12:24 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 22 Sep 2009 20:12:24 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <20090922135049.583BB3A4079@sparrow.telecommunity.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<20090922135049.583BB3A4079@sparrow.telecommunity.com>
Message-ID: <94bdd2610909221112u48484e93oc49f70445cf1fbe4@mail.gmail.com>

2009/9/22 P.J. Eby <pje at telecommunity.com>:
> At 03:21 PM 9/22/2009 +0200, Tarek Ziad? wrote:
>>
>> Hello Here's a wrapup of the Distutils-SIG discussion we had on the
>> "static metadata" topic. I realize that it's a good thing to send in.
>> python-dev such wrapup on distutils design decisions, to keep everyone
>> informed and get more feedback when required
>
> The above implies that a consensus was reached on this topic; you should
> perhaps at least note that in this case it actually means that you just
> terminated the discussion and pronounced your choice, just as a simpler
> solution for dynamic dependencies (proposed by Sridhar) was beginning to
> gain some support.

I pronounced my choice on the syntax of the conditions in the setup.cfg file,
and the setuptools-like syntax to express them Sridhar presented
looked more complex.

If you are talking about another solution than using setup.cfg with conditional
expressions, please start a new thread that demonstrates how it will work,
we all want the best solution of course.

Tarek

From ziade.tarek at gmail.com  Tue Sep 22 20:15:30 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 22 Sep 2009 20:15:30 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <ea2499da0909220702o5c1156d2o6f8e4e4608b3cb6e@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<loom.20090922T155226-326@post.gmane.org>
	<4AB8D753.3040800@voidspace.org.uk>
	<ea2499da0909220702o5c1156d2o6f8e4e4608b3cb6e@mail.gmail.com>
Message-ID: <94bdd2610909221115r658a0258yd0c84a2628d8e4b7@mail.gmail.com>

On Tue, Sep 22, 2009 at 4:02 PM, Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
> On Tue, Sep 22, 2009 at 15:55, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>> ConfigParser recognises both, but other tools (*coff* ConfigObj) only
>> recognises the latter (=).
>
> So does Mercurial, BTW, because this aspect of ConfigParser was pretty
> annoying for some of the stuff we try to do (and not at all obvious to
> most users, who have never seen anything but = used in config-style
> files).

yes, I've just used ConfigParser syntax because that's what Distutils uses.

If ':' is deprecated in favor of '=' only, Distutils will be fine with
that change.

I would be +1 for its deprecation

Tarek

-- 
Tarek Ziad? | http://ziade.org |????????????!

From ziade.tarek at gmail.com  Tue Sep 22 20:16:03 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Tue, 22 Sep 2009 20:16:03 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909221112u48484e93oc49f70445cf1fbe4@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<20090922135049.583BB3A4079@sparrow.telecommunity.com>
	<94bdd2610909221112u48484e93oc49f70445cf1fbe4@mail.gmail.com>
Message-ID: <94bdd2610909221116m26549490k2c45cfe188566207@mail.gmail.com>

On Tue, Sep 22, 2009 at 8:12 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> If you are talking about another solution than using setup.cfg with conditional
> expressions, please start a new thread

(in Distutils SIG)

From fuzzyman at voidspace.org.uk  Tue Sep 22 20:26:42 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 22 Sep 2009 19:26:42 +0100
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909221115r658a0258yd0c84a2628d8e4b7@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<loom.20090922T155226-326@post.gmane.org>
	<4AB8D753.3040800@voidspace.org.uk>
	<ea2499da0909220702o5c1156d2o6f8e4e4608b3cb6e@mail.gmail.com>
	<94bdd2610909221115r658a0258yd0c84a2628d8e4b7@mail.gmail.com>
Message-ID: <CEC460CA-1671-4C84-B9D3-D752E5DD70DC@voidspace.org.uk>




--
http://www.ironpythoninaction.com

On 22 Sep 2009, at 19:15, Tarek Ziad? <ziade.tarek at gmail.com> wrote:

> On Tue, Sep 22, 2009 at 4:02 PM, Dirkjan Ochtman  
> <dirkjan at ochtman.nl> wrote:
>> On Tue, Sep 22, 2009 at 15:55, Michael Foord <fuzzyman at voidspace.org.uk 
>> > wrote:
>>> ConfigParser recognises both, but other tools (*coff* ConfigObj)  
>>> only
>>> recognises the latter (=).
>>
>> So does Mercurial, BTW, because this aspect of ConfigParser was  
>> pretty
>> annoying for some of the stuff we try to do (and not at all obvious  
>> to
>> most users, who have never seen anything but = used in config-style
>> files).
>
> yes, I've just used ConfigParser syntax because that's what  
> Distutils uses.
>
> If ':' is deprecated in favor of '=' only, Distutils will be fine with
> that change.
>
> I would be +1 for its deprecation
>

I don't think : is formally deprecated but = is more common and it  
would be great if distutils could default to (and document) this syntax.

Michael



> Tarek
>
> -- 
> Tarek Ziad? | http://ziade.org | ???????????!

From pje at telecommunity.com  Tue Sep 22 20:52:45 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 22 Sep 2009 14:52:45 -0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909221112u48484e93oc49f70445cf1fbe4@mail.gmail.co
 m>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<20090922135049.583BB3A4079@sparrow.telecommunity.com>
	<94bdd2610909221112u48484e93oc49f70445cf1fbe4@mail.gmail.com>
Message-ID: <20090922185248.377CC3A403D@sparrow.telecommunity.com>

At 08:12 PM 9/22/2009 +0200, Tarek Ziad? wrote:
>If you are talking about another solution than using setup.cfg with 
>conditional
>expressions, please start a new thread that demonstrates how it will work,
>we all want the best solution of course.

I already did that, on the distutils-sig.  Or more precisely, someone 
else did.  I only elaborated on the idea they proposed.  (I believe 
it was Sridhar who first proposed it, but my memory may be incorrect.)


From chris at simplistix.co.uk  Tue Sep 22 21:51:58 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 22 Sep 2009 20:51:58 +0100
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
Message-ID: <4AB92ADE.9060700@simplistix.co.uk>

Tarek Ziad? wrote:
> Yes, for the syntax bikeshedding on how conditions should be expressed,
> I have decided I'll be some kind of distutils dictator to end up the
> endless discussions
> on how they should be expressed so we can move on.

...except in this case, spelling is important and I'm far from the only 
person who would prefer you not to foist an unpalatable mess on the 
entire python community just because the ideas that other people prefer 
are not yours.

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From brett at python.org  Tue Sep 22 22:02:54 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 22 Sep 2009 13:02:54 -0700
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <4AB92ADE.9060700@simplistix.co.uk>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> 
	<4AB92ADE.9060700@simplistix.co.uk>
Message-ID: <bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>

On Tue, Sep 22, 2009 at 12:51, Chris Withers <chris at simplistix.co.uk> wrote:
> Tarek Ziad? wrote:
>>
>> Yes, for the syntax bikeshedding on how conditions should be expressed,
>> I have decided I'll be some kind of distutils dictator to end up the
>> endless discussions
>> on how they should be expressed so we can move on.
>
> ...except in this case, spelling is important and I'm far from the only
> person who would prefer you not to foist an unpalatable mess on the entire
> python community just because the ideas that other people prefer are not
> yours.

OK, stop before it gets any nastier as your email already sounds rude, Chris.

Since the language summit at PyCon 2009 various committers, including
me, have been encouraging Tarek to act as distutils dictator to get
things finished as we all know people are prone to bikeshedding that
would kill any forward momentum we have towards improving the
packaging situation. Since Tarek has been doing the legwork on all of
this and continues to be the main contributor/pusher of distutils
improvements I see no reason why he should not be able to
short-circuit discussions when he sees them as having gone on long
enough when he is the one that is going to ultimately end up doing all
the coding. I honestly think Tarek listens better than most when it
comes to input and tends to be extremely cordial about things.

Every decision Tarek makes, just like any decision involving Python
made by anyone, is going to upset/alienate/piss off some subset of the
Python community. That's just life. If it came down to distutils just
sitting there with no improvement or Tarek choosing some syntax for
config files that I didn't love, I personally would choose the latter
as it's better than nothing and syntax is definitely one place where
you can't please everyone.

-Brett

From dinov at microsoft.com  Tue Sep 22 23:55:27 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Tue, 22 Sep 2009 21:55:27 +0000
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both 	object have implementations
In-Reply-To: <f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
Message-ID: <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>

Willem wrote:
> 
> It's easy for other implementations to get this wrong. :)

For IronPython we wrote a set of tests which go through and define 
the various operator methods in all sorts of combinations on both 
new-style and old-style classes as well as subclasses of those classes
and then do the comparisons w/ logging.

We've got this for both comparisons and the normal binary operators. We've
also done it in other areas such as throwing exceptions and other 
complicated areas.  But having the the coercion rules documented still 
helped a lot (although my recollection is they were only about 95% right).

From chris at simplistix.co.uk  Wed Sep 23 00:48:08 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Tue, 22 Sep 2009 23:48:08 +0100
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
Message-ID: <4AB95428.1040704@simplistix.co.uk>

Brett Cannon wrote:
> OK, stop before it gets any nastier as your email already sounds rude, Chris.

Yes, apologies.

> Since the language summit at PyCon 2009 various committers, including
> me, have been encouraging Tarek to act as distutils dictator to get
> things finished as we all know people are prone to bikeshedding that
> would kill any forward momentum we have towards improving the
> packaging situation.

...except that this isn't bikeshedding. Use of that term just to stifle 
productive discussion is also rude.

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From sridharr at activestate.com  Wed Sep 23 00:53:46 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Tue, 22 Sep 2009 15:53:46 -0700
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <4AB95428.1040704@simplistix.co.uk>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<4AB95428.1040704@simplistix.co.uk>
Message-ID: <op.u0okjwuz1q4jlt@whymac.activestate.com>

On Tue, 22 Sep 2009 15:48:08 -0700, Chris Withers <chris at simplistix.co.uk>  
wrote:

>
> Since the language summit at PyCon 2009 various committers, including
> me, have been encouraging Tarek to act as distutils dictator to get
> things finished as we all know people are prone to bikeshedding that
> would kill any forward momentum we have towards improving the
> packaging situation.
>  ...except that this isn't bikeshedding. Use of that term just to stifle  
> productive discussion is also rude.

As I see it, the choice of syntax *as such* is an issue of bikeshedding  
except that the choice of one syntax over the another could make it  
supportive to add an useful feature in setuptools to distutils. For  
instance, this syntax which I proposed -  
http://mail.python.org/pipermail/distutils-sig/2009-September/013289.html  
- will make us think about adding install_requires/extra_requires to  
distutils *in addition to* the conditional metadata .. rather than  
inventing a new syntax to support only conditional metadata.

-srid

From brett at python.org  Wed Sep 23 02:06:49 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 22 Sep 2009 17:06:49 -0700
Subject: [Python-Dev] how to properly object to something (was: Distutils ML
	wrap-up: setup.cfg new format)
Message-ID: <bbaeab100909221706v5187291apa1417c7dd13d2784@mail.gmail.com>

2009/9/22 Chris Withers <chris at simplistix.co.uk>:
> Brett Cannon wrote:
>>
>> OK, stop before it gets any nastier as your email already sounds rude,
>> Chris.
>
> Yes, apologies.

Thanks.

>
>> Since the language summit at PyCon 2009 various committers, including
>> me, have been encouraging Tarek to act as distutils dictator to get
>> things finished as we all know people are prone to bikeshedding that
>> would kill any forward momentum we have towards improving the
>> packaging situation.
>
> ...except that this isn't bikeshedding. Use of that term just to stifle
> productive discussion is also rude.

This sounds like a differing of opinion; Tarek doesn't think it's
critical, you do. Obviously that changes what the two of you view as
bikeshedding on this topic.

I also just checked the thread based on the link you and Sridhar sent
and I didn't see anyone publicly say they preferred Sridhar's version.
If people preferred Sridhar's version they should have spoken up.
While Sridhar's email came in Friday and Tarek pronounced Saturday (as
a reply to one of Sridhar's emails to his idea), that doesn't mean no
one could have spoken up over the the past three days in support of
Sridhar's idea if they preferred it regardless of what Tarek decided;
no code has been committed so decisions can be changed (gods know
plenty of people have said after Guido has pronounced "I disagree" and
no one has ever taken offense). So I don't think it's fair to claim
that Tarek is not listening considering his pronouncement was in reply
to one of the proposals you said he was ignoring.

In other words, Chris, you brought up objections to Tarek's decision
claiming he was ignoring two full days after he made it, and used as
support at least one email which no one publicly showed support for
and for which Tarek indirectly replied to when he made his decision.
So I see no evidence that Tarek is ignoring people's input from where
I stand. If you didn't like Tarek's final decision or preferred
Sridhar's idea then you should have replied to one of those emails
with at least a vote of +1/+0/-0/-1 over on the distutils-SIG, not
here. As of right now this feels like an end-run around Tarek using
extremely weak evidence which, in my opinion, is bikeshedding.

-Brett

From dinov at microsoft.com  Wed Sep 23 02:23:56 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Wed, 23 Sep 2009 00:23:56 +0000
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
Message-ID: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>

Is there a reason or a rule by which CPython reports different error message for different failures to subscript?

For example:

>>> set()[2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'set' object does not support indexing
>>> class c(object): pass
...
>>> c()[2]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'c' object does not support indexing

But compare this to:

>>> [].append[42]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'builtin_function_or_method' object is unsubscriptable
>>> (lambda: 42)[42]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'function' object is unsubscriptable
>>> property()[42]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'property' object is unsubscriptable

IronPython had a bug report that we were getting this wrong for set objects and that "does not support indexing" was also a clearer error message.  I'm wondering if there's some reason why the different error messages occur which I'm missing.   Otherwise I could switch to using the more clear message or start marking types which should report the unsubscriptable error.  Does anyone have any insights into this?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090923/dbb5efab/attachment.htm>

From david.lyon at preisshare.net  Wed Sep 23 02:07:33 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 22 Sep 2009 20:07:33 -0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
Message-ID: <ba13fddccbc97956b7e03f8973c94805@preisshare.net>


Tarek,

Are you claiming this as your own work and ideas?

Given:

http://mail.python.org/pipermail/distutils-sig/2009-August/012998.html

Regards

David


On Tue, 22 Sep 2009 15:21:06 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
wrote:
> Hello
> 
> Here's a wrapup of the Distutils-SIG discussion
> we had on the "static metadata" topic.
> 
> I realize that it's a good thing to send in.
> python-dev such wrapup on distutils design
> decisions, to keep everyone informed and get
> more feedback when required.
> 
> I will try to do it for every upcoming changes
> that are not going in a PEP process (when it's not
> a 'big' change). The rate of such mails should
> not be very high. (around 1 mail in python-dev
> for +150 mails in distutils-SIG)
> 
> If you feel that what we are about to change in distutils
> is wrong, you can go ahead and help us by participating
> in Distutils-ML, so we keep one and only one media
> for these discussions.
> 
> The four sentences summary for people in a hurry:
> 
>     Getting metadata of a distribution that is not.
>     installed means running its setup.py. This means.
>     downloading the whole archive, and running.
>     third party code on your system.
> 
>     To avoid it, we are adding a [setup] section in.
>     setup.cfg where you can express the package metadata
>     statically.
> 
>     Conditional sections, specific to some system.
>     can be added to add some specific fields in [setup].
> 
>     At the end, you will be able to get metadata fields
>     without running any third-party code, and possibly
>     get only the distribution setup.cfg for this.
> 
> Now the long version.
> 
> 
> Rational
> ========
> 
> Today, if you want to list all the metadata (PEP 314) of a.
> distribution that is not installed, you need to use it's.
> setup.py CLI.
> 
> So, basically, you download it, and run::
> 
>     $ python setup.py --name
>     Foo
> 
>     $ python setup.py --version
>     1.2
> 
> Where `name` and `version` are metadata fields. That's OK but as.
> soon as the developers add more code in setup.py, this feature
> might break or (worse) might do unwanted things on the target.
> system.
> 
> Why should we run third-party code just to get its metadata ?
> 
> So we worked on a way to express those metadata statically,
> by pushing them in `setup.cfg`. That's not hard, since all.
> the metadata fields are static values.
> 
> Adding a setup section in setup.cfg
> ===================================
> 
> So the first thing we want to introduce is a [setup] section,
> that may contain any field from the metadata::
> 
>     [setup]
>     name: Foo
>     version: 1.2
> 
> Distutils will look for them, and will use them. Of course
> setup.py is still required, and in case an option that is.
> a metadata field is passed to setup(), it will override one.
> located in setup.cfg.
> 
> PEP 341 is coming up
> ====================
> 
> Remember the last Pycon Summit ? We said that we would
> introduce a new metadata field to describe requirements..
> That's basically what PEP 341 is about, and we are still.
> working on it.
> 
> Basically, we will be able to write things like::
> 
>     requires: Twisted == 8.2.0
> 
> What takes us so long is that adding requirements like
> this in the metadata requires more things:
> 
> - the ability to compare versions (this was described in.
>   PEP 386 but not finished yet)
> 
> - the ability to make these requirements vary depending on.
>   the target system
> 
> And the later makes our "setup.cfg" new [setup] section.
> obsolete as soon as this new metadata field is added in.
> Python.
> 
> So we need more that what I've shown in the previous section
> 
> Context-dependant sections
> ==========================
> 
> The second change we are going to introduce is context-dependant
> sections in setup.cfg.
> 
> A context dependant section is a section with a condition that is.
> used only if the execution environment meets its condition.
> 
> Here's an example::
> 
>     [setup]
>     name: Foo
>     version: 1.3
> 
>     [setup:sys_platform == 'win32']
>     requires: pywin32
>     requires: bar > 1.0
> 
>     [setup:os_machine == '64bits']
>     requires: some_package
> 
>     [setup:python_version == '2.4' or python_version == '2.5']
>     requires: some_package
> 
> 
> Every [setup:condition] section will be added in [setup]
> only if the condition is met.
> 
> The micro-language behind this is the simplest possible:
> it compares only strings, with usual string operators,
> and with the ability to combine expressions. It makes it also
> easy to understand to non-pythoneers (os packagers).
> 
> The pseudo-grammar is (I don't know how to write those but you'll
> get it hopefully)::
> 
>     comp: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'
>     comparison: expr (comp_op expr)*
>     expr: STRING
>     test: or_test
>     or_test: and_test ('or' and_test)*
>     and_test: not_test ('and' not_test)*
>     not_test: 'not' not_test | comparison
> 
> where STRING belongs to any of those:
> 
> - python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])
> - os_name = os.name
> - sys_platform = sys.platform
> - os_sysname = os.uname()[0].
> - os_nodename = os.uname()[1]
> - os_release = os.uname()[2].
> - os_version = os.uname()[3]..
> - os_machine = os.uname()[4].
> - a free string, like '2.4', or 'win32'
> 
> Distutils will provide a function that is able to read the metadata
> of a distribution, given a setup.cfg file, for the target environment::
> 
>     >>> from distutils.util import local_metadata
>     >>> local_metadata('setup.cfg')
>     <DistributionMetadata instance>
> 
> Meaning that a Vanilla Python will be able to read the metadata
> of a package without running any code.
> 
> That's what distutils will use when setup.py is run.
> 
> That's what any package manager will be able to use to work.
> with distributions.
> 
> Limitations
> ===========
> 
> If a distribution is unable for obscure reasons (or does not
> wish) to set all metadata fields in setup.cfg, that's fine :.
> the fields will be set to UNKNOWN when `local_metadata` is called.
> 
> When setup.py is run, options passed to setup() will complete
> those.
> 
> When a package manager gets 'UNKOWN' values, it knows it might
> need to do further processing by running setup.py.
> 
> Why this change is good
> =====================
> 
> Once the requires metadata is added in PEP 341,
> being able to query the metadata for a distribution can be done
> without doing anything else than reading a static file and interpreting
> its conditions in a restricted and secured manner.
> 
> So there's no need to run third-party code anymore.
> 
> Possible use cases:
> 
> - setup.cfg is published on PyPI. Packages managers like easy_install
>   or pip will be able to list all the distributions needed and their
>   versions for installing a distribution without having to download
>   anything !
> 
> - OS packagers will be able to list the dependencies of a distribution
>   without having to browse Python code.
> 
> And, again, this new [setup] section is backward-compatible, and will
> be a good complement to the work done in PEP 376.
> 
> Thanks for reading so far. This code will be added in 2.7 and 3.2.
> 
> Tarek

From fwierzbicki at gmail.com  Wed Sep 23 02:43:22 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Tue, 22 Sep 2009 20:43:22 -0400
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
Message-ID: <4dab5f760909221743l4de4d983s2d5eabe78f052e62@mail.gmail.com>

On Tue, Sep 22, 2009 at 5:55 PM, Dino Viehland <dinov at microsoft.com> wrote:
> For IronPython we wrote a set of tests which go through and define
> the various operator methods in all sorts of combinations on both
> new-style and old-style classes as well as subclasses of those classes
> and then do the comparisons w/ logging.
We've talked about this before, but now I'm reminded again -- do you
happen to have a pointer to these tests just in case someone might
want to grab them :) ?

-Frank

From python at mrabarnett.plus.com  Wed Sep 23 03:01:15 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 23 Sep 2009 02:01:15 +0100
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
Message-ID: <4AB9735B.4000102@mrabarnett.plus.com>

Dino Viehland wrote:
> Is there a reason or a rule by which CPython reports different error 
> message for different failures to subscript?
> 
>  
> 
> For example:
> 
>  
> 
>  >>> set()[2]
> 
> Traceback (most recent call last):
> 
>   File "<stdin>", line 1, in <module>
> 
> TypeError: 'set' object does not support indexing
> 
>  >>> class c(object): pass
> 
> ...
> 
>  >>> c()[2]
> 
> Traceback (most recent call last):
> 
>   File "<stdin>", line 1, in <module>
> 
> TypeError: 'c' object does not support indexing
> 
>  
> 
> But compare this to:
> 
>  
> 
>  >>> [].append[42]
> 
> Traceback (most recent call last):
> 
>   File "<stdin>", line 1, in <module>
> 
> TypeError: 'builtin_function_or_method' object is unsubscriptable
> 
>  >>> (lambda: 42)[42]
> 
> Traceback (most recent call last):
> 
>   File "<stdin>", line 1, in <module>
> 
> TypeError: 'function' object is unsubscriptable
> 
>  >>> property()[42]
> 
> Traceback (most recent call last):
> 
>   File "<stdin>", line 1, in <module>
> 
> TypeError: 'property' object is unsubscriptable
> 
>  
> 
> IronPython had a bug report that we were getting this wrong for set 
> objects and that ?does not support indexing? was also a clearer error 
> message.  I?m wondering if there?s some reason why the different error 
> messages occur which I?m missing.   Otherwise I could switch to using 
> the more clear message or start marking types which should report the 
> unsubscriptable error.  Does anyone have any insights into this?
> 
I thought that the difference might be between iterable and non-iterable
objects, but then I'd expect the class 'c' to behave like the other
non-iterables. In fact, the result is different again if it's an
old-style class:

 >>> class c: pass

 >>> c()[2]

Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
     c()[2]
AttributeError: c instance has no attribute '__getitem__'


From dinov at microsoft.com  Wed Sep 23 03:15:22 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Wed, 23 Sep 2009 01:15:22 +0000
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both 	object have implementations
In-Reply-To: <4dab5f760909221743l4de4d983s2d5eabe78f052e62@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<4dab5f760909221743l4de4d983s2d5eabe78f052e62@mail.gmail.com>
Message-ID: <1A472770E042064698CB5ADC83A12ACD049385AF@TK5EX14MBXC120.redmond.corp.microsoft.com>

Frank wrote:
> > For IronPython we wrote a set of tests which go through and define
> > the various operator methods in all sorts of combinations on both
> > new-style and old-style classes as well as subclasses of those
> classes
> > and then do the comparisons w/ logging.
> We've talked about this before, but now I'm reminded again -- do you
> happen to have a pointer to these tests just in case someone might
> want to grab them :) ?
> 

We include them in the source distribution which you can download the
2.6RC1 release which just came out today here:

http://ironpython.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=30315#DownloadId=84669

All of the side-by-side tests are in:

IronPython-2.6\Src\Tests\compat

Or you can view the source via the web at:

http://ironpython.codeplex.com/SourceControl/ListDownloadableCommits.aspx
(this page also has instructions for Subversion and other source control 
clients)

And the latest version there is in:

IronPython_Main\Src\Tests\compat

Hopefully the infrastructure will just work on Jython because it also runs on
CPython but there may be some Windows specific code in there (e.g. import nt, 
a general problem w/ our tests from the days before we always ran tests w/ 
the CPython std lib present).  

If you run into any problems let me know and I'd be happy to fix it to make 
them more compatible.




From guido at python.org  Wed Sep 23 04:28:41 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 22 Sep 2009 19:28:41 -0700
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <ba13fddccbc97956b7e03f8973c94805@preisshare.net>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<ba13fddccbc97956b7e03f8973c94805@preisshare.net>
Message-ID: <ca471dc20909221928m6ebccb7du2b565da45aaa4901@mail.gmail.com>

David,

Isn't it clear from Tarek's email that he is summarizing
distutils-sig? Isn't it odd that you ask this question pointing to
your own message? Are you worried that you won't get credit for the
idea (assuming it's yours)? Should you perhaps just ask for credit
instead of asking a rhetorical question?

--Guido

On Tue, Sep 22, 2009 at 5:07 PM, David Lyon <david.lyon at preisshare.net> wrote:
>
> Tarek,
>
> Are you claiming this as your own work and ideas?
>
> Given:
>
> http://mail.python.org/pipermail/distutils-sig/2009-August/012998.html
>
> Regards
>
> David
>
>
> On Tue, 22 Sep 2009 15:21:06 +0200, Tarek Ziad? <ziade.tarek at gmail.com>
> wrote:
>> Hello
>>
>> Here's a wrapup of the Distutils-SIG discussion
>> we had on the "static metadata" topic.
>>
>> I realize that it's a good thing to send in.
>> python-dev such wrapup on distutils design
>> decisions, to keep everyone informed and get
>> more feedback when required.
>>
>> I will try to do it for every upcoming changes
>> that are not going in a PEP process (when it's not
>> a 'big' change). The rate of such mails should
>> not be very high. (around 1 mail in python-dev
>> for +150 mails in distutils-SIG)
>>
>> If you feel that what we are about to change in distutils
>> is wrong, you can go ahead and help us by participating
>> in Distutils-ML, so we keep one and only one media
>> for these discussions.
>>
>> The four sentences summary for people in a hurry:
>>
>> ? ? Getting metadata of a distribution that is not.
>> ? ? installed means running its setup.py. This means.
>> ? ? downloading the whole archive, and running.
>> ? ? third party code on your system.
>>
>> ? ? To avoid it, we are adding a [setup] section in.
>> ? ? setup.cfg where you can express the package metadata
>> ? ? statically.
>>
>> ? ? Conditional sections, specific to some system.
>> ? ? can be added to add some specific fields in [setup].
>>
>> ? ? At the end, you will be able to get metadata fields
>> ? ? without running any third-party code, and possibly
>> ? ? get only the distribution setup.cfg for this.
>>
>> Now the long version.
>>
>>
>> Rational
>> ========
>>
>> Today, if you want to list all the metadata (PEP 314) of a.
>> distribution that is not installed, you need to use it's.
>> setup.py CLI.
>>
>> So, basically, you download it, and run::
>>
>> ? ? $ python setup.py --name
>> ? ? Foo
>>
>> ? ? $ python setup.py --version
>> ? ? 1.2
>>
>> Where `name` and `version` are metadata fields. That's OK but as.
>> soon as the developers add more code in setup.py, this feature
>> might break or (worse) might do unwanted things on the target.
>> system.
>>
>> Why should we run third-party code just to get its metadata ?
>>
>> So we worked on a way to express those metadata statically,
>> by pushing them in `setup.cfg`. That's not hard, since all.
>> the metadata fields are static values.
>>
>> Adding a setup section in setup.cfg
>> ===================================
>>
>> So the first thing we want to introduce is a [setup] section,
>> that may contain any field from the metadata::
>>
>> ? ? [setup]
>> ? ? name: Foo
>> ? ? version: 1.2
>>
>> Distutils will look for them, and will use them. Of course
>> setup.py is still required, and in case an option that is.
>> a metadata field is passed to setup(), it will override one.
>> located in setup.cfg.
>>
>> PEP 341 is coming up
>> ====================
>>
>> Remember the last Pycon Summit ? We said that we would
>> introduce a new metadata field to describe requirements..
>> That's basically what PEP 341 is about, and we are still.
>> working on it.
>>
>> Basically, we will be able to write things like::
>>
>> ? ? requires: Twisted == 8.2.0
>>
>> What takes us so long is that adding requirements like
>> this in the metadata requires more things:
>>
>> - the ability to compare versions (this was described in.
>> ? PEP 386 but not finished yet)
>>
>> - the ability to make these requirements vary depending on.
>> ? the target system
>>
>> And the later makes our "setup.cfg" new [setup] section.
>> obsolete as soon as this new metadata field is added in.
>> Python.
>>
>> So we need more that what I've shown in the previous section
>>
>> Context-dependant sections
>> ==========================
>>
>> The second change we are going to introduce is context-dependant
>> sections in setup.cfg.
>>
>> A context dependant section is a section with a condition that is.
>> used only if the execution environment meets its condition.
>>
>> Here's an example::
>>
>> ? ? [setup]
>> ? ? name: Foo
>> ? ? version: 1.3
>>
>> ? ? [setup:sys_platform == 'win32']
>> ? ? requires: pywin32
>> ? ? requires: bar > 1.0
>>
>> ? ? [setup:os_machine == '64bits']
>> ? ? requires: some_package
>>
>> ? ? [setup:python_version == '2.4' or python_version == '2.5']
>> ? ? requires: some_package
>>
>>
>> Every [setup:condition] section will be added in [setup]
>> only if the condition is met.
>>
>> The micro-language behind this is the simplest possible:
>> it compares only strings, with usual string operators,
>> and with the ability to combine expressions. It makes it also
>> easy to understand to non-pythoneers (os packagers).
>>
>> The pseudo-grammar is (I don't know how to write those but you'll
>> get it hopefully)::
>>
>> ? ? comp: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'
>> ? ? comparison: expr (comp_op expr)*
>> ? ? expr: STRING
>> ? ? test: or_test
>> ? ? or_test: and_test ('or' and_test)*
>> ? ? and_test: not_test ('and' not_test)*
>> ? ? not_test: 'not' not_test | comparison
>>
>> where STRING belongs to any of those:
>>
>> - python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])
>> - os_name = os.name
>> - sys_platform = sys.platform
>> - os_sysname = os.uname()[0].
>> - os_nodename = os.uname()[1]
>> - os_release = os.uname()[2].
>> - os_version = os.uname()[3]..
>> - os_machine = os.uname()[4].
>> - a free string, like '2.4', or 'win32'
>>
>> Distutils will provide a function that is able to read the metadata
>> of a distribution, given a setup.cfg file, for the target environment::
>>
>> ? ? >>> from distutils.util import local_metadata
>> ? ? >>> local_metadata('setup.cfg')
>> ? ? <DistributionMetadata instance>
>>
>> Meaning that a Vanilla Python will be able to read the metadata
>> of a package without running any code.
>>
>> That's what distutils will use when setup.py is run.
>>
>> That's what any package manager will be able to use to work.
>> with distributions.
>>
>> Limitations
>> ===========
>>
>> If a distribution is unable for obscure reasons (or does not
>> wish) to set all metadata fields in setup.cfg, that's fine :.
>> the fields will be set to UNKNOWN when `local_metadata` is called.
>>
>> When setup.py is run, options passed to setup() will complete
>> those.
>>
>> When a package manager gets 'UNKOWN' values, it knows it might
>> need to do further processing by running setup.py.
>>
>> Why this change is good
>> =====================
>>
>> Once the requires metadata is added in PEP 341,
>> being able to query the metadata for a distribution can be done
>> without doing anything else than reading a static file and interpreting
>> its conditions in a restricted and secured manner.
>>
>> So there's no need to run third-party code anymore.
>>
>> Possible use cases:
>>
>> - setup.cfg is published on PyPI. Packages managers like easy_install
>> ? or pip will be able to list all the distributions needed and their
>> ? versions for installing a distribution without having to download
>> ? anything !
>>
>> - OS packagers will be able to list the dependencies of a distribution
>> ? without having to browse Python code.
>>
>> And, again, this new [setup] section is backward-compatible, and will
>> be a good complement to the work done in PEP 376.
>>
>> Thanks for reading so far. This code will be added in 2.7 and 3.2.
>>
>> Tarek
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rdmurray at bitdance.com  Wed Sep 23 05:08:07 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 22 Sep 2009 23:08:07 -0400 (EDT)
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <4AB9735B.4000102@mrabarnett.plus.com>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<4AB9735B.4000102@mrabarnett.plus.com>
Message-ID: <Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net>

On Wed, 23 Sep 2009 at 02:01, MRAB wrote:
> Dino Viehland wrote:
>>  Is there a reason or a rule by which CPython reports different error
>>  message for different failures to subscript?
>>
>>  For example:
>> 
>> > > >  set()[2]
>>  Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>>  TypeError: 'set' object does not support indexing
>> > > >  class c(object): pass
>>  ...
[....]
>> > > >  [].append[42]
>>  Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>>  TypeError: 'builtin_function_or_method' object is unsubscriptable
[...]
>>  IronPython had a bug report that we were getting this wrong for set
>>  objects and that ?does not support indexing? was also a clearer error
>>  message.  I?m wondering if there?s some reason why the different error
>>  messages occur which I?m missing.   Otherwise I could switch to using the
>>  more clear message or start marking types which should report the
>>  unsubscriptable error.  Does anyone have any insights into this?
>> 
> I thought that the difference might be between iterable and non-iterable
> objects, but then I'd expect the class 'c' to behave like the other
> non-iterables. In fact, the result is different again if it's an
> old-style class:
>
>> > >  class c: pass
>> > >  c()[2]
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>    c()[2]
> AttributeError: c instance has no attribute '__getitem__'

Looking at the source, these messages are generated from abstract.c, and
the difference appears to be whether or not the tp_as_sequence slot is
filled in or not.  If it is, but there is no sq_item method, then
PySequence_GetItem gives the "does not support indexing" message.
If it isn't filled in, you get the 'not subscriptable"(*) message from
PyObject_GetItem.

The logic appears to be, roughly, if an object does not have mapping
methods, or has them but does not have mp_subscript, and does have
sequence methods but does not have sq_item, then you get a 'does
not support indexing'.  Otherwise you get 'not subscriptable'.

The question is, is this a useful distinction, or is it an
implementation artifact?

--David

(*) The error message changed format slightly in 2.7 due to
http://bugs.python.org/issue5760, and the discussion there is
worth reading in this context.

From david.lyon at preisshare.net  Wed Sep 23 05:55:29 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Tue, 22 Sep 2009 23:55:29 -0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <ca471dc20909221928m6ebccb7du2b565da45aaa4901@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<ba13fddccbc97956b7e03f8973c94805@preisshare.net>
	<ca471dc20909221928m6ebccb7du2b565da45aaa4901@mail.gmail.com>
Message-ID: <3e228a5145369e7d9932040c73a7d58e@preisshare.net>


Hello Guido, 

> Isn't it clear from Tarek's email that he is summarizing
> distutils-sig? 

Well, I'm new to this. Summarising is a word that could be 
certainly applied.

> Isn't it odd that you ask this question pointing to
> your own message? 

It's not odd at all. When I see my ideas, which Tarek originally
argued against, published with his name on them, and my name left
out, I just felt it fair to myself to ask him how that might have 
happened.

> Are you worried that you won't get credit for the
> idea (assuming it's yours)? 

As I said, I'm new. I've been very happy to help out. This is
all a learning process for me.

I can accept whatever decision Tarek makes on distutils and the
direction of the code. If he chose to ignore my postings and
go a different way from what I suggested then that is his 
perogative.

If the failing was on my part to not fully write up PEPS and
do full examples, and diagrams and so forth, then I accept that.

> Should you perhaps just ask for credit instead of asking a 
> rhetorical question?

I don't really believe that the work is complete enough for
anybody to claim credit for just yet. That's why I was surprised 
to see the announcement. Hence me asking about it.

In all fairness, I'm trying to help Tarek here. And the python
packaging effort.

Getting my name on what I believe are my contributions shouldn't 
be such a big deal and I'm very sure you'd agree on that. But it
is an ongoing struggle for whatever reason that I've been having.

I'm sure your guidance can resolve the issues easily..

David




From dinov at microsoft.com  Wed Sep 23 06:23:59 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Wed, 23 Sep 2009 04:23:59 +0000
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<4AB9735B.4000102@mrabarnett.plus.com>
	<Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net>
Message-ID: <1A472770E042064698CB5ADC83A12ACD04939586@TK5EX14MBXC120.redmond.corp.microsoft.com>

R. David Murray wrote:
> Looking at the source, these messages are generated from abstract.c,
> and the difference appears to be whether or not the tp_as_sequence slot
> is filled in or not.  If it is, but there is no sq_item method, then
> PySequence_GetItem gives the "does not support indexing" message.
> If it isn't filled in, you get the 'not subscriptable"(*) message from
> PyObject_GetItem.
> 
> The logic appears to be, roughly, if an object does not have mapping
> methods, or has them but does not have mp_subscript, and does have
> sequence methods but does not have sq_item, then you get a 'does not
> support indexing'.  Otherwise you get 'not subscriptable'.
> 
> The question is, is this a useful distinction, or is it an
> implementation artifact?

First on the useful distinction: for better or worse users do seem to
have an affinity towards CPython's error messages and especially so 
when they're better than ours :).  But I certainly hope no one is taking 
a dependency on them by parsing them or anything like that such that it
would actually break someone's program.  So basically I regard all
error messages as implementation artifacts but I have no problem matching
them if it'll make someone happier and it's not too crazy to do it. 

That being said there does seem to be a useful distinction here.  It
sounds like the idea here is to report a better error message when
something is more collection like than a full blown sequence - in
particular I'm thinking of .NET's ICollection and Java's Collection 
interfaces that are very sequence like but don't support getting 
elements by index.

The only oddity in the error message to me is that user defined objects 
are effectively always treated as sequences.  That's completely an
implementation artifact but I'm going to copy it too because it's easy,
may prevent another bug report, and checking for all the various 
sequence methods seems like unnecessary overkill.

Thanks for the information!


From greg.ewing at canterbury.ac.nz  Wed Sep 23 06:27:21 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 23 Sep 2009 16:27:21 +1200
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
Message-ID: <4AB9A3A9.1090800@canterbury.ac.nz>

Willem Broekema wrote:

> The AND above (which I uppercased) is subtle but important. In the "x
> op y" case with y being of a subclass of the class of x, if there is
> no class in between x and y (excluding x, including y) that overrides
> the __rop__ method, then y,__rop__(x) is *not* tried before
> x.__op__(y).

How does this work at the C typeslot level, where
there are no __rop__ methods?

-- 
Greg

From brett at python.org  Wed Sep 23 07:39:04 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 22 Sep 2009 22:39:04 -0700
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <3e228a5145369e7d9932040c73a7d58e@preisshare.net>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<ba13fddccbc97956b7e03f8973c94805@preisshare.net>
	<ca471dc20909221928m6ebccb7du2b565da45aaa4901@mail.gmail.com> 
	<3e228a5145369e7d9932040c73a7d58e@preisshare.net>
Message-ID: <bbaeab100909222239ge93c51dr89f14dd178567cc1@mail.gmail.com>

On Tue, Sep 22, 2009 at 20:55, David Lyon <david.lyon at preisshare.net> wrote:
>
> Hello Guido,
>
>> Isn't it clear from Tarek's email that he is summarizing
>> distutils-sig?
>
> Well, I'm new to this. Summarising is a word that could be
> certainly applied.
>
>> Isn't it odd that you ask this question pointing to
>> your own message?
>
> It's not odd at all. When I see my ideas, which Tarek originally
> argued against, published with his name on them, and my name left
> out, I just felt it fair to myself to ask him how that might have
> happened.
>
>> Are you worried that you won't get credit for the
>> idea (assuming it's yours)?
>
> As I said, I'm new. I've been very happy to help out. This is
> all a learning process for me.
>
> I can accept whatever decision Tarek makes on distutils and the
> direction of the code. If he chose to ignore my postings and
> go a different way from what I suggested then that is his
> perogative.
>
> If the failing was on my part to not fully write up PEPS and
> do full examples, and diagrams and so forth, then I accept that.
>
>> Should you perhaps just ask for credit instead of asking a
>> rhetorical question?
>
> I don't really believe that the work is complete enough for
> anybody to claim credit for just yet. That's why I was surprised
> to see the announcement. Hence me asking about it.
>
> In all fairness, I'm trying to help Tarek here. And the python
> packaging effort.
>
> Getting my name on what I believe are my contributions shouldn't
> be such a big deal and I'm very sure you'd agree on that. But it
> is an ongoing struggle for whatever reason that I've been having.
>
> I'm sure your guidance can resolve the issues easily..

If you re-read Tarek's email is uses "we" throughout the entire
discussion, only using "I" when referring to the email itself. I don't
think anyone on python-dev thought that it was exclusively Tarek's
idea or necessarily even mostly his. Personally I just attribute the
idea to the distutils-SIG and don't think beyond that.

If you want explicit credit, you can co-author a PEP or get thanked in
a checkin as you mentioned. But honestly, from my observations of open
source, ideas are not what get you noticed, it's producing something
tangible like code. Plus ideas and such pull from so many people on
mailing lists you can claim you came up with the initial idea, but I
am sure a ton of people provided feedback which makes ownership of any
idea practically moot. Trust me, if you are doing open source for
anything other than altruistic reasons you are bound to be
disappointed.

-Brett

From david.lyon at preisshare.net  Wed Sep 23 08:20:47 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 23 Sep 2009 02:20:47 -0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <bbaeab100909222239ge93c51dr89f14dd178567cc1@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<ba13fddccbc97956b7e03f8973c94805@preisshare.net>
	<ca471dc20909221928m6ebccb7du2b565da45aaa4901@mail.gmail.com>
	<3e228a5145369e7d9932040c73a7d58e@preisshare.net>
	<bbaeab100909222239ge93c51dr89f14dd178567cc1@mail.gmail.com>
Message-ID: <b15493e175655b4c72b4b8b83b7a34eb@preisshare.net>

On Tue, 22 Sep 2009 22:39:04 -0700, Brett Cannon <brett at python.org> wrote:
> I don't think anyone on python-dev thought that it was exclusively
Tarek's
> idea or necessarily even mostly his. 

Actually, he originally argued against it, but that is irrelevant.

I'm happy that he's come around and embraced it. Perhaps a bit too
much could be my issue.

> If you want explicit credit, you can co-author a PEP or get thanked in
> a checkin as you mentioned. 

Good idea.

How can I get check-in privileges on distutils ? What is the process?

> But honestly, from my observations of open
> source, ideas are not what get you noticed, it's producing something
> tangible like code. 

Sure.. you're 100% right. So I need to be able to work on code and
be able to check it in. I would love that. How do I get that?

> Plus ideas and such pull from so many people on
> mailing lists you can claim you came up with the initial idea, but I
> am sure a ton of people provided feedback which makes ownership of any
> idea practically moot. 

Claiming ownership? No No. Lets not go there.

A worklist and checkin rights would completely suffice.

David


From brett at python.org  Wed Sep 23 08:42:36 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 22 Sep 2009 23:42:36 -0700
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <b15493e175655b4c72b4b8b83b7a34eb@preisshare.net>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<ba13fddccbc97956b7e03f8973c94805@preisshare.net>
	<ca471dc20909221928m6ebccb7du2b565da45aaa4901@mail.gmail.com> 
	<3e228a5145369e7d9932040c73a7d58e@preisshare.net>
	<bbaeab100909222239ge93c51dr89f14dd178567cc1@mail.gmail.com> 
	<b15493e175655b4c72b4b8b83b7a34eb@preisshare.net>
Message-ID: <bbaeab100909222342t14517d67k1d4360c53c620754@mail.gmail.com>

On Tue, Sep 22, 2009 at 23:20, David Lyon <david.lyon at preisshare.net> wrote:
> On Tue, 22 Sep 2009 22:39:04 -0700, Brett Cannon <brett at python.org> wrote:
>> I don't think anyone on python-dev thought that it was exclusively
> Tarek's
>> idea or necessarily even mostly his.
>
> Actually, he originally argued against it, but that is irrelevant.
>
> I'm happy that he's come around and embraced it. Perhaps a bit too
> much could be my issue.
>
>> If you want explicit credit, you can co-author a PEP or get thanked in
>> a checkin as you mentioned.
>
> Good idea.
>
> How can I get check-in privileges on distutils ? What is the process?
>

Contribute patches on the Python issue tracker for six months to a
year (depends on how much you contribute) and be generally liked by
other committers. Distutils is not separate from Python itself.

But notice I said being thanked in a checkin, not making a checkin yourself.

>> But honestly, from my observations of open
>> source, ideas are not what get you noticed, it's producing something
>> tangible like code.
>
> Sure.. you're 100% right. So I need to be able to work on code and
> be able to check it in. I would love that. How do I get that?
>

You are conflating contributing code and checking it in. You can do
the former without the latter.

>> Plus ideas and such pull from so many people on
>> mailing lists you can claim you came up with the initial idea, but I
>> am sure a ton of people provided feedback which makes ownership of any
>> idea practically moot.
>
> Claiming ownership? No No. Lets not go there.
>
> A worklist and checkin rights would completely suffice.

Talk to Tarek or look at bugs.python.org for a worklist; checkin
rights come MUCH later.

-Brett

From stephen at xemacs.org  Wed Sep 23 09:03:45 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 23 Sep 2009 16:03:45 +0900
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
Message-ID: <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>

Brett Cannon writes:

 > OK, stop before it gets any nastier as your email already sounds
 > rude

Rude, yes, but not entirely lacking in truth.  Tarek's post to Python-
Dev *is* a mess from the point of view of English and organization --
it's *way* below Tarek's normal command of the language.  Granted,
something that complex is much harder to polish than a 3-sentence
post, but it could have been run by a native speaker before posting.
Better yet, it could have been run by Distutils-SIG first.

At the very least, that would have kept this discussion on Distutils-
SIG, and Chris couldn't be accused of trying to make an end run around
that process.  I suggest that posting progress reports to Python-Dev
is a good idea (attracting attention and maybe participation to the
process), but making one an opportunity to test the degree of internal
consensus (or lack of it) on Distutils-SIG by posting there first is
an even better one.

 > I see no reason why [Tarek] should not be able to short-circuit
 > discussions when he sees them as having gone on long enough when he
 > is the one that is going to ultimately end up doing all the coding.

Of course there are benefits to having a strong impartial chairman,
and installing a benevolent dictator is often a second-best strategy
for project management.  However, in this case, the question of what
is being short-circuited is important.  If it's bikeshedding, that's
good.  If it's a substantial difference, it's not.  And it matters,
because the benefits of a New! Improved! distutils! really start
piling up when adoption becomes nearly universal.

How do you tell the difference?  Simple.  If Philip wants the bikeshed
painted pink, but he'll still move his bike to Tarek's blue bikeshed
from the existing pink one, further discussion at this point is
bikeshedding.  If Chris wants the new bikeshed painted chartreuse, and
he'll keep parking in the existing green one if Tarek paints the new
one blue, that's a real difference.  (This assumes Chris represents a
sizable group of users/projects, of course.)


From mal at egenix.com  Wed Sep 23 09:49:16 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 23 Sep 2009 09:49:16 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
Message-ID: <4AB9D2FC.6040705@egenix.com>

Tarek Ziad? wrote:
> Hello
> 
> Here's a wrapup of the Distutils-SIG discussion
> we had on the "static metadata" topic.
> 
> I realize that it's a good thing to send in.
> python-dev such wrapup on distutils design
> decisions, to keep everyone informed and get
> more feedback when required.
> 
> I will try to do it for every upcoming changes
> that are not going in a PEP process (when it's not
> a 'big' change). The rate of such mails should
> not be very high. (around 1 mail in python-dev
> for +150 mails in distutils-SIG)
> 
> If you feel that what we are about to change in distutils
> is wrong, you can go ahead and help us by participating
> in Distutils-ML, so we keep one and only one media
> for these discussions.
> 
> The four sentences summary for people in a hurry:
> 
>     Getting metadata of a distribution that is not.
>     installed means running its setup.py. This means.
>     downloading the whole archive, and running.
>     third party code on your system.
> 
>     To avoid it, we are adding a [setup] section in.
>     setup.cfg where you can express the package metadata
>     statically.
> 
>     Conditional sections, specific to some system.
>     can be added to add some specific fields in [setup].
> 
>     At the end, you will be able to get metadata fields
>     without running any third-party code, and possibly
>     get only the distribution setup.cfg for this.

While it's a good idea to put up some form of meta-data
into an index, I wonder why you are using setup.cfg
for this.

setup.cfg has traditionally been used to configure distutils,
not to define meta-data. As such you wouldn't want to
put such a configuration file up on PyPI.

Wouldn't it be better use a new file for this (with the
same syntax), e.g. metadata.cfg ?! This would then just
contain the meta data that needs to be published.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 23 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From chris at simplistix.co.uk  Wed Sep 23 10:12:21 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 23 Sep 2009 09:12:21 +0100
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both 	object have implementations
In-Reply-To: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>	
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>	
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
Message-ID: <4AB9D865.1070701@simplistix.co.uk>

Mark Dickinson wrote:
> I (still :-) think this is covered, for Python 2.x at least, by:
> 
> http://docs.python.org/reference/datamodel.html#coercion-rules

But this isn't coercion! :-)

> - For objects x and y, first x.__op__(y) is tried. If this is not
> implemented or returns NotImplemented, y.__rop__(x) is tried. 

Also, the above is not so:

Python 2.5.1
 >>> class X:
...   def __eq__(self,other):
...     print "X __eq__"
 >>> class Z: pass
...
 >>> Z()==X()
X __eq__

No __req__ in sight...

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From floris.bruynooghe at gmail.com  Wed Sep 23 10:37:45 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Wed, 23 Sep 2009 09:37:45 +0100
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <4AB9D2FC.6040705@egenix.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB9D2FC.6040705@egenix.com>
Message-ID: <20090923083745.GA14012@laurie.devork>

On Wed, Sep 23, 2009 at 09:49:16AM +0200, M.-A. Lemburg wrote:
> While it's a good idea to put up some form of meta-data
> into an index, I wonder why you are using setup.cfg
> for this.
> 
> setup.cfg has traditionally been used to configure distutils,
> not to define meta-data. As such you wouldn't want to
> put such a configuration file up on PyPI.
> 
> Wouldn't it be better use a new file for this (with the
> same syntax), e.g. metadata.cfg ?! This would then just
> contain the meta data that needs to be published.

IIRC the goal was to make minimal changes and setup.cfg is known
already.  It's use is similar too: it configures the setup.py script
with static data.

PyPI could only extract the [setup:*] sections if you'r worried about
it having random configuration from developers.  I don't think this is
enough of a argument in favour or not of a metadata file separate from
setup.cfg.

Regards
Floris

-- 
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

From skip at pobox.com  Wed Sep 23 05:43:59 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 22 Sep 2009 22:43:59 -0500
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
Message-ID: <19129.39295.970536.916767@montanaro.dyndns.org>


    Dino> For IronPython we wrote a set of tests which go through and define
    Dino> the various operator methods in all sorts of combinations on both
    Dino> new-style and old-style classes as well as subclasses of those
    Dino> classes and then do the comparisons w/ logging.

It would be very nice if these complex corner cases had a set of test
cases which could be run by all implementations (CPython, Jython,
IronPython, PyPy, etc).  I don't know.  Maybe the CPython test suite serves
that purpose, but it seems like it would be helpful if this sort of
"validation suite" was maintained as a separate project all implementations
could use and contribute to.

Skip

From dickinsm at gmail.com  Wed Sep 23 13:05:31 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 23 Sep 2009 12:05:31 +0100
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <4AB9D865.1070701@simplistix.co.uk>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<4AB9D865.1070701@simplistix.co.uk>
Message-ID: <5c6f2a5d0909230405k9a0cd8ew8b36c9b147145632@mail.gmail.com>

On Wed, Sep 23, 2009 at 9:12 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> Mark Dickinson wrote:
>>
>> I (still :-) think this is covered, for Python 2.x at least, by:
>>
>> http://docs.python.org/reference/datamodel.html#coercion-rules
>
> But this isn't coercion! :-)

Agreed.  FWIW this behaviour for arithmetic operations is also mentioned in

http://docs.python.org/reference/datamodel.html#emulating-numeric-types

but then again that section doesn't include the comparison operators.

>
>> - For objects x and y, first x.__op__(y) is tried. If this is not
>> implemented or returns NotImplemented, y.__rop__(x) is tried.
>
> Also, the above is not so:
>
> Python 2.5.1
>>>> class X:
> ... ? def __eq__(self,other):
> ... ? ? print "X __eq__"
>>>> class Z: pass
> ...
>>>> Z()==X()
> X __eq__
>
> No __req__ in sight...

Okay, so combine this with the sentence under:

http://docs.python.org/reference/datamodel.html#object.__eq__

that says:

"There are no swapped-argument versions of these methods (to be used
when the left argument does not support the operation but the right
argument does); rather, __lt__() and __gt__() are each other?s
reflection, __le__() and __ge__() are each other?s reflection, and
__eq__() and __ne__() are their own reflection."

So in the earlier doc snippets, if __op__ is __eq__, then __rop__
should also be interpreted as __eq__;  similarly if __op__ is __lt__
then __rop__ is __gt__.

I'm not saying that the documentation here couldn't be improved;  just
that IMO the docs do (with a little bit of extrapolation) describe
what should happen, giving the 'official' specification that you were
after.

I don't know where/whether the behaviour for classes that define both
__cmp__ and __eq__ are documented, though, and I'm far from sure what
the rules are in that case.

One minor oddity is that for arithmetic operations like __add__(self,
other), if type(self) == type(other) then __radd__ isn't checked, on
the basis that if __add__ fails then the operation is presumably not
supported.  This makes sense, but I wonder why this doesn't apply
equally well to __eq__:  that is, in doing A() == A() for a class A,
why do we try the __eq__ method of both instances?

Mark

From dickinsm at gmail.com  Wed Sep 23 13:06:09 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 23 Sep 2009 12:06:09 +0100
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <19129.39295.970536.916767@montanaro.dyndns.org>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
Message-ID: <5c6f2a5d0909230406q217eb528oe52902d86f17a0e2@mail.gmail.com>

On Wed, Sep 23, 2009 at 4:43 AM,  <skip at pobox.com> wrote:
>
> ? ?Dino> For IronPython we wrote a set of tests which go through and define
> ? ?Dino> the various operator methods in all sorts of combinations on both
> ? ?Dino> new-style and old-style classes as well as subclasses of those
> ? ?Dino> classes and then do the comparisons w/ logging.
>
> It would be very nice if these complex corner cases had a set of test
> cases which could be run by all implementations (CPython, Jython,
> IronPython, PyPy, etc). ?I don't know. ?Maybe the CPython test suite serves
> that purpose, but it seems like it would be helpful if this sort of
> "validation suite" was maintained as a separate project all implementations
> could use and contribute to.

+1

Mark

From solipsis at pitrou.net  Wed Sep 23 13:36:49 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 23 Sep 2009 11:36:49 +0000 (UTC)
Subject: [Python-Dev] altruism
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<ba13fddccbc97956b7e03f8973c94805@preisshare.net>
	<ca471dc20909221928m6ebccb7du2b565da45aaa4901@mail.gmail.com>
	<3e228a5145369e7d9932040c73a7d58e@preisshare.net>
	<bbaeab100909222239ge93c51dr89f14dd178567cc1@mail.gmail.com>
Message-ID: <loom.20090923T132003-720@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:
> 
> Trust me, if you are doing open source for
> anything other than altruistic reasons you are bound to be
> disappointed.

I'm surprised by this statement but perhaps it's a matter of vocabulary.
Having fun and doing things you like to do does not strike me as "altruistic".
Being involved in a FOSS project is not the same as participating in a charity.

See this quick presentation about motivation of FOSS developers (second slide
says: "Altruism? Not really"):
http://www.infonomics.nl/FLOSS/papers/20030619/index.htm
The full study is at:
http://www.infonomics.nl/FLOSS/report/index.htm
("Part IV: Survey of Developers")

(but of course there's the question of whether "altruism" exists at large, or is
a political and moral fiction)

Regards

Antoine.



From g.brandl at gmx.net  Wed Sep 23 13:48:21 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 23 Sep 2009 13:48:21 +0200
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <4AB8E136.8040706@simplistix.co.uk>
References: <4AB8D9F2.6050807@simplistix.co.uk>	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
Message-ID: <h9d1s3$2s7$1@ger.gmane.org>

Chris Withers schrieb:
> Jared Flatow wrote:
>> This might help:
>> 
>> http://mail.python.org/pipermail/python-dev/2008-June/080111.html
>> 
>> Here is the most relevant part (quoting Guido):
>> 
>>  > Does it help if I tell you that for "x <binop> y" we always try
>>  > x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the
>>  > case where y is an instance of a subclass of the class of x?
> 
> Okay, but does that count as a pronouncement that should go across all 
> versions and platforms?

Yes. Unfortunately it's just documented for the other binops, but not
for the comparisons; the note in the docs should be copied to the
respective section.

Georg


From ziade.tarek at gmail.com  Wed Sep 23 13:58:54 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 23 Sep 2009 13:58:54 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <op.u0okjwuz1q4jlt@whymac.activestate.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<4AB95428.1040704@simplistix.co.uk>
	<op.u0okjwuz1q4jlt@whymac.activestate.com>
Message-ID: <94bdd2610909230458g27fb8229q58e0325e1418eb88@mail.gmail.com>

On Wed, Sep 23, 2009 at 12:53 AM, Sridhar Ratnakumar
<sridharr at activestate.com> wrote:
> On Tue, 22 Sep 2009 15:48:08 -0700, Chris Withers <chris at simplistix.co.uk>
> wrote:
>
>>
>> Since the language summit at PyCon 2009 various committers, including
>> me, have been encouraging Tarek to act as distutils dictator to get
>> things finished as we all know people are prone to bikeshedding that
>> would kill any forward momentum we have towards improving the
>> packaging situation.
>> ?...except that this isn't bikeshedding. Use of that term just to stifle
>> productive discussion is also rude.
>
> As I see it, the choice of syntax *as such* is an issue of bikeshedding
> except that the choice of one syntax over the another could make it
> supportive to add an useful feature in setuptools to distutils. For
> instance, this syntax which I proposed -
> http://mail.python.org/pipermail/distutils-sig/2009-September/013289.html -
> will make us think about adding install_requires/extra_requires to distutils
> *in addition to* the conditional metadata .. rather than inventing a new
> syntax to support only conditional metadata.

Please keep in mind that the original proposal is having the ability to
get static metadata (metadata: PEP 314) without running any code.

Your mail contains two things:

- yet another way to express the conditions  (that's the bikeshedding)
- a setuptools mechanism located in setup.py that works with an option
to build a
  requires.txt file when you install a package or create a binary distribution.

The later is not related to what we want to do, or if it is, please
refine your proposal
and make it clear at Distutils-SIG.


Tarek
-- 
Tarek Ziad? | http://ziade.org |????????????!

From ziade.tarek at gmail.com  Wed Sep 23 14:05:04 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 23 Sep 2009 14:05:04 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <4AB95428.1040704@simplistix.co.uk>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<4AB95428.1040704@simplistix.co.uk>
Message-ID: <94bdd2610909230505t7bfebca8lf6cba182878e53fa@mail.gmail.com>

2009/9/23 Chris Withers <chris at simplistix.co.uk>:
>> Since the language summit at PyCon 2009 various committers, including
>> me, have been encouraging Tarek to act as distutils dictator to get
>> things finished as we all know people are prone to bikeshedding that
>> would kill any forward momentum we have towards improving the
>> packaging situation.
>
> ...except that this isn't bikeshedding. Use of that term just to stifle
> productive discussion is also rude.


- choosing if we are going to use "py<2.6" instead of
"python_version<2.6" is bikeshedding.

- explaining that os.uname() contains irrelevant infos and we
shouldn't add it as-is in the
  execution context is not bikeshedding (see Ronald last mail in the thread)

Please go back to the thread at distutils-SIG and list all points in
the proposal
that seems to you as non-bikeshedding things.

Tarek
-- 
Tarek Ziad? | http://ziade.org |????????????!

From ncoghlan at gmail.com  Wed Sep 23 14:05:29 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 23 Sep 2009 22:05:29 +1000
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both 	object have implementations
In-Reply-To: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
Message-ID: <4ABA0F09.9060703@gmail.com>

Mark Dickinson wrote:
> On Tue, Sep 22, 2009 at 3:37 PM, Chris Withers
> <chris at simplistix.co.uk> wrote:
>> Where are the specifications on what happens if two objects are
>> compared and both have implementations of __eq__? Which __eq__ is
>> called? What happens if the first one called returns False? Is the
>> second one called? What is one implements __eq__ and the other
>> __ne__?
> 
> I (still :-) think this is covered, for Python 2.x at least, by:
> 
> http://docs.python.org/reference/datamodel.html#coercion-rules

Quite some time ago, I attempted to accurately document all of this - my
efforts are buried in an ODF document in the SVN sandbox:

http://svn.python.org/view/sandbox/trunk/userref/ODF/Chapter02_StatementsAndExpressions.odt

Extracting all of that text with some kind of ODT2ReST converter has now
been on my to-do list for over a year :P

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Wed Sep 23 14:09:13 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 23 Sep 2009 22:09:13 +1000
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <4AB9A3A9.1090800@canterbury.ac.nz>
References: <4AB8D9F2.6050807@simplistix.co.uk>	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>	<4AB8E136.8040706@simplistix.co.uk>	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<4AB9A3A9.1090800@canterbury.ac.nz>
Message-ID: <4ABA0FE9.7020401@gmail.com>

Greg Ewing wrote:
> Willem Broekema wrote:
> 
>> The AND above (which I uppercased) is subtle but important. In the "x
>> op y" case with y being of a subclass of the class of x, if there is
>> no class in between x and y (excluding x, including y) that overrides
>> the __rop__ method, then y,__rop__(x) is *not* tried before
>> x.__op__(y).
> 
> How does this work at the C typeslot level, where
> there are no __rop__ methods?

The interpreter doesn't promise to call those slots with "self" first -
self will be the second argument in the "rop" case.

See binary_op1() in abstract.c for the gory details (I had to go look it
up myself in order to rediscover how it worked).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ziade.tarek at gmail.com  Wed Sep 23 14:20:31 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 23 Sep 2009 14:20:31 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>

On Wed, Sep 23, 2009 at 9:03 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> At the very least, that would have kept this discussion on Distutils-
> SIG, and Chris couldn't be accused of trying to make an end run around
> that process. ?I suggest that posting progress reports to Python-Dev
> is a good idea (attracting attention and maybe participation to the
> process), but making one an opportunity to test the degree of internal
> consensus (or lack of it) on Distutils-SIG by posting there first is
> an even better one.

Please define what "internal consensus on Distutils-SIG" means.

If it means that everyone present in the Mailing-List
needs to agree on a distutils proposal, this will never happen for many reasons.

Here's one :

That's not the distutils mailing list anymore. Here's a list of the projects
"officialy" hosted in this mailing list:

- distutils
- setuptools
- zc.buildout

So, while it's legitimate to get some feedback from setuptools and
zc.buildout users
and developers, (on anyone else) on what we are planning for
distutils, don't you agree that
it's impossible in this context to get a 100% consensus ? and that I need to
take some decisions to move distutils on ?

Especially for this topic. If you take the time to read everything you'll see
that there were no real alternative design proposed, and that I am
just working out
the details because I maintain and code distutils.

Tarek
-- 
Tarek Ziad? | http://ziade.org |????????????!

From ncoghlan at gmail.com  Wed Sep 23 14:22:46 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 23 Sep 2009 22:22:46 +1000
Subject: [Python-Dev] altruism
In-Reply-To: <loom.20090923T132003-720@post.gmane.org>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<ba13fddccbc97956b7e03f8973c94805@preisshare.net>	<ca471dc20909221928m6ebccb7du2b565da45aaa4901@mail.gmail.com>	<3e228a5145369e7d9932040c73a7d58e@preisshare.net>	<bbaeab100909222239ge93c51dr89f14dd178567cc1@mail.gmail.com>
	<loom.20090923T132003-720@post.gmane.org>
Message-ID: <4ABA1316.9000502@gmail.com>

Antoine Pitrou wrote:
> Brett Cannon <brett <at> python.org> writes:
>> Trust me, if you are doing open source for
>> anything other than altruistic reasons you are bound to be
>> disappointed.
> 
> I'm surprised by this statement but perhaps it's a matter of vocabulary.
> Having fun and doing things you like to do does not strike me as "altruistic".
> Being involved in a FOSS project is not the same as participating in a charity.

I'd agree that 'altruism' isn't quite the right word - there's also the
fact that plenty of folks these days contribute to open source because
someone is paying them to :)

However, Brett's basic point that good input may sometimes go uncredited
through no fault of the poster's remains valid.

Patches and particularly good bug reports/suggestions get credited in
commit messages and sometimes NEWS items and the What's New, significant
patches generally earn a mention in the ACKS file and each PEP usually
has an acknowledgement section that lists major contributors to the
associated discussion and reviews.

Posts (even well-thought out ones) on the various discussion lists? Not
so much - it's too easy to lose track of who posted what in an involved
discussion. The highest respect you can really earn in those is to make
a valid point clearly enough that you convince others to adopt your
point of view. Although sometimes you can still persuade others even
when you later turn out to be wrong* ;)

Cheers,
Nick.

* See a couple of the footnotes to PEP 343 if you want to know what that
smiley is about :)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From fwierzbicki at gmail.com  Wed Sep 23 15:04:41 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Wed, 23 Sep 2009 09:04:41 -0400
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <19129.39295.970536.916767@montanaro.dyndns.org>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
Message-ID: <4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com>

On Tue, Sep 22, 2009 at 11:43 PM,  <skip at pobox.com> wrote:
>
> ? ?Dino> For IronPython we wrote a set of tests which go through and define
> ? ?Dino> the various operator methods in all sorts of combinations on both
> ? ?Dino> new-style and old-style classes as well as subclasses of those
> ? ?Dino> classes and then do the comparisons w/ logging.
>
> It would be very nice if these complex corner cases had a set of test
> cases which could be run by all implementations (CPython, Jython,
> IronPython, PyPy, etc). ?I don't know. ?Maybe the CPython test suite serves
> that purpose, but it seems like it would be helpful if this sort of
> "validation suite" was maintained as a separate project all implementations
> could use and contribute to.
Talk has started up again on the stdlib-sig list about finding a core
stdlib + tests that can be shared by all implementations, potentially
living apart from CPython.  I have volunteered to put together a PEP
on the subject, with Jessie Noller and Brett Canon are helping me out.
 When I have something worth showing, I'll start the real PEP process.

-Frank

From fwierzbicki at gmail.com  Wed Sep 23 15:06:35 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Wed, 23 Sep 2009 09:06:35 -0400
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD049385AF@TK5EX14MBXC120.redmond.corp.microsoft.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<4dab5f760909221743l4de4d983s2d5eabe78f052e62@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD049385AF@TK5EX14MBXC120.redmond.corp.microsoft.com>
Message-ID: <4dab5f760909230606j6fc88a34tac6bc95fb42456ce@mail.gmail.com>

On Tue, Sep 22, 2009 at 9:15 PM, Dino Viehland <dinov at microsoft.com> wrote:
> And the latest version there is in:
>
> IronPython_Main\Src\Tests\compat
>
> Hopefully the infrastructure will just work on Jython because it also runs on
> CPython but there may be some Windows specific code in there (e.g. import nt,
> a general problem w/ our tests from the days before we always ran tests w/
> the CPython std lib present).
>
> If you run into any problems let me know and I'd be happy to fix it to make
> them more compatible.
Great, thanks!  I'll take a look (for real this time :)

-Frank

From eric at trueblade.com  Wed Sep 23 15:01:52 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 23 Sep 2009 09:01:52 -0400
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <19129.39295.970536.916767@montanaro.dyndns.org>
References: <4AB8D9F2.6050807@simplistix.co.uk>	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>	<4AB8E136.8040706@simplistix.co.uk>	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
Message-ID: <4ABA1C40.7080001@trueblade.com>

skip at pobox.com wrote:
>     Dino> For IronPython we wrote a set of tests which go through and define
>     Dino> the various operator methods in all sorts of combinations on both
>     Dino> new-style and old-style classes as well as subclasses of those
>     Dino> classes and then do the comparisons w/ logging.
> 
> It would be very nice if these complex corner cases had a set of test
> cases which could be run by all implementations (CPython, Jython,
> IronPython, PyPy, etc).  I don't know.  Maybe the CPython test suite serves
> that purpose, but it seems like it would be helpful if this sort of
> "validation suite" was maintained as a separate project all implementations
> could use and contribute to.

IIRC, one of the reasons for "breaking out"[1] the standard library (and 
its test suite) was to allow for things like this.

Eric.

[1] For whatever definition "breaking out" ends up having.


From stephen at xemacs.org  Wed Sep 23 16:04:05 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 23 Sep 2009 23:04:05 +0900
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
Message-ID: <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>

Tarek Ziad? writes:
 > On Wed, Sep 23, 2009 at 9:03 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
 > > At the very least, that would have kept this discussion on Distutils-
 > > SIG, and Chris couldn't be accused of trying to make an end run around
 > > that process. ?I suggest that posting progress reports to Python-Dev
 > > is a good idea (attracting attention and maybe participation to the
 > > process), but making one an opportunity to test the degree of internal
 > > consensus (or lack of it) on Distutils-SIG by posting there first is
 > > an even better one.
 > 
 > Please define what "internal consensus on Distutils-SIG" means.

Definition is your job, as chairman/dictator.

I did offer a concrete criterion for an individual's participation in
a "internal consensus": that you expect that they will adopt the new
features of distutils as a foundation for their own distribution
systems, or at least not implement and promote an alternative.

As for who needs to be included, if the author of setuptools isn't
part of the internal consensus (on that, I'm just guessing from the
fact that he went off to "start a new thread"), I think you should be
concerned.  He's already implemented and promoted an alternative in
the past, so he doesn't even have to do any implementation.  Just keep
on using and promoting his preferred tools and formats.

A third point is that you can have a consensus on "agreeing to
disagree".  But it's unlikely that you will have 100% *dis*agreement.
If you can get a consensus that on certain points certain people are
simply not going to change their minds, it's often possible to move
past that to work on things that you can agree on.  (And if some
people are unwilling to even admit that, it's usually clear to
absolutely everybody else.)

 > distutils, don't you agree that it's impossible in this context to
 > get a 100% consensus ?

Depends on how you define "100% consensus".  If you mean 100% of
people agree on 100% of the proposal, of course not.  But there may be
40% of the proposal you can get 90% of the people to agree on, and
maybe 90% of the proposal is acceptable to 40% of the people.  (That
would be pretty good in this case, but this community regularly
manages to come close to that, so there's some hope.)

If you can get the setuptools and buildout people to agree to use some
parts of "new distutils", that's a form of consensus, and definite
progress.

 > and that I need to take some decisions to move distutils on ?

I don't know; a better distribution system is not something I need as
a user or a developer.  What worries me is that a simple progress
report caused dissension to spill over onto the Python-Dev list.  That
is relatively rare in this community.  And people like Brett think
it's important that some progress be made on distutils, so I'd like to
see it done as quickly as possible -- but no quicker.<wink>

 > Especially for this topic. If you take the time to read everything
 > you'll see that there were no real alternative design proposed, and
 > that I am just working out the details because I maintain and code
 > distutils.

Well, from the behavior of Philip and Chris, it seems that their
position is that there was insufficient time to put forward an
alternative design before the summary was posted to Python-Dev.  *I
don't care whether its true or not*, it's your job as chairman/
dictator to decide that, and we shouldn't discuss it here.  But merely
leaving the *impression* is damaging, and I suggested a simple
procedure (posting the summary to your mailing list and requesting
comments) that would very likely improve the summary, and also be
likely to keep unnecessary controversy off Python-Dev.

From benjamin at python.org  Wed Sep 23 16:35:07 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 23 Sep 2009 09:35:07 -0500
Subject: [Python-Dev] thinking about 2.7
Message-ID: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>

Hi everyone,
I've started plotting the release of 2.7. I'd like to try for a final
release mid next summer. 3.2 should be released, if not at the same
time as 2.7, within a few weeks to avoid 2.x having features which 3.x
doesn't. If no one has problems with this, I will draft a schedule.

Are we still planning to make 3.3 the main development focus and start
the 5 years of 2.x maintenance after this release?

Additionally, I'm very apprehensive about doing any kind of release
without the buildbots running. Does anyone know when they might be up?

-- 
Regards,
Benjamin

From skip at pobox.com  Wed Sep 23 17:06:48 2009
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 23 Sep 2009 10:06:48 -0500
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <4ABA1C40.7080001@trueblade.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
	<4ABA1C40.7080001@trueblade.com>
Message-ID: <19130.14728.970627.817149@montanaro.dyndns.org>

    Eric> IIRC, one of the reasons for "breaking out"[1] the standard library (and 
    Eric> its test suite) was to allow for things like this.

In my opinion the standard library and the core test suite (the language
validation stuff) are entirely independent beasts.  I can understand pieces
of the standard library not being available in one variant or another, but
key semantic aspects of the language proper should be constant across
implementations.

That said, I agree that if the standard library is split off from CPython
then the relevant portions of the test suite should go along with it.

Skip

From exarkun at twistedmatrix.com  Wed Sep 23 17:13:55 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 23 Sep 2009 15:13:55 -0000
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
Message-ID: <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain>

On 02:35 pm, benjamin at python.org wrote:
>Hi everyone,
>I've started plotting the release of 2.7. I'd like to try for a final
>release mid next summer. 3.2 should be released, if not at the same
>time as 2.7, within a few weeks to avoid 2.x having features which 3.x
>doesn't. If no one has problems with this, I will draft a schedule.
>
>Are we still planning to make 3.3 the main development focus and start
>the 5 years of 2.x maintenance after this release?

I hope that this decision will be delayed until the release is closer, 
so that it can be based on how 3.x adoption is progressing.
>Additionally, I'm very apprehensive about doing any kind of release
>without the buildbots running. Does anyone know when they might be up?

I was planning on replying to Antoine's earlier message about the 
buildbots after a sufficiently long silence.  I'll reply here instead.

Quite a few years of experience with a distributed team of build slave 
managers has shown me that by far the most reliable way to keep slaves 
online is to have them managed by a dedicated team.  This team doesn't 
need to be small, but since finding dedicated people can sometimes be 
challenging, I think small teams are the most likely outcome (possibly 
resulting in a team of one).  Adding more people who are only mildly 
interested doesn't help.  If, as I believe is the case with Python's 
buildbot configuration, the mildly interested people have sole control 
over certain slaves, then it is actually detrimental.

It's easy for someone to volunteer to set up a new slave.  It's even 
easy to make sure it keeps running for 6 months.  But it's not as easy 
to keep it running indefinitely.  This isn't about the software involved 
(at least not entirely).  It's about someone paying attention to whether 
the slave restarts on reboots, and about paying attention to whether the 
slave host has lots its network connection, or been decomissioned, or 
whether a system upgrade disabled the slave, or whatever other random 
administrative-like tasks are necessary to keep things running.  Casual 
volunteers generally just won't keep up with these tasks.

I suggest finding someone who's seriously interested in the quality of 
CPython and giving them the responsibility of keeping things operating 
properly.  This includes paying attention to the status of slaves, 
cajoling hardware operators into bringing hosts back online and fixing 
network issues, and finding replacements of the appropriate type 
(hardware/software platform) when a slave host is permanently lost.

I would also personally recommend that this person first (well, after 
tracking down all the slave operators and convincing them to bring their 
slaves back online) acquire shell access to all of the slave machines so 
that the owners of the slave hosts themselves no longer need to be the 
gating factor for most issues.

Jean-Paul

From fuzzyman at voidspace.org.uk  Wed Sep 23 17:36:44 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 23 Sep 2009 16:36:44 +0100
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
Message-ID: <4ABA408C.6080103@voidspace.org.uk>

Benjamin Peterson wrote:
> Hi everyone,
> I've started plotting the release of 2.7. I'd like to try for a final
> release mid next summer. 3.2 should be released, if not at the same
> time as 2.7, within a few weeks to avoid 2.x having features which 3.x
> doesn't. If no one has problems with this, I will draft a schedule.
>   
+1

I'm keen to see a 2.7 release as there are good new features on trunk 
(particularly in unittest :-)

Michael

> Are we still planning to make 3.3 the main development focus and start
> the 5 years of 2.x maintenance after this release?
>
> Additionally, I'm very apprehensive about doing any kind of release
> without the buildbots running. Does anyone know when they might be up?
>
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From dinov at microsoft.com  Wed Sep 23 17:54:35 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Wed, 23 Sep 2009 15:54:35 +0000
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <19129.39295.970536.916767@montanaro.dyndns.org>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
Message-ID: <1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com>

Skip wrote:
>     Dino> For IronPython we wrote a set of tests which go through and define
>     Dino> the various operator methods in all sorts of combinations on both
>     Dino> new-style and old-style classes as well as subclasses of those
>     Dino> classes and then do the comparisons w/ logging.
> 
> It would be very nice if these complex corner cases had a set of test
> cases which could be run by all implementations (CPython, Jython,
> IronPython, PyPy, etc).  I don't know.  Maybe the CPython test suite serves
> that purpose, but it seems like it would be helpful if this sort of
> "validation suite" was maintained as a separate project all implementations
> could use and contribute to.

We are going to start contributing tests back real soon now.  I'm not sure
that these are the best tests to contribute as they require a version of
Python to compare against rather than being nice and stand alone.  But I'm
sure we have other tests which cover this as well just not as exhaustively.  
We could also possibly check in the baseline file and then CPython could 
compare it's self to previous versions but it'd probably be a pretty
big file - so it probably shouldn't be included in the standard install
in the tests directory.



From solipsis at pitrou.net  Wed Sep 23 18:29:12 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 23 Sep 2009 16:29:12 +0000 (UTC)
Subject: [Python-Dev] thinking about 2.7
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
Message-ID: <loom.20090923T182821-620@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:
> 
> Hi everyone,
> I've started plotting the release of 2.7. I'd like to try for a final
> release mid next summer. 3.2 should be released, if not at the same
> time as 2.7, within a few weeks to avoid 2.x having features which 3.x
> doesn't. If no one has problems with this, I will draft a schedule.

If you want to avoid the low-activity low-responsiveness holiday period, you
might shoot for June or September/October instead.

Regards

Antoine.



From dickinsm at gmail.com  Wed Sep 23 18:34:37 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 23 Sep 2009 17:34:37 +0100
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
	<1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com>
Message-ID: <5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com>

On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland <dinov at microsoft.com> wrote:
> We are going to start contributing tests back real soon now. ?I'm not sure
> that these are the best tests to contribute as they require a version of
> Python to compare against rather than being nice and stand alone. ?But I'm
> sure we have other tests which cover this as well just not as exhaustively.
> We could also possibly check in the baseline file and then CPython could
> compare it's self to previous versions but it'd probably be a pretty
> big file - so it probably shouldn't be included in the standard install
> in the tests directory.

How big is big?  For comparison, CPython's Lib/test/decimaltestdata
directory alone is already over 4Mb, so maybe size isn't an issue?

Mark

From dinov at microsoft.com  Wed Sep 23 18:42:09 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Wed, 23 Sep 2009 16:42:09 +0000
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both 	object have implementations
In-Reply-To: <5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
	<1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com>
Message-ID: <1A472770E042064698CB5ADC83A12ACD0493D9E1@TK5EX14MBXC120.redmond.corp.microsoft.com>

Mark wrote:
> On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland <dinov at microsoft.com> wrote:
> > We are going to start contributing tests back real soon now. ?I'm not sure
> > that these are the best tests to contribute as they require a version of
> > Python to compare against rather than being nice and stand alone. ?But I'm
> > sure we have other tests which cover this as well just not as exhaustively.
> > We could also possibly check in the baseline file and then CPython could
> > compare it's self to previous versions but it'd probably be a pretty
> > big file - so it probably shouldn't be included in the standard install
> > in the tests directory.
> 
> How big is big?  For comparison, CPython's Lib/test/decimaltestdata
> directory alone is already over 4Mb, so maybe size isn't an issue?

Running all of the side by side tests except our exceptions test it's 
about 73Mb.  It might be highly compressible though.

From fuzzyman at voidspace.org.uk  Wed Sep 23 18:50:16 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 23 Sep 2009 17:50:16 +0100
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both 	object have implementations
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0493D9E1@TK5EX14MBXC120.redmond.corp.microsoft.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>	<4AB8E136.8040706@simplistix.co.uk>	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>	<19129.39295.970536.916767@montanaro.dyndns.org>	<1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com>	<5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD0493D9E1@TK5EX14MBXC120.redmond.corp.microsoft.com>
Message-ID: <4ABA51C8.3020407@voidspace.org.uk>

Dino Viehland wrote:
> Mark wrote:
>   
>> On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland <dinov at microsoft.com> wrote:
>>     
>>> We are going to start contributing tests back real soon now.  I'm not sure
>>> that these are the best tests to contribute as they require a version of
>>> Python to compare against rather than being nice and stand alone.  But I'm
>>> sure we have other tests which cover this as well just not as exhaustively.
>>> We could also possibly check in the baseline file and then CPython could
>>> compare it's self to previous versions but it'd probably be a pretty
>>> big file - so it probably shouldn't be included in the standard install
>>> in the tests directory.
>>>       
>> How big is big?  For comparison, CPython's Lib/test/decimaltestdata
>> directory alone is already over 4Mb, so maybe size isn't an issue?
>>     
>
> Running all of the side by side tests except our exceptions test it's 
> about 73Mb.  It might be highly compressible though.
>   

It sounds like many of these belong in an 'implementation-compliance' 
test suite separate from the main test suite.

On the other hand many of the tests probably cover areas that are really 
'sparsely' covered by current Python tests. For many of them getting a 
'defined' set of results and changing the tests to compare against that 
rather than against CPython could be really useful - but probably a lot 
of work to create.

Michael
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ziade.tarek at gmail.com  Wed Sep 23 19:00:20 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 23 Sep 2009 19:00:20 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>

On Wed, Sep 23, 2009 at 4:04 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> I did offer a concrete criterion for an individual's participation in
> a "internal consensus": that you expect that they will adopt the new
> features of distutils as a foundation for their own distribution
> systems, or at least not implement and promote an alternative.
>
> As for who needs to be included, if the author of setuptools isn't
> part of the internal consensus (on that, I'm just guessing from the
> fact that he went off to "start a new thread"), I think you should be
> concerned. ?He's already implemented and promoted an alternative in
> the past, so he doesn't even have to do any implementation. ?Just keep
> on using and promoting his preferred tools and formats.

While it's great to have Philipp being part of our distutils design
discussions,
for his experience, I am not concerned of not having him in this "internal
consensus" since Setuptools is not maintained anymore.

He said some months ago, he would work on a brand new setuptools
version with zero
dependency to distutils. But it's still vaporware (from his own
words), and the previous version is unmaintained for a year, so it was
forked.

The Distribute (setuptools fork), which is likely to be the first and
only public packaging system
on the top of distutils working under Python 3 (the trunk is
py3k-ready and should be released
in a few days), is pretty active, and none of his contributor raised
against the proposal.

But you are right about the need of making sure every package management
project is involved. We should make sure that Enthought,
which has its own package management system, is part of that consensus.

Also, I am more concerned of not having enough end users involved in
that process.
End users would be: any python developer that needs
to package his code, or any os packager that needs to package a python
distribution
for his system. But those are hard to get involved.

> Well, from the behavior of Philip and Chris, it seems that their
> position is that there was insufficient time to put forward an
> alternative design before the summary was posted to Python-Dev. ?*I
> don't care whether its true or not*, it's your job as chairman/
> dictator to decide that, and we shouldn't discuss it here. ?But merely
> leaving the *impression* is damaging, and I suggested a simple
> procedure (posting the summary to your mailing list and requesting
> comments) that would very likely improve the summary, and also be
> likely to keep unnecessary controversy off Python-Dev.

Please note that the controversy that popped in python-dev didn't popped in
distutils-SIG after I clearly stated that I was going to send a
summary in python-dev,
(which I did four days after). No one commented on it then.

Next time I'll wait a week and I will also send the summary as you suggested,
to make sure everyone sees the message, not hidden inside a big thread.

But I doubt this will cut all further controversy once it's in
Pyton-dev, because
being controversial in Python-dev doesn't have the same impact for the writer
and the readers.

Tarek

From fuzzyman at voidspace.org.uk  Wed Sep 23 19:08:31 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 23 Sep 2009 18:08:31 +0100
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<4AB8D910.8050601@simplistix.co.uk>	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>	<4AB92ADE.9060700@simplistix.co.uk>	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
Message-ID: <4ABA560F.4040406@voidspace.org.uk>

Tarek Ziad? wrote:
> On Wed, Sep 23, 2009 at 4:04 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
>   
>> I did offer a concrete criterion for an individual's participation in
>> a "internal consensus": that you expect that they will adopt the new
>> features of distutils as a foundation for their own distribution
>> systems, or at least not implement and promote an alternative.
>>
>> As for who needs to be included, if the author of setuptools isn't
>> part of the internal consensus (on that, I'm just guessing from the
>> fact that he went off to "start a new thread"), I think you should be
>> concerned.  He's already implemented and promoted an alternative in
>> the past, so he doesn't even have to do any implementation.  Just keep
>> on using and promoting his preferred tools and formats.
>>     
>
> While it's great to have Philipp being part of our distutils design
> discussions,
> for his experience, I am not concerned of not having him in this "internal
> consensus" since Setuptools is not maintained anymore.
>
> He said some months ago, he would work on a brand new setuptools
> version with zero
> dependency to distutils. But it's still vaporware (from his own
> words), and the previous version is unmaintained for a year, so it was
> forked.
>
> The Distribute (setuptools fork), which is likely to be the first and
> only public packaging system
> on the top of distutils working under Python 3 (the trunk is
> py3k-ready and should be released
> in a few days), is pretty active, and none of his contributor raised
> against the proposal.
>
> But you are right about the need of making sure every package management
> project is involved. We should make sure that Enthought,
> which has its own package management system, is part of that consensus.
>
>   

Note that Activestate also have a fledgling package management system 
for Python (unreleased yet I *believe*) so it is probably worth reaching 
out to them as well.

> Also, I am more concerned of not having enough end users involved in
> that process.
> End users would be: any python developer that needs
> to package his code, or any os packager that needs to package a python
> distribution
> for his system. But those are hard to get involved.
>   

Perhaps post your summaries on your blog as well (which is aggregated on 
Planet Python right?) - including a description of the problem it aims 
to solve and how it will be used.

You'll *mostly* be reaching the same set of people, but at least it 
spreads the net a bit wider.

Michael

>   
>> Well, from the behavior of Philip and Chris, it seems that their
>> position is that there was insufficient time to put forward an
>> alternative design before the summary was posted to Python-Dev.  *I
>> don't care whether its true or not*, it's your job as chairman/
>> dictator to decide that, and we shouldn't discuss it here.  But merely
>> leaving the *impression* is damaging, and I suggested a simple
>> procedure (posting the summary to your mailing list and requesting
>> comments) that would very likely improve the summary, and also be
>> likely to keep unnecessary controversy off Python-Dev.
>>     
>
> Please note that the controversy that popped in python-dev didn't popped in
> distutils-SIG after I clearly stated that I was going to send a
> summary in python-dev,
> (which I did four days after). No one commented on it then.
>
> Next time I'll wait a week and I will also send the summary as you suggested,
> to make sure everyone sees the message, not hidden inside a big thread.
>
> But I doubt this will cut all further controversy once it's in
> Pyton-dev, because
> being controversial in Python-dev doesn't have the same impact for the writer
> and the readers.
>
> Tarek
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ziade.tarek at gmail.com  Wed Sep 23 19:11:18 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 23 Sep 2009 19:11:18 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <4ABA560F.4040406@voidspace.org.uk>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<4ABA560F.4040406@voidspace.org.uk>
Message-ID: <94bdd2610909231011n1b81a102scbb5660619ca8697@mail.gmail.com>

On Wed, Sep 23, 2009 at 7:08 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Note that Activestate also have a fledgling package management system for
> Python (unreleased yet I *believe*) so it is probably worth reaching out to
> them as well.

Yes I didn't mention it because the project is private afaik. I'll ask
them if it's planned
to be released.


> Perhaps post your summaries on your blog as well (which is aggregated on
> Planet Python right?) - including a description of the problem it aims to
> solve and how it will be used.
>
> You'll *mostly* be reaching the same set of people, but at least it spreads
> the net a bit wider.

That's right, I'll do it as well.

From solipsis at pitrou.net  Wed Sep 23 19:15:07 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 23 Sep 2009 17:15:07 +0000 (UTC)
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
Message-ID: <loom.20090923T191242-680@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> Also, I am more concerned of not having enough end users involved in
> that process.
> End users would be: any python developer that needs
> to package his code, or any os packager that needs to package a python
> distribution
> for his system. But those are hard to get involved.

I'm sure there are some of them on this list.
If you are concerned about opinion diversity, you can always try to probe
python-list (c.l.py) about it ;)

As far as I'm concerned, anything which looks intuitive enough (e.g. ini-style)
and not overly complicated is fine. The details of the syntax aren't really
important as long as they make sense, and don't get in the way.

Regards

Antoine.



From a.badger at gmail.com  Wed Sep 23 19:49:56 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Wed, 23 Sep 2009 10:49:56 -0700
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<4AB8D910.8050601@simplistix.co.uk>	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>	<4AB92ADE.9060700@simplistix.co.uk>	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
Message-ID: <4ABA5FC4.4080006@gmail.com>

On 09/23/2009 10:00 AM, Tarek Ziad? wrote:

> But you are right about the need of making sure every package management
> project is involved. We should make sure that Enthought,
> which has its own package management system, is part of that consensus.
> 
> Also, I am more concerned of not having enough end users involved in
> that process.
> End users would be: any python developer that needs
> to package his code, or any os packager that needs to package a python
> distribution
> for his system. But those are hard to get involved.
> 
As one of the people who deals with packaging python modules for
distributions, I'm sorry for not having spent more time looking into
this.  I simply haven't had the time lately.  One helpful resource for
engaging linux distributions is:

##distros on irc.freenode.net and their mailing list:
http://lists.freedesktop.org/mailman/listinfo/distributions

Both are low traffic but people from the various linux distributions are
watching and responding on it even if they don't make much noise on
their own.  Just bear in mind that sometimes they will need to connect
you with the right person within their distro rather than being able to
give feedback directly.

-Toshio

From brett at python.org  Wed Sep 23 19:53:50 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Sep 2009 10:53:50 -0700
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<4AB9735B.4000102@mrabarnett.plus.com>
	<Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net>
Message-ID: <bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>

On Tue, Sep 22, 2009 at 20:08, R. David Murray <rdmurray at bitdance.com> wrote:
> On Wed, 23 Sep 2009 at 02:01, MRAB wrote:
>>
>> Dino Viehland wrote:
>>>
>>> ?Is there a reason or a rule by which CPython reports different error
>>> ?message for different failures to subscript?
>>>
>>> ?For example:
>>>
>>> > > > ?set()[2]
>>> ?Traceback (most recent call last):
>>> ? File "<stdin>", line 1, in <module>
>>> ?TypeError: 'set' object does not support indexing
>>> > > > ?class c(object): pass
>>> ?...
>
> [....]
>>>
>>> > > > ?[].append[42]
>>> ?Traceback (most recent call last):
>>> ? File "<stdin>", line 1, in <module>
>>> ?TypeError: 'builtin_function_or_method' object is unsubscriptable
>
> [...]
>>>
>>> ?IronPython had a bug report that we were getting this wrong for set
>>> ?objects and that ?does not support indexing? was also a clearer error
>>> ?message. ?I?m wondering if there?s some reason why the different error
>>> ?messages occur which I?m missing. ? Otherwise I could switch to using
>>> the
>>> ?more clear message or start marking types which should report the
>>> ?unsubscriptable error. ?Does anyone have any insights into this?
>>>
>> I thought that the difference might be between iterable and non-iterable
>> objects, but then I'd expect the class 'c' to behave like the other
>> non-iterables. In fact, the result is different again if it's an
>> old-style class:
>>
>>> > > ?class c: pass
>>> > > ?c()[2]
>>
>> Traceback (most recent call last):
>> ?File "<stdin>", line 1, in <module>
>> ? c()[2]
>> AttributeError: c instance has no attribute '__getitem__'
>
> Looking at the source, these messages are generated from abstract.c, and
> the difference appears to be whether or not the tp_as_sequence slot is
> filled in or not. ?If it is, but there is no sq_item method, then
> PySequence_GetItem gives the "does not support indexing" message.
> If it isn't filled in, you get the 'not subscriptable"(*) message from
> PyObject_GetItem.
>
> The logic appears to be, roughly, if an object does not have mapping
> methods, or has them but does not have mp_subscript, and does have
> sequence methods but does not have sq_item, then you get a 'does
> not support indexing'. ?Otherwise you get 'not subscriptable'.
>
> The question is, is this a useful distinction, or is it an
> implementation artifact?

Let's ignore history, which I bet is the reason for the distinction,
and just look at the error messages; does the distinction make sense
to a newbie? I would say no and that the "does not support indexing"
error message is more useful. For expert programmers they could figure
out the problem with either error message. The only help is if you are
trying to debug a type, but I am willing to bet most of us didn't know
the distinction at the C level until David looked it up.

So I am +1 on unified the message and +1 on using the "does not
support indexing" one.

-Brett

From brett at python.org  Wed Sep 23 19:57:40 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Sep 2009 10:57:40 -0700
Subject: [Python-Dev] altruism
In-Reply-To: <loom.20090923T132003-720@post.gmane.org>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<ba13fddccbc97956b7e03f8973c94805@preisshare.net>
	<ca471dc20909221928m6ebccb7du2b565da45aaa4901@mail.gmail.com> 
	<3e228a5145369e7d9932040c73a7d58e@preisshare.net>
	<bbaeab100909222239ge93c51dr89f14dd178567cc1@mail.gmail.com> 
	<loom.20090923T132003-720@post.gmane.org>
Message-ID: <bbaeab100909231057w262aa622p885263399ab509eb@mail.gmail.com>

On Wed, Sep 23, 2009 at 04:36, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Brett Cannon <brett <at> python.org> writes:
>>
>> Trust me, if you are doing open source for
>> anything other than altruistic reasons you are bound to be
>> disappointed.
>
> I'm surprised by this statement but perhaps it's a matter of vocabulary.
> Having fun and doing things you like to do does not strike me as "altruistic".
> Being involved in a FOSS project is not the same as participating in a charity.
>
> See this quick presentation about motivation of FOSS developers (second slide
> says: "Altruism? Not really"):
> http://www.infonomics.nl/FLOSS/papers/20030619/index.htm
> The full study is at:
> http://www.infonomics.nl/FLOSS/report/index.htm
> ("Part IV: Survey of Developers")
>
> (but of course there's the question of whether "altruism" exists at large, or is
> a political and moral fiction)

Ignoring the temptation to pull out my philosophy degree and devolve
into a ridiculously involved conversation, I meant "altruism" in
regard to not expecting to get anything from others for what you do. I
do think you should only do open source work if you have fun and enjoy
it, but I also think you should not do it for anyone but yourself
unless you are drawing a paycheck.

-Brett

From brett at python.org  Wed Sep 23 20:03:37 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Sep 2009 11:03:37 -0700
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
Message-ID: <bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>

On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson <benjamin at python.org> wrote:
> Hi everyone,
> I've started plotting the release of 2.7. I'd like to try for a final
> release mid next summer. 3.2 should be released, if not at the same
> time as 2.7, within a few weeks to avoid 2.x having features which 3.x
> doesn't. If no one has problems with this, I will draft a schedule.
>

Yes. Does this mean you are volunteering to be the 3.2 release
manager? Or simply prodding for volunteers? If it's the former are you
at all worried about burn-out from doing two release so close together
and being on the hook for two point releases at roughly the same time
for about two years after?

> Are we still planning to make 3.3 the main development focus and start
> the 5 years of 2.x maintenance after this release?

I think so. As Jean-Paul somewhat pointed out the decision can change,
but at this point I know I am willing to say that I don't want to have
to port to four branches anymore if I don't need to.

>
> Additionally, I'm very apprehensive about doing any kind of release
> without the buildbots running. Does anyone know when they might be up?

I don't know the answer, but it might be "never". We used to do
releases without them, so it's not impossible. Just means you have to
really push the alphas, betas, and RCs.

Titus Brown is actually working on a buildbot alternative that's more
volunteer-friendly (pony-build). Hopefully that will be up and going
by then so we can contemplate using that.

-Brett

From exarkun at twistedmatrix.com  Wed Sep 23 20:24:40 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 23 Sep 2009 18:24:40 -0000
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
Message-ID: <20090923182440.1533.961315222.divmod.xquotient.202@localhost.localdomain>

On 06:03 pm, brett at python.org wrote:
>On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson <benjamin at python.org> 
>wrote:
>[snip]
>>
>>Additionally, I'm very apprehensive about doing any kind of release
>>without the buildbots running. Does anyone know when they might be up?
>
>I don't know the answer, but it might be "never". We used to do
>releases without them, so it's not impossible. Just means you have to
>really push the alphas, betas, and RCs.
>
>Titus Brown is actually working on a buildbot alternative that's more
>volunteer-friendly (pony-build). Hopefully that will be up and going
>by then so we can contemplate using that.

I certainly wish Titus luck in his project, but I'm skeptical about 
pony-build magically fixing the problems CPython development has with 
buildbot.

Jean-Paul

From pje at telecommunity.com  Wed Sep 23 22:05:23 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 23 Sep 2009 16:05:23 -0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com
 >
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
Message-ID: <20090923200526.007313A407A@sparrow.telecommunity.com>

At 02:20 PM 9/23/2009 +0200, Tarek Ziad? wrote:
>If you take the time to read everything you'll see that there were 
>no real alternative design proposed,

You did not respond to repeated requests (from more than one person), 
for clarification regarding the requirements that your proposal was 
trying to fill.  More than one person expressed reservations about 
the complexity of your candidate proposal, and wanted to know why we 
needed a fully conditional syntax, if the only use case for 
conditionals was expressing dependencies.

That there was not a "complete" alternative proposal is true...  but 
that's only because you would not answer the question that several 
people asked: that is, why do we *need* conditionals, for anything 
besides dependencies?  What are the use cases?

More than one person asked that question, but I never saw you answer 
it.  (In one case, you answered with the use cases for what the 
conditionals needed to be able to *check*, but not *what the 
conditionals were conditioning* -- which was the whole point of the question.)

A lack of polished alternatives to your proposal does not constitute 
a positive rationale for your own proposal, especially if people are 
asking for the rationale in order to determine whether a simpler 
proposal would suffice!

I dropped my previous competing proposal some time ago when you 
presented strong use cases for static metadata.  And I'd have happily 
dropped my support for Sridhar's proposal, too, if you'd given 
similarly strong use cases for the proposal you went with.

But even if you *didn't* give those use cases, I'd have been fine 
with you saying to Python-Dev, "I got tired of the discussion and 
chose to Pronounce."  (After all, that is what you more or less said 
on the distutils-sig.)

But what hacked me off is that *here*, you presented your 
pronouncement as if it were a summary of distutils-sig discussion, 
when the last flurry of traffic on the distutils-sig right beforehand 
was a bunch of questions and requests for use cases.  Not nice.


From sridharr at activestate.com  Wed Sep 23 22:10:16 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Wed, 23 Sep 2009 13:10:16 -0700
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909231011n1b81a102scbb5660619ca8697@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<4ABA560F.4040406@voidspace.org.uk>
	<94bdd2610909231011n1b81a102scbb5660619ca8697@mail.gmail.com>
Message-ID: <op.u0p7ne161q4jlt@whymac.activestate.com>

On Wed, 23 Sep 2009 10:11:18 -0700, Tarek Ziad? <ziade.tarek at gmail.com>  
wrote:

> On Wed, Sep 23, 2009 at 7:08 PM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>> Note that Activestate also have a fledgling package management system  
>> for
>> Python (unreleased yet I *believe*) so it is probably worth reaching  
>> out to
>> them as well.
> Yes I didn't mention it because the project is private afaik. I'll ask
> them if it's planned
> to be released.

PyPM is planned to be released to the public shortly after the planned  
2.6.3 release. Most likely during the first week of October.

-srid

From benjamin at python.org  Wed Sep 23 22:21:12 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 23 Sep 2009 15:21:12 -0500
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <loom.20090923T182821-620@post.gmane.org>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<loom.20090923T182821-620@post.gmane.org>
Message-ID: <1afaf6160909231321p5ca27b95wec7347bb22cad89a@mail.gmail.com>

2009/9/23 Antoine Pitrou <solipsis at pitrou.net>:
> Benjamin Peterson <benjamin <at> python.org> writes:
>>
>> Hi everyone,
>> I've started plotting the release of 2.7. I'd like to try for a final
>> release mid next summer. 3.2 should be released, if not at the same
>> time as 2.7, within a few weeks to avoid 2.x having features which 3.x
>> doesn't. If no one has problems with this, I will draft a schedule.
>
> If you want to avoid the low-activity low-responsiveness holiday period, you
> might shoot for June or September/October instead.

That is a good point, so I think we should shoot for June.



-- 
Regards,
Benjamin

From guido at python.org  Wed Sep 23 22:22:31 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 23 Sep 2009 13:22:31 -0700
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com> 
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
Message-ID: <ca471dc20909231322u112e7649w7881711babe556ce@mail.gmail.com>

All,

I don't have the bandwidth right now to help out much in this thread,
so I'm glad that the collective effort has mostly figured out what the
rules are (as well as what they should be :-). It truly is a mess,
mostly thanks to the various concerns for backwards compatibility
(classic vs. new classes *and* __cmp__ vs rich comparisons makes for a
truly maddening compatibility matrix).

The double __eq__ calls in the pastebin examples are a surprise even
to me; it seems the logic for first giving the left object a chance
and then the right object is duplicated in the generic rich comparison
logic in object.c and again in instance_richcompare() in
classobject.c.

As Brett cautioned in the bug, I would be hesitant to try and fix this
in 2.x; there's simply too much code that relies on whatever odd
behavior CPython happens to exhibit, especially in test suites that
often over-constrain the behavior (see e.g. the other thread about
error messages, where I have nothing to add that Brett didn't already
say). In 3.2 I'm fine with the (ever so slight) semantic change,
especially since the behavior will rarely be exhibited in practice.
(You'd have to have two different objects with __eq__ methods that
return NotImplemented *and* have some kind of side effect that is
expected.)

--Guido

On Tue, Sep 22, 2009 at 8:12 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Tue, Sep 22, 2009 at 3:37 PM, Chris Withers <chris at simplistix.co.uk> wrote:
>> Where are the specifications on what happens if two objects are compared and both have implementations of __eq__? Which __eq__ is called?
>> What happens if the first one called returns False? Is the second one called? What is one implements __eq__ and the other __ne__?
>
> I (still :-) think this is covered, for Python 2.x at least, by:
>
> http://docs.python.org/reference/datamodel.html#coercion-rules
>
> Specifically, the bits that say:
>
> - For objects x and y, first x.__op__(y) is tried. If this is not
> implemented or returns NotImplemented, y.__rop__(x) is tried. If this
> is also not implemented or returns NotImplemented, a TypeError
> exception is raised. But see the following exception:
>
> - Exception to the previous item: if the left operand is an instance
> of a built-in type or a new-style class, and the right operand is an
> instance of a proper subclass of that type or class and overrides the
> base?s __rop__() method, the right operand?s __rop__() method is tried
> before the left operand?s __op__() method.
>
> I agree that having these rules in a section called 'Coercion rules'
> is a bit confusing.
>
>> Python 2
>> http://pastebin.com/f8f19ab3
>>
>> Python 3
>> http://pastebin.com/f55e44630
>
> The duplicate __eq__ calls in these pastes are disturbing: ?in short,
> if A() and B() are new-style classes defining __eq__, it seems that
> A() == B() ends up calling A.__eq__ twice *and* B.__eq__ twice, in the
> order A.__eq__, B.__eq__, B.__eq__, A.__eq__.
>
> In 3.x, slot_tp_richcompare (in typeobject.c) makes two calls to
> half_richcompare; ?I think the second is redundant. ?The coercion
> rules are already taken care of in do_richcompare (in object.c). ?I
> tried removing the second call to half_richcompare, and the entire
> test-suite still runs without errors.
>
> In 2.x, it's possible that this call is necessary for some bizarre
> combinations of __cmp__ and __eq__; ?I haven't tried to get my head
> around this yet.
>
> I'll open an issue for the duplicate __eq__ calls.
>
> Mark
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From benjamin at python.org  Wed Sep 23 22:34:10 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 23 Sep 2009 15:34:10 -0500
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
Message-ID: <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>

2009/9/23 Brett Cannon <brett at python.org>:
> On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson <benjamin at python.org> wrote:
>> Hi everyone,
>> I've started plotting the release of 2.7. I'd like to try for a final
>> release mid next summer. 3.2 should be released, if not at the same
>> time as 2.7, within a few weeks to avoid 2.x having features which 3.x
>> doesn't. If no one has problems with this, I will draft a schedule.
>>
>
> Yes. Does this mean you are volunteering to be the 3.2 release
> manager? Or simply prodding for volunteers? If it's the former are you
> at all worried about burn-out from doing two release so close together
> and being on the hook for two point releases at roughly the same time
> for about two years after?

While I certainly wouldn't mind doing only 2.7, I don't know how we
could preserve everyone's sanity and release them at about the same
time. Different RMs would have different times they can do releases,
so I would worry about there being a release in a slightly different
stage of a different branch every couple weeks.

As for burn out, I expected 2.7.x, as the last 2.x release, to be
different in that several people would do the maintenance releases
(perhaps on a 6 month schedule or so) for the 5 year period, so that
would leave me with just 3.2.x (and maybe another 3.1.x release).

...
>>
>> Additionally, I'm very apprehensive about doing any kind of release
>> without the buildbots running. Does anyone know when they might be up?
>
> I don't know the answer, but it might be "never". We used to do
> releases without them, so it's not impossible. Just means you have to
> really push the alphas, betas, and RCs.

What do you mean "push"?





-- 
Regards,
Benjamin

From pje at telecommunity.com  Wed Sep 23 22:47:34 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 23 Sep 2009 16:47:34 -0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.co
 m>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
Message-ID: <20090923204737.A97DC3A4079@sparrow.telecommunity.com>

At 07:00 PM 9/23/2009 +0200, Tarek Ziad? wrote:
>While it's great to have Philipp being part of our distutils design
>discussions,
>for his experience, I am not concerned of not having him in this "internal
>consensus" since Setuptools is not maintained anymore.
>
>He said some months ago, he would work on a brand new setuptools
>version with zero
>dependency to distutils. But it's still vaporware (from his own
>words), and the previous version is unmaintained for a year, so it was
>forked.

Here's what actually happened, if anyone cares.  Tarek and friends 
announced a fork of setuptools.  I reviewed the work and saw that -- 
for the most part -- I was happy with it, and opined as how I might 
be willing to bless the the "package inquisition" team as official 
maintainers of the 0.6 branch of setuptools, so that I could work on 
the fun bits I've long planned for 0.7, but never felt free to start 
on while there was so much still needing to be done on 0.6.

However, just as I mentioned this, and suggested an option for what I 
could do that would be helpful to his Distribute 0.7 project as well 
as various other tools (e.g. implementing some of Jim Fulton's 
long-requested features for better modularization of setuptools), 
Tarek accused me of somehow trying to undermine his plans.

In addition, it appears Tarek was also offended by my earlier 
statement that there were only a few people in the Python community 
who had *already* earned my implicit trust to not only hack on 
setuptools unsupervised, but also to take over its *future* direction 
and BDFL-ship.  (For example, Jim Fulton and Ian Bicking.)

Tarek, however, appears to have taken this to mean that I personally 
thought he was an incompetent programmer or something (when I 
actually had no opinion one way or the other), and ever since he has 
taken to levelling potshots like the above at me on a semi-regular basis.

I've tried to ignore this and play nice, because he is actually 
working on this stuff and I am not.  But it's hard for me to actually 
give any help in practice, if Tarek is too busy projecting hidden 
plots onto everything I say and do.

If you read Tarek's distutils-sig posts, it appears my 
already-existing trust in Ian and Jim was not only a personal insult 
to Tarek, but also a plot to ensure that nobody with any time to do 
so would ever work on setuptools, just as my excitement about working 
on setuptools again was a plot to steal thunder from his fork.

All I want is for good stuff to happen for setuptools users and 
Python users in general, so I don't think all the suspicion and 
backbiting is merited.  I certainly don't appreciate it, and I would 
like it to stop.  It also isn't even relevant to the thread, since my 
lack of work on setuptools says exactly zero about the merits or lack 
thereof of Tarek's proposals for the distutils!

Hell, I *support* the bulk of Tarek's setup.cfg proposal, and don't 
even object to him Pronouncing it or cutting off the discussion!  My 
only issue on Python-Dev was his inaccurate implication that it was a 
SIG consensus rather than a pronouncement on it.  There is and was no 
need for any of this to get personal, and I have continually strived 
to keep my posts here and distutils-sig civil, even when I didn't 
feel like being civil in response to Tarek's jabs.  I have in fact 
bent over backwards to be *nice* to Tarek, because he seemed so damn 
sensitive about everything.  Apparently, however, this does not 
actually help things.  :-(


From dinov at microsoft.com  Wed Sep 23 22:47:41 2009
From: dinov at microsoft.com (Dino Viehland)
Date: Wed, 23 Sep 2009 20:47:41 +0000
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<4AB9735B.4000102@mrabarnett.plus.com>
	<Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net>
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>
Message-ID: <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>

Brett wrote:
> Let's ignore history, which I bet is the reason for the distinction,
> and just look at the error messages; does the distinction make sense
> to a newbie? I would say no and that the "does not support indexing"
> error message is more useful. For expert programmers they could figure
> out the problem with either error message. The only help is if you are
> trying to debug a type, but I am willing to bet most of us didn't know
> the distinction at the C level until David looked it up.
> 
> So I am +1 on unified the message and +1 on using the "does not
> support indexing" one.

I'd be +1 on the unified message as well - but it seems what that message
should be may be contentious (and quite a bike shed discussion at that).
The bug David linked to (http://bugs.python.org/issue5760) has a 
preference for subscript because that's what's used elsewhere in the 
language. 

From mal at egenix.com  Wed Sep 23 23:17:16 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 23 Sep 2009 23:17:16 +0200
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>	<4AB9735B.4000102@mrabarnett.plus.com>	<Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net>
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>
Message-ID: <4ABA905C.2040807@egenix.com>

Brett Cannon wrote:
> On Tue, Sep 22, 2009 at 20:08, R. David Murray <rdmurray at bitdance.com> wrote:
>> On Wed, 23 Sep 2009 at 02:01, MRAB wrote:
>>>
>>> Dino Viehland wrote:
>>>>
>>>>  Is there a reason or a rule by which CPython reports different error
>>>>  message for different failures to subscript?
>>>>
>>>>  For example:
>>>>
>>>>>>>  set()[2]
>>>>  Traceback (most recent call last):
>>>>   File "<stdin>", line 1, in <module>
>>>>  TypeError: 'set' object does not support indexing
>>>>>>>  class c(object): pass
>>>>  ...
>>
>> [....]
>>>>
>>>>>>>  [].append[42]
>>>>  Traceback (most recent call last):
>>>>   File "<stdin>", line 1, in <module>
>>>>  TypeError: 'builtin_function_or_method' object is unsubscriptable
>>
>> [...]
>>>>
>>>>  IronPython had a bug report that we were getting this wrong for set
>>>>  objects and that ?does not support indexing? was also a clearer error
>>>>  message.  I?m wondering if there?s some reason why the different error
>>>>  messages occur which I?m missing.   Otherwise I could switch to using
>>>> the
>>>>  more clear message or start marking types which should report the
>>>>  unsubscriptable error.  Does anyone have any insights into this?
>>>>
>>> I thought that the difference might be between iterable and non-iterable
>>> objects, but then I'd expect the class 'c' to behave like the other
>>> non-iterables. In fact, the result is different again if it's an
>>> old-style class:
>>>
>>>>>>  class c: pass
>>>>>>  c()[2]
>>>
>>> Traceback (most recent call last):
>>>  File "<stdin>", line 1, in <module>
>>>   c()[2]
>>> AttributeError: c instance has no attribute '__getitem__'
>>
>> Looking at the source, these messages are generated from abstract.c, and
>> the difference appears to be whether or not the tp_as_sequence slot is
>> filled in or not.  If it is, but there is no sq_item method, then
>> PySequence_GetItem gives the "does not support indexing" message.
>> If it isn't filled in, you get the 'not subscriptable"(*) message from
>> PyObject_GetItem.
>>
>> The logic appears to be, roughly, if an object does not have mapping
>> methods, or has them but does not have mp_subscript, and does have
>> sequence methods but does not have sq_item, then you get a 'does
>> not support indexing'.  Otherwise you get 'not subscriptable'.
>>
>> The question is, is this a useful distinction, or is it an
>> implementation artifact?
> 
> Let's ignore history, which I bet is the reason for the distinction,
> and just look at the error messages; does the distinction make sense
> to a newbie? I would say no and that the "does not support indexing"
> error message is more useful. For expert programmers they could figure
> out the problem with either error message. The only help is if you are
> trying to debug a type, but I am willing to bet most of us didn't know
> the distinction at the C level until David looked it up.
> 
> So I am +1 on unified the message and +1 on using the "does not
> support indexing" one.

+1 as well.

Note that the wording of error message has never been something
we guaranteed backwards compatibility for, so such changes are a
lot easier to implement than API changes.

(I just hope that doctest doesn't rely on the message texts.)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 23 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From fuzzyman at voidspace.org.uk  Wed Sep 23 23:19:00 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 23 Sep 2009 22:19:00 +0100
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <4ABA905C.2040807@egenix.com>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>	<4AB9735B.4000102@mrabarnett.plus.com>	<Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net>	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>
	<4ABA905C.2040807@egenix.com>
Message-ID: <4ABA90C4.2040401@voidspace.org.uk>

M.-A. Lemburg wrote:
> Brett Cannon wrote:
>   
>> On Tue, Sep 22, 2009 at 20:08, R. David Murray <rdmurray at bitdance.com> wrote:
>>     
>>> On Wed, 23 Sep 2009 at 02:01, MRAB wrote:
>>>       
>>>> Dino Viehland wrote:
>>>>         
>>>>>  Is there a reason or a rule by which CPython reports different error
>>>>>  message for different failures to subscript?
>>>>>
>>>>>  For example:
>>>>>
>>>>>           
>>>>>>>>  set()[2]
>>>>>>>>                 
>>>>>  Traceback (most recent call last):
>>>>>   File "<stdin>", line 1, in <module>
>>>>>  TypeError: 'set' object does not support indexing
>>>>>           
>>>>>>>>  class c(object): pass
>>>>>>>>                 
>>>>>  ...
>>>>>           
>>> [....]
>>>       
>>>>>>>>  [].append[42]
>>>>>>>>                 
>>>>>  Traceback (most recent call last):
>>>>>   File "<stdin>", line 1, in <module>
>>>>>  TypeError: 'builtin_function_or_method' object is unsubscriptable
>>>>>           
>>> [...]
>>>       
>>>>>  IronPython had a bug report that we were getting this wrong for set
>>>>>  objects and that ?does not support indexing? was also a clearer error
>>>>>  message.  I?m wondering if there?s some reason why the different error
>>>>>  messages occur which I?m missing.   Otherwise I could switch to using
>>>>> the
>>>>>  more clear message or start marking types which should report the
>>>>>  unsubscriptable error.  Does anyone have any insights into this?
>>>>>
>>>>>           
>>>> I thought that the difference might be between iterable and non-iterable
>>>> objects, but then I'd expect the class 'c' to behave like the other
>>>> non-iterables. In fact, the result is different again if it's an
>>>> old-style class:
>>>>
>>>>         
>>>>>>>  class c: pass
>>>>>>>  c()[2]
>>>>>>>               
>>>> Traceback (most recent call last):
>>>>  File "<stdin>", line 1, in <module>
>>>>   c()[2]
>>>> AttributeError: c instance has no attribute '__getitem__'
>>>>         
>>> Looking at the source, these messages are generated from abstract.c, and
>>> the difference appears to be whether or not the tp_as_sequence slot is
>>> filled in or not.  If it is, but there is no sq_item method, then
>>> PySequence_GetItem gives the "does not support indexing" message.
>>> If it isn't filled in, you get the 'not subscriptable"(*) message from
>>> PyObject_GetItem.
>>>
>>> The logic appears to be, roughly, if an object does not have mapping
>>> methods, or has them but does not have mp_subscript, and does have
>>> sequence methods but does not have sq_item, then you get a 'does
>>> not support indexing'.  Otherwise you get 'not subscriptable'.
>>>
>>> The question is, is this a useful distinction, or is it an
>>> implementation artifact?
>>>       
>> Let's ignore history, which I bet is the reason for the distinction,
>> and just look at the error messages; does the distinction make sense
>> to a newbie? I would say no and that the "does not support indexing"
>> error message is more useful. For expert programmers they could figure
>> out the problem with either error message. The only help is if you are
>> trying to debug a type, but I am willing to bet most of us didn't know
>> the distinction at the C level until David looked it up.
>>
>> So I am +1 on unified the message and +1 on using the "does not
>> support indexing" one.
>>     
>
> +1 as well.
>   

Also +1. I had a friend (an experienced programmer) who started using 
Python recently. The cryptic nature of some of the error messages was a 
sore point with him.

Michael

> Note that the wording of error message has never been something
> we guaranteed backwards compatibility for, so such changes are a
> lot easier to implement than API changes.
>
> (I just hope that doctest doesn't rely on the message texts.)
>
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From fuzzyman at voidspace.org.uk  Wed Sep 23 23:19:31 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 23 Sep 2009 22:19:31 +0100
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
Message-ID: <4ABA90E3.60200@voidspace.org.uk>

Benjamin Peterson wrote:
> 2009/9/23 Brett Cannon <brett at python.org>:
>   
>> On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson <benjamin at python.org> wrote:
>>     
>>> Hi everyone,
>>> I've started plotting the release of 2.7. I'd like to try for a final
>>> release mid next summer. 3.2 should be released, if not at the same
>>> time as 2.7, within a few weeks to avoid 2.x having features which 3.x
>>> doesn't. If no one has problems with this, I will draft a schedule.
>>>
>>>       
>> Yes. Does this mean you are volunteering to be the 3.2 release
>> manager? Or simply prodding for volunteers? If it's the former are you
>> at all worried about burn-out from doing two release so close together
>> and being on the hook for two point releases at roughly the same time
>> for about two years after?
>>     
>
> While I certainly wouldn't mind doing only 2.7, I don't know how we
> could preserve everyone's sanity and release them at about the same
> time. Different RMs would have different times they can do releases,
> so I would worry about there being a release in a slightly different
> stage of a different branch every couple weeks.
>
> As for burn out, I expected 2.7.x, as the last 2.x release, 

Are we definitely decided that 2.7 will be the last major release in the 
2.x cycle?

Michael

> to be
> different in that several people would do the maintenance releases
> (perhaps on a 6 month schedule or so) for the 5 year period, so that
> would leave me with just 3.2.x (and maybe another 3.1.x release).
>
> ...
>   
>>> Additionally, I'm very apprehensive about doing any kind of release
>>> without the buildbots running. Does anyone know when they might be up?
>>>       
>> I don't know the answer, but it might be "never". We used to do
>> releases without them, so it's not impossible. Just means you have to
>> really push the alphas, betas, and RCs.
>>     
>
> What do you mean "push"?
>
>
>
>
>
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From solipsis at pitrou.net  Wed Sep 23 23:26:28 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 23 Sep 2009 21:26:28 +0000 (UTC)
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.co m>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
Message-ID: <loom.20090923T232018-116@post.gmane.org>

P.J. Eby <pje <at> telecommunity.com> writes:
> 
> Hell, I *support* the bulk of Tarek's setup.cfg proposal, and don't 
> even object to him Pronouncing it or cutting off the discussion!  My 
> only issue on Python-Dev was his inaccurate implication that it was a 
> SIG consensus rather than a pronouncement on it.  There is and was no 
> need for any of this to get personal, and I have continually strived 
> to keep my posts here and distutils-sig civil, even when I didn't 
> feel like being civil in response to Tarek's jabs.  I have in fact 
> bent over backwards to be *nice* to Tarek, because he seemed so damn 
> sensitive about everything.  Apparently, however, this does not 
> actually help things.

Ok, so Tarek and Philip, are you both ok that those little disagreements should
belong to the past now? :)

Philip did an important job in bringing setuptools to the Python community, and
Tarek took a difficult decision when he finally decided to fork setuptools (he
had probably been privately encouraged for months by people like me, and was
still reluctant to do it AFAICT). Both of you have been tremendously helpful to
Python, and I'm sure we can move forward graciously.

Regards

Antoine.



From solipsis at pitrou.net  Wed Sep 23 23:28:04 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 23 Sep 2009 21:28:04 +0000 (UTC)
Subject: [Python-Dev] thinking about 2.7
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
Message-ID: <loom.20090923T232645-287@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:
> 
> Different RMs would have different times they can do releases,
> so I would worry about there being a release in a slightly different
> stage of a different branch every couple weeks.

Assuming you can do it, +1 for you (Benjamin) being RM for both 2.7 and 3.1.
Indeed, both branches need to be synchronized anyway and the best way to achieve
this is for a single person to overview both of them.

Regards

Antoine.



From benjamin at python.org  Wed Sep 23 23:34:17 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 23 Sep 2009 16:34:17 -0500
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <4ABA90E3.60200@voidspace.org.uk>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
	<4ABA90E3.60200@voidspace.org.uk>
Message-ID: <1afaf6160909231434t6922d30eu5eb0b894617b0340@mail.gmail.com>

2009/9/23 Michael Foord <fuzzyman at voidspace.org.uk>:
> Benjamin Peterson wrote:
>> As for burn out, I expected 2.7.x, as the last 2.x release,
>
> Are we definitely decided that 2.7 will be the last major release in the 2.x
> cycle?

No, but that's what we're planning for atm.


-- 
Regards,
Benjamin

From brett at python.org  Wed Sep 23 23:26:30 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Sep 2009 14:26:30 -0700
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <4ABA90C4.2040401@voidspace.org.uk>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<4AB9735B.4000102@mrabarnett.plus.com>
	<Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net> 
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com> 
	<4ABA905C.2040807@egenix.com> <4ABA90C4.2040401@voidspace.org.uk>
Message-ID: <bbaeab100909231426q4bdb4a47n372f1dc5ae27bee2@mail.gmail.com>

On Wed, Sep 23, 2009 at 14:19, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
[SNIP]
> Also +1. I had a friend (an experienced programmer) who started using Python
> recently. The cryptic nature of some of the error messages was a sore point
> with him.

Do you know which error messages? We can change them. We have always
said that we view exception messages as something that can change in
minor releases and that you should be very careful if you ever use
them in tests.

-Brett

From brett at python.org  Wed Sep 23 23:12:12 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Sep 2009 14:12:12 -0700
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> 
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com> 
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
Message-ID: <bbaeab100909231412s58b45906tb6da48b2ef643717@mail.gmail.com>

On Wed, Sep 23, 2009 at 13:34, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/9/23 Brett Cannon <brett at python.org>:
>> On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson <benjamin at python.org> wrote:
>>> Hi everyone,
>>> I've started plotting the release of 2.7. I'd like to try for a final
>>> release mid next summer. 3.2 should be released, if not at the same
>>> time as 2.7, within a few weeks to avoid 2.x having features which 3.x
>>> doesn't. If no one has problems with this, I will draft a schedule.
>>>
>>
>> Yes. Does this mean you are volunteering to be the 3.2 release
>> manager? Or simply prodding for volunteers? If it's the former are you
>> at all worried about burn-out from doing two release so close together
>> and being on the hook for two point releases at roughly the same time
>> for about two years after?
>
> While I certainly wouldn't mind doing only 2.7, I don't know how we
> could preserve everyone's sanity and release them at about the same
> time. Different RMs would have different times they can do releases,
> so I would worry about there being a release in a slightly different
> stage of a different branch every couple weeks.
>

If you are okay with it then that's fine.

> As for burn out, I expected 2.7.x, as the last 2.x release, to be
> different in that several people would do the maintenance releases
> (perhaps on a 6 month schedule or so) for the 5 year period, so that
> would leave me with just 3.2.x (and maybe another 3.1.x release).

Well, if you want that to happen you should make that clear now as I
expected you to do the first few micro releases of 2.7.

>
> ...
>>>
>>> Additionally, I'm very apprehensive about doing any kind of release
>>> without the buildbots running. Does anyone know when they might be up?
>>
>> I don't know the answer, but it might be "never". We used to do
>> releases without them, so it's not impossible. Just means you have to
>> really push the alphas, betas, and RCs.
>
> What do you mean "push"?

Promote.

-Brett

From solipsis at pitrou.net  Wed Sep 23 23:36:30 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 23 Sep 2009 21:36:30 +0000 (UTC)
Subject: [Python-Dev] thinking about 2.7
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
	<4ABA90E3.60200@voidspace.org.uk>
Message-ID: <loom.20090923T233425-298@post.gmane.org>

Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
> 
> Are we definitely decided that 2.7 will be the last major release in the 
> 2.x cycle?

I don't think any "definitive" decision was made, but judgeing by Benjamin's and
Brett's answers (and by my own sentiment :-)), it certainly is the expectation
of some core developers that we focus primarily on 3.x after 2.7 is released.

Of course, if a large enough number of core developers want to backport stuff to
2.x, they probably can arrange to do so.

Regards

Antoine.



From brett at python.org  Wed Sep 23 23:24:53 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Sep 2009 14:24:53 -0700
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<4AB9735B.4000102@mrabarnett.plus.com>
	<Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net> 
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com> 
	<1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>
Message-ID: <bbaeab100909231424p22c3b761t5fe5f30b0d46d26d@mail.gmail.com>

On Wed, Sep 23, 2009 at 13:47, Dino Viehland <dinov at microsoft.com> wrote:
> Brett wrote:
>> Let's ignore history, which I bet is the reason for the distinction,
>> and just look at the error messages; does the distinction make sense
>> to a newbie? I would say no and that the "does not support indexing"
>> error message is more useful. For expert programmers they could figure
>> out the problem with either error message. The only help is if you are
>> trying to debug a type, but I am willing to bet most of us didn't know
>> the distinction at the C level until David looked it up.
>>
>> So I am +1 on unified the message and +1 on using the "does not
>> support indexing" one.
>
> I'd be +1 on the unified message as well - but it seems what that message
> should be may be contentious (and quite a bike shed discussion at that).
> The bug David linked to (http://bugs.python.org/issue5760) has a
> preference for subscript because that's what's used elsewhere in the
> language.

That's what's used in the language spec which VERY few people read.
That can use more technical jargon, but I still don't see experienced
programmers having issuesvfiguring out that note being able to index
an object is the same as not being subscriptable. And we can change
the docs as well if needed, but I don't think that is necessary.

-Brett

From benjamin at python.org  Wed Sep 23 23:42:05 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 23 Sep 2009 16:42:05 -0500
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <bbaeab100909231412s58b45906tb6da48b2ef643717@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
	<bbaeab100909231412s58b45906tb6da48b2ef643717@mail.gmail.com>
Message-ID: <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com>

2009/9/23 Brett Cannon <brett at python.org>:
> On Wed, Sep 23, 2009 at 13:34, Benjamin Peterson <benjamin at python.org> wrote:
>> As for burn out, I expected 2.7.x, as the last 2.x release, to be
>> different in that several people would do the maintenance releases
>> (perhaps on a 6 month schedule or so) for the 5 year period, so that
>> would leave me with just 3.2.x (and maybe another 3.1.x release).
>
> Well, if you want that to happen you should make that clear now as I
> expected you to do the first few micro releases of 2.7.

I will do the first few 2.7 bug fix releases.

>>>> Additionally, I'm very apprehensive about doing any kind of release
>>>> without the buildbots running. Does anyone know when they might be up?
>>>
>>> I don't know the answer, but it might be "never". We used to do
>>> releases without them, so it's not impossible. Just means you have to
>>> really push the alphas, betas, and RCs.
>>
>> What do you mean "push"?
>
> Promote.

That basically means we will be relying on 3rd party libraries for our
quality control?


-- 
Regards,
Benjamin

From fuzzyman at voidspace.org.uk  Wed Sep 23 23:49:38 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 23 Sep 2009 22:49:38 +0100
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <bbaeab100909231426q4bdb4a47n372f1dc5ae27bee2@mail.gmail.com>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<4AB9735B.4000102@mrabarnett.plus.com>
	<Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net>
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>
	<4ABA905C.2040807@egenix.com> <4ABA90C4.2040401@voidspace.org.uk>
	<bbaeab100909231426q4bdb4a47n372f1dc5ae27bee2@mail.gmail.com>
Message-ID: <4ABA97F2.6040803@voidspace.org.uk>

Brett Cannon wrote:
> On Wed, Sep 23, 2009 at 14:19, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> [SNIP]
>   
>> Also +1. I had a friend (an experienced programmer) who started using Python
>> recently. The cryptic nature of some of the error messages was a sore point
>> with him.
>>     
>
> Do you know which error messages? We can change them. We have always
> said that we view exception messages as something that can change in
> minor releases and that you should be very careful if you ever use
> them in tests.
>   

There were a couple and I can only remember the last one he talked 
about. He ran some code with "python -3" that used someone else's 
library and resulted in the following warning, that he felt was cryptic:

    DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ 
in 3.x

Hmm... not entirely sure how we can make it clearer. There is a great 
blog entry on the subject by Eric Lippert though (Microsoft I'm afraid):

    http://blogs.msdn.com/ericlippert/archive/2006/07/07/659259.aspx

Error messages: diagnostic is preferable to prescriptive

    * Polite: making the user feel like an idiot is very, very bad.
    * Readable: poor grammar and tortured sentence structure is bad.
    * Accurate: error messages must accurately describe the problem.
    * Precise: "Something is wrong" is an accurate error message but not 
a very precise one!
    * Diagnostic but not prescriptive: describe the problem, not the 
solution.

How about (?):

    If you implement equality (__eq__) you should also implement hashing 
(__hash__), it won't be inherited in 3.x.

Although that is prescriptive of course...

Michael
> -Brett
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From brett at python.org  Wed Sep 23 23:27:40 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Sep 2009 14:27:40 -0700
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <4ABA90E3.60200@voidspace.org.uk>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> 
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com> 
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> 
	<4ABA90E3.60200@voidspace.org.uk>
Message-ID: <bbaeab100909231427p4df5ffe2x216f8fc5862edb1e@mail.gmail.com>

On Wed, Sep 23, 2009 at 14:19, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> Benjamin Peterson wrote:
>>
>> 2009/9/23 Brett Cannon <brett at python.org>:
>>
>>>
>>> On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson <benjamin at python.org>
>>> wrote:
>>>
>>>>
>>>> Hi everyone,
>>>> I've started plotting the release of 2.7. I'd like to try for a final
>>>> release mid next summer. 3.2 should be released, if not at the same
>>>> time as 2.7, within a few weeks to avoid 2.x having features which 3.x
>>>> doesn't. If no one has problems with this, I will draft a schedule.
>>>>
>>>>
>>>
>>> Yes. Does this mean you are volunteering to be the 3.2 release
>>> manager? Or simply prodding for volunteers? If it's the former are you
>>> at all worried about burn-out from doing two release so close together
>>> and being on the hook for two point releases at roughly the same time
>>> for about two years after?
>>>
>>
>> While I certainly wouldn't mind doing only 2.7, I don't know how we
>> could preserve everyone's sanity and release them at about the same
>> time. Different RMs would have different times they can do releases,
>> so I would worry about there being a release in a slightly different
>> stage of a different branch every couple weeks.
>>
>> As for burn out, I expected 2.7.x, as the last 2.x release,
>
> Are we definitely decided that 2.7 will be the last major release in the 2.x
> cycle?

I would say we won't be sure until we are closer to release, but I
would be willing to put money down that it will be.

-Brett

From fuzzyman at voidspace.org.uk  Wed Sep 23 23:52:38 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 23 Sep 2009 22:52:38 +0100
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>	<bbaeab100909231412s58b45906tb6da48b2ef643717@mail.gmail.com>
	<1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com>
Message-ID: <4ABA98A6.7060803@voidspace.org.uk>

Benjamin Peterson wrote:
> 2009/9/23 Brett Cannon <brett at python.org>:
>   
>> On Wed, Sep 23, 2009 at 13:34, Benjamin Peterson <benjamin at python.org> wrote:
>>     
>>> As for burn out, I expected 2.7.x, as the last 2.x release, to be
>>> different in that several people would do the maintenance releases
>>> (perhaps on a 6 month schedule or so) for the 5 year period, so that
>>> would leave me with just 3.2.x (and maybe another 3.1.x release).
>>>       
>> Well, if you want that to happen you should make that clear now as I
>> expected you to do the first few micro releases of 2.7.
>>     
>
> I will do the first few 2.7 bug fix releases.
>
>   
>>>>> Additionally, I'm very apprehensive about doing any kind of release
>>>>> without the buildbots running. Does anyone know when they might be up?
>>>>>           
>>>> I don't know the answer, but it might be "never". We used to do
>>>> releases without them, so it's not impossible. Just means you have to
>>>> really push the alphas, betas, and RCs.
>>>>         
>>> What do you mean "push"?
>>>       
>> Promote.
>>     
>
> That basically means we will be relying on 3rd party libraries for our
> quality control?
>
>
>   
Isn't that the real compatibility test *anyway* - how successful a new 
version of Python is at running all the existing Python code...

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From brett at python.org  Thu Sep 24 00:00:07 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Sep 2009 15:00:07 -0700
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <4ABA97F2.6040803@voidspace.org.uk>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<4AB9735B.4000102@mrabarnett.plus.com>
	<Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net> 
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com> 
	<4ABA905C.2040807@egenix.com> <4ABA90C4.2040401@voidspace.org.uk> 
	<bbaeab100909231426q4bdb4a47n372f1dc5ae27bee2@mail.gmail.com> 
	<4ABA97F2.6040803@voidspace.org.uk>
Message-ID: <bbaeab100909231500o761fb3c9l901dfe06bb713273@mail.gmail.com>

On Wed, Sep 23, 2009 at 14:49, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> Brett Cannon wrote:
>>
>> On Wed, Sep 23, 2009 at 14:19, Michael Foord <fuzzyman at voidspace.org.uk>
>> wrote:
>> [SNIP]
>>
>>>
>>> Also +1. I had a friend (an experienced programmer) who started using
>>> Python
>>> recently. The cryptic nature of some of the error messages was a sore
>>> point
>>> with him.
>>>
>>
>> Do you know which error messages? We can change them. We have always
>> said that we view exception messages as something that can change in
>> minor releases and that you should be very careful if you ever use
>> them in tests.
>>
>
> There were a couple and I can only remember the last one he talked about. He
> ran some code with "python -3" that used someone else's library and resulted
> in the following warning, that he felt was cryptic:
>
> ? DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in
> 3.x
>
> Hmm... not entirely sure how we can make it clearer. There is a great blog
> entry on the subject by Eric Lippert though (Microsoft I'm afraid):
>
> ? http://blogs.msdn.com/ericlippert/archive/2006/07/07/659259.aspx
>
> Error messages: diagnostic is preferable to prescriptive
>
> ? * Polite: making the user feel like an idiot is very, very bad.
> ? * Readable: poor grammar and tortured sentence structure is bad.
> ? * Accurate: error messages must accurately describe the problem.
> ? * Precise: "Something is wrong" is an accurate error message but not a
> very precise one!
> ? * Diagnostic but not prescriptive: describe the problem, not the solution.
>
> How about (?):
>
> ? If you implement equality (__eq__) you should also implement hashing
> (__hash__), it won't be inherited in 3.x.
>
> Although that is prescriptive of course...

So the original makes sense if you are implementing an object's
comparison operator. But if you are not it won't make much sense,
although if you are not implementing it then it shouldn't be your
concern. Then again, if you are trying to upgrade your code and some
library you used has not, I can see how that message would throw you
for a loop.

So +0 on your specific improvment and +1 for trying to think about
newbies when writing exception messages.

-Brett

From ziade.tarek at gmail.com  Thu Sep 24 00:40:30 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 24 Sep 2009 00:40:30 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<94bdd2610909231526l62c20d1cub6f2dff9d7d0480b@mail.gmail.com>
	<94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com>
Message-ID: <94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.com>

(Sorry if it's top posting I am on a mobile)

Come on phillip, no one is "plotting" against you.

You didn't maintain setuptools for a year while people where begging you to
do bug fixes. You blessed Ian and Jim to take over but they are too busy to
do it . I even sent them a mail on my side to try to convince them.

So we asked you to bless someone else that was active (not in particular me
as your mail seem to say) but you did not. So we forked. And people were
pissed off at you. (Which I am sorry about)

If a project is not maintained and if the maintainer does not open it to
other maintainers, that s the way to go .

And the fact that I took the lead of that fork doesn't mean I am offended
because you did not bless me to maintain setuptools. It just means that I
want to move forward and have a working tool for python 3.

So let me make it clear that when you say "Tarek appears" it is something I
have never said but rather something you are thinking like being the truth.

Now for the Distribute work , your patches are very welcome. It s a
community project.

On Sep 23, 2009 10:47 PM, "P.J. Eby" <pje at telecommunity.com> wrote:

At 07:00 PM 9/23/2009 +0200, Tarek Ziad? wrote: > > While it's great to have
Philipp being part of o...
Here's what actually happened, if anyone cares.  Tarek and friends announced
a fork of setuptools.  I reviewed the work and saw that -- for the most part
-- I was happy with it, and opined as how I might be willing to bless the
the "package inquisition" team as official maintainers of the 0.6 branch of
setuptools, so that I could work on the fun bits I've long planned for 0.7,
but never felt free to start on while there was so much still needing to be
done on 0.6.

However, just as I mentioned this, and suggested an option for what I could
do that would be helpful to his Distribute 0.7 project as well as various
other tools (e.g. implementing some of Jim Fulton's long-requested features
for better modularization of setuptools), Tarek accused me of somehow trying
to undermine his plans.

In addition, it appears Tarek was also offended by my earlier statement that
there were only a few people in the Python community who had *already*
earned my implicit trust to not only hack on setuptools unsupervised, but
also to take over its *future* direction and BDFL-ship.  (For example, Jim
Fulton and Ian Bicking.)

Tarek, however, appears to have taken this to mean that I personally thought
he was an incompetent programmer or something (when I actually had no
opinion one way or the other), and ever since he has taken to levelling
potshots like the above at me on a semi-regular basis.

I've tried to ignore this and play nice, because he is actually working on
this stuff and I am not.  But it's hard for me to actually give any help in
practice, if Tarek is too busy projecting hidden plots onto everything I say
and do.

If you read Tarek's distutils-sig posts, it appears my already-existing
trust in Ian and Jim was not only a personal insult to Tarek, but also a
plot to ensure that nobody with any time to do so would ever work on
setuptools, just as my excitement about working on setuptools again was a
plot to steal thunder from his fork.

All I want is for good stuff to happen for setuptools users and Python users
in general, so I don't think all the suspicion and backbiting is merited.  I
certainly don't appreciate it, and I would like it to stop.  It also isn't
even relevant to the thread, since my lack of work on setuptools says
exactly zero about the merits or lack thereof of Tarek's proposals for the
distutils!

Hell, I *support* the bulk of Tarek's setup.cfg proposal, and don't even
object to him Pronouncing it or cutting off the discussion!  My only issue
on Python-Dev was his inaccurate implication that it was a SIG consensus
rather than a pronouncement on it.  There is and was no need for any of this
to get personal, and I have continually strived to keep my posts here and
distutils-sig civil, even when I didn't feel like being civil in response to
Tarek's jabs.  I have in fact bent over backwards to be *nice* to Tarek,
because he seemed so damn sensitive about everything.  Apparently, however,
this does not actually help things.  :-(
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090924/72b37732/attachment.htm>

From ziade.tarek at gmail.com  Thu Sep 24 00:43:59 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 24 Sep 2009 00:43:59 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <loom.20090923T232018-116@post.gmane.org>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<loom.20090923T232018-116@post.gmane.org>
Message-ID: <94bdd2610909231543u16827048k98a773af43c993bf@mail.gmail.com>

Fine with me. Let's move forward.

On Sep 23, 2009 11:27 PM, "Antoine Pitrou" <solipsis at pitrou.net> wrote:

P.J. Eby <pje <at> telecommunity.com> writes: > > Hell, I *support* the bulk
of Tarek's setup.cfg p...
Ok, so Tarek and Philip, are you both ok that those little disagreements
should
belong to the past now? :)

Philip did an important job in bringing setuptools to the Python community,
and
Tarek took a difficult decision when he finally decided to fork setuptools
(he
had probably been privately encouraged for months by people like me, and was
still reluctant to do it AFAICT). Both of you have been tremendously helpful
to
Python, and I'm sure we can move forward graciously.

Regards

Antoine.

_______________________________________________ Python-Dev mailing list
Python-Dev at python.org http...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090924/883a1cfe/attachment-0001.htm>

From python at rcn.com  Thu Sep 24 01:02:37 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 23 Sep 2009 16:02:37 -0700
Subject: [Python-Dev] thinking about 2.7
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com><1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
	<4ABA90E3.60200@voidspace.org.uk>
Message-ID: <85E6E4B220224880B4AFD8A105D39832@RaymondLaptop1>


[Michael Foord]
> Are we definitely decided that 2.7 will be the last major release in the 
> 2.x cycle?

ISTM, that would depend on the uptake for 3.2.
The users get a say in the matter.


Raymond

From fuzzyman at voidspace.org.uk  Thu Sep 24 01:05:35 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 24 Sep 2009 00:05:35 +0100
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <85E6E4B220224880B4AFD8A105D39832@RaymondLaptop1>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com><1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
	<4ABA90E3.60200@voidspace.org.uk>
	<85E6E4B220224880B4AFD8A105D39832@RaymondLaptop1>
Message-ID: <4ABAA9BF.7040609@voidspace.org.uk>

Raymond Hettinger wrote:
>
> [Michael Foord]
>> Are we definitely decided that 2.7 will be the last major release in 
>> the 2.x cycle?
>
> ISTM, that would depend on the uptake for 3.2.
> The users get a say in the matter.
>

That would be my feeling...

Michael

>
> Raymond


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From benjamin at python.org  Thu Sep 24 01:17:17 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 23 Sep 2009 18:17:17 -0500
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <4ABA98A6.7060803@voidspace.org.uk>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
	<bbaeab100909231412s58b45906tb6da48b2ef643717@mail.gmail.com>
	<1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com>
	<4ABA98A6.7060803@voidspace.org.uk>
Message-ID: <1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com>

2009/9/23 Michael Foord <fuzzyman at voidspace.org.uk>:
> Isn't that the real compatibility test *anyway* - how successful a new
> version of Python is at running all the existing Python code...

Yes, but we should have expect 3rd party code to be detecting bugs for
us that our test suite could have shown on a platform.


-- 
Regards,
Benjamin

From fuzzyman at voidspace.org.uk  Thu Sep 24 01:19:21 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 24 Sep 2009 00:19:21 +0100
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>	
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>	
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>	
	<bbaeab100909231412s58b45906tb6da48b2ef643717@mail.gmail.com>	
	<1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com>	
	<4ABA98A6.7060803@voidspace.org.uk>
	<1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com>
Message-ID: <4ABAACF9.60607@voidspace.org.uk>

Benjamin Peterson wrote:
> 2009/9/23 Michael Foord <fuzzyman at voidspace.org.uk>:
>   
>> Isn't that the real compatibility test *anyway* - how successful a new
>> version of Python is at running all the existing Python code...
>>     
>
> Yes, but we should have expect 3rd party code to be detecting bugs for
> us that our test suite could have shown on a platform.
>
>
>   
Well, Trent Nelson is going to devote some real time to Snakebite - so 
there is a very good chance that it will be up and active before the 
release. Naturally I agree it would be *preferable* to run tests on all 
supported platforms prior to release.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From benjamin at python.org  Thu Sep 24 01:22:20 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 23 Sep 2009 18:22:20 -0500
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <4ABAACF9.60607@voidspace.org.uk>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>
	<bbaeab100909231412s58b45906tb6da48b2ef643717@mail.gmail.com>
	<1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com>
	<4ABA98A6.7060803@voidspace.org.uk>
	<1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com>
	<4ABAACF9.60607@voidspace.org.uk>
Message-ID: <1afaf6160909231622o5f677b48yde371b2d8a8147f4@mail.gmail.com>

2009/9/23 Michael Foord <fuzzyman at voidspace.org.uk>:
> Benjamin Peterson wrote:
>>
>> 2009/9/23 Michael Foord <fuzzyman at voidspace.org.uk>:
>>
>>>
>>> Isn't that the real compatibility test *anyway* - how successful a new
>>> version of Python is at running all the existing Python code...
>>>
>>
>> Yes, but we should have expect 3rd party code to be detecting bugs for
>> us that our test suite could have shown on a platform.
>>
>>
>>
>
> Well, Trent Nelson is going to devote some real time to Snakebite - so there
> is a very good chance that it will be up and active before the release.
> Naturally I agree it would be *preferable* to run tests on all supported
> platforms prior to release.

Could I inquiry about your source of this information?



-- 
Regards,
Benjamin

From fuzzyman at voidspace.org.uk  Thu Sep 24 01:23:46 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 24 Sep 2009 00:23:46 +0100
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <1afaf6160909231622o5f677b48yde371b2d8a8147f4@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>	
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>	
	<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>	
	<bbaeab100909231412s58b45906tb6da48b2ef643717@mail.gmail.com>	
	<1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com>	
	<4ABA98A6.7060803@voidspace.org.uk>	
	<1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com>	
	<4ABAACF9.60607@voidspace.org.uk>
	<1afaf6160909231622o5f677b48yde371b2d8a8147f4@mail.gmail.com>
Message-ID: <4ABAAE02.1060800@voidspace.org.uk>

Benjamin Peterson wrote:
> 2009/9/23 Michael Foord <fuzzyman at voidspace.org.uk>:
>   
>> Benjamin Peterson wrote:
>>     
>>> 2009/9/23 Michael Foord <fuzzyman at voidspace.org.uk>:
>>>
>>>       
>>>> Isn't that the real compatibility test *anyway* - how successful a new
>>>> version of Python is at running all the existing Python code...
>>>>
>>>>         
>>> Yes, but we should have expect 3rd party code to be detecting bugs for
>>> us that our test suite could have shown on a platform.
>>>
>>>
>>>
>>>       
>> Well, Trent Nelson is going to devote some real time to Snakebite - so there
>> is a very good chance that it will be up and active before the release.
>> Naturally I agree it would be *preferable* to run tests on all supported
>> platforms prior to release.
>>     
>
> Could I inquiry about your source of this information?
>
>
>
>   
 From Trent on the snakebite mailing list. Too late for me to look it up 
though; an exercise I leave to the reader.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From steve at pearwood.info  Thu Sep 24 01:33:08 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 24 Sep 2009 09:33:08 +1000
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>
Message-ID: <200909240933.09773.steve@pearwood.info>

On Thu, 24 Sep 2009 06:47:41 am Dino Viehland wrote:
> > So I am +1 on unified the message and +1 on using the "does not
> > support indexing" one.
>
> I'd be +1 on the unified message as well - but it seems what that
> message should be may be contentious (and quite a bike shed
> discussion at that). The bug David linked to
> (http://bugs.python.org/issue5760) has a preference for subscript
> because that's what's used elsewhere in the language.

For what it's worth, "unsubscriptable object" seems to me to be 
mysterious to many newbies, and even a few non-newbies. It took me 
something like seven years of coding to stop reading it 
as "unscriptable object", and I'm sure I won't be the only one.

As far as I can see, in practice, people talk about obj[i] as the item 
at index i, not the item at subscript i -- the term "subscript" in this 
context seems to be rare to non-existent except for the error message.



-- 
Steven D'Aprano

From janzert at janzert.com  Thu Sep 24 01:56:22 2009
From: janzert at janzert.com (Janzert)
Date: Wed, 23 Sep 2009 19:56:22 -0400
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <200909240933.09773.steve@pearwood.info>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<200909240933.09773.steve@pearwood.info>
Message-ID: <h9ecj6$oti$1@ger.gmane.org>

Steven D'Aprano wrote:
> On Thu, 24 Sep 2009 06:47:41 am Dino Viehland wrote:
>>> So I am +1 on unified the message and +1 on using the "does not
>>> support indexing" one.
>> I'd be +1 on the unified message as well - but it seems what that
>> message should be may be contentious (and quite a bike shed
>> discussion at that). The bug David linked to
>> (http://bugs.python.org/issue5760) has a preference for subscript
>> because that's what's used elsewhere in the language.
> 
> For what it's worth, "unsubscriptable object" seems to me to be 
> mysterious to many newbies, and even a few non-newbies. It took me 
> something like seven years of coding to stop reading it 
> as "unscriptable object", and I'm sure I won't be the only one.
> 
> As far as I can see, in practice, people talk about obj[i] as the item 
> at index i, not the item at subscript i -- the term "subscript" in this 
> context seems to be rare to non-existent except for the error message.
> 
> 
> 

How about if it's obj["item"]? To me the following makes complete sense,
but then it seems that I may just be the odd one out.

>>> class A(object):
...     pass
...
>>> a = A()
>>> a[1]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'A' object is unindexable
>>> a["a"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'A' object is unsubscriptable
>>>

Janzert


From david.lyon at preisshare.net  Thu Sep 24 01:59:29 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Wed, 23 Sep 2009 19:59:29 -0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <4AB9D2FC.6040705@egenix.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB9D2FC.6040705@egenix.com>
Message-ID: <d096608c6a93af28ff7d5907f9f99b06@preisshare.net>

On Wed, 23 Sep 2009 09:49:16 +0200, "M.-A. Lemburg" <mal at egenix.com> wrote:

> While it's a good idea to put up some form of meta-data
> into an index, I wonder why you are using setup.cfg
> for this.
> 
> setup.cfg has traditionally been used to configure distutils,
> not to define meta-data. As such you wouldn't want to
> put such a configuration file up on PyPI.
> 
> Wouldn't it be better use a new file for this (with the
> same syntax), e.g. metadata.cfg ?! This would then just
> contain the meta data that needs to be published.

Forgive me if I answer this from being on the distutils-ml
but it is easy to answer and might save somebody else time.

Currently, the metadata is stored within setup.py and the
biggest issue with that is maintaining the version number.
For instance, getting the same version number to go into the
documentation files, etc. So consensus was that keeping that 
information in setup.py is somewhat cumbersome and putting
it somewhere else would be better. Tarek suggested setup.cfg
and it seems to make the most sense.

By moving the metadata values into a configParser format file, 
it would mean that external scripts could more easily 
access/update the version number as required.

Of course, setup.cfg wouldn't get uploaded. Nobody would
want to do that.

But distutils would create the .PKG_INFO file and metadata
from configuration, and not from hardcoded string values
within setup.py as it does now.

David






From ben+python at benfinney.id.au  Thu Sep 24 02:10:43 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 24 Sep 2009 10:10:43 +1000
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<200909240933.09773.steve@pearwood.info>
Message-ID: <87tyyt8anw.fsf@benfinney.id.au>

Steven D'Aprano <steve at pearwood.info> writes:

> As far as I can see, in practice, people talk about obj[i] as the item
> at index i, not the item at subscript i -- the term "subscript" in
> this context seems to be rare to non-existent except for the error
> message.

Presumably, the same people would also call ?obj[i]? the item at *key*
?i?, if ?obj? is a dictionary. For an object that supports neither
indexes nor keys, though, how is Python to know which the user meant?
It's a single operation as far as the parser is concerned, so there
needs to be a single term for it. That term is ?subscript?.

Your point about the awkward word ?unsubscriptable? is well-taken,
though. Perhaps a simple improvement to the message wording:

    >>> foo = 3
    >>> foo[6]
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: 'int' object does not support subscripts
    >>> foo['spam']
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: 'int' object does not support subscripts

-- 
 \                              ?I lost a button-hole.? ?Steven Wright |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From eric at trueblade.com  Thu Sep 24 02:12:13 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 23 Sep 2009 20:12:13 -0400
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <4ABAAE02.1060800@voidspace.org.uk>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>		<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>		<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>		<bbaeab100909231412s58b45906tb6da48b2ef643717@mail.gmail.com>		<1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com>		<4ABA98A6.7060803@voidspace.org.uk>		<1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com>		<4ABAACF9.60607@voidspace.org.uk>	<1afaf6160909231622o5f677b48yde371b2d8a8147f4@mail.gmail.com>
	<4ABAAE02.1060800@voidspace.org.uk>
Message-ID: <4ABAB95D.1020906@trueblade.com>

Michael Foord wrote:
> Benjamin Peterson wrote:
>> 2009/9/23 Michael Foord <fuzzyman at voidspace.org.uk>:
>>  
>>> Benjamin Peterson wrote:
>>>    
>>>> 2009/9/23 Michael Foord <fuzzyman at voidspace.org.uk>:
>>>>
>>>>      
>>>>> Isn't that the real compatibility test *anyway* - how successful a new
>>>>> version of Python is at running all the existing Python code...
>>>>>
>>>>>         
>>>> Yes, but we should have expect 3rd party code to be detecting bugs for
>>>> us that our test suite could have shown on a platform.
>>>>
>>>>
>>>>
>>>>       
>>> Well, Trent Nelson is going to devote some real time to Snakebite - 
>>> so there
>>> is a very good chance that it will be up and active before the release.
>>> Naturally I agree it would be *preferable* to run tests on all supported
>>> platforms prior to release.
>>>     
>>
>> Could I inquiry about your source of this information?
>>
>>
>>
>>   
>  From Trent on the snakebite mailing list. Too late for me to look it up 
> though; an exercise I leave to the reader.

http://groups.google.com/group/snakebite-list/browse_thread/thread/d08642261f2cc502

Eric.


From python at mrabarnett.plus.com  Thu Sep 24 02:15:19 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 24 Sep 2009 01:15:19 +0100
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <h9ecj6$oti$1@ger.gmane.org>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>	<200909240933.09773.steve@pearwood.info>
	<h9ecj6$oti$1@ger.gmane.org>
Message-ID: <4ABABA17.5010403@mrabarnett.plus.com>

Janzert wrote:
> Steven D'Aprano wrote:
>> On Thu, 24 Sep 2009 06:47:41 am Dino Viehland wrote:
>>>> So I am +1 on unified the message and +1 on using the "does not
>>>> support indexing" one.
>>> I'd be +1 on the unified message as well - but it seems what that
>>> message should be may be contentious (and quite a bike shed
>>> discussion at that). The bug David linked to
>>> (http://bugs.python.org/issue5760) has a preference for subscript
>>> because that's what's used elsewhere in the language.
>> For what it's worth, "unsubscriptable object" seems to me to be 
>> mysterious to many newbies, and even a few non-newbies. It took me 
>> something like seven years of coding to stop reading it 
>> as "unscriptable object", and I'm sure I won't be the only one.
>>
>> As far as I can see, in practice, people talk about obj[i] as the item 
>> at index i, not the item at subscript i -- the term "subscript" in this 
>> context seems to be rare to non-existent except for the error message.
>>
>>
>>
> 
> How about if it's obj["item"]? To me the following makes complete sense,
> but then it seems that I may just be the odd one out.
> 
>>>> class A(object):
> ...     pass
> ...
>>>> a = A()
>>>> a[1]
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: 'A' object is unindexable
>>>> a["a"]
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: 'A' object is unsubscriptable
> 
I prefer "index" to "subscript" and "does not support foo" or "cannot be
fooed" to "is unfooable" (because, to me, "not" is more explicit than
"un-"). I also prefer the same term to be used irrespective of the
value used.

From python at mrabarnett.plus.com  Thu Sep 24 02:20:22 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 24 Sep 2009 01:20:22 +0100
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <87tyyt8anw.fsf@benfinney.id.au>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>	<200909240933.09773.steve@pearwood.info>
	<87tyyt8anw.fsf@benfinney.id.au>
Message-ID: <4ABABB46.2040105@mrabarnett.plus.com>

Ben Finney wrote:
> Steven D'Aprano <steve at pearwood.info> writes:
> 
>> As far as I can see, in practice, people talk about obj[i] as the item
>> at index i, not the item at subscript i -- the term "subscript" in
>> this context seems to be rare to non-existent except for the error
>> message.
> 
> Presumably, the same people would also call ?obj[i]? the item at *key*
> ?i?, if ?obj? is a dictionary. For an object that supports neither
> indexes nor keys, though, how is Python to know which the user meant?
> It's a single operation as far as the parser is concerned, so there
> needs to be a single term for it. That term is ?subscript?.
> 
> Your point about the awkward word ?unsubscriptable? is well-taken,
> though. Perhaps a simple improvement to the message wording:
> 
>     >>> foo = 3
>     >>> foo[6]
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     TypeError: 'int' object does not support subscripts
>     >>> foo['spam']
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     TypeError: 'int' object does not support subscripts
> 
It's called a 'subscript' because conventional mathematical notation
uses subscripting. Newbies might be acquainted with the term 'index'
from books, where the 'value' is non-numeric. It's a bit unfortunate
that dicts have keys+value instead of index+value! :-)

From solipsis at pitrou.net  Thu Sep 24 02:25:37 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 24 Sep 2009 00:25:37 +0000 (UTC)
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>	<200909240933.09773.steve@pearwood.info>
	<87tyyt8anw.fsf@benfinney.id.au>
	<4ABABB46.2040105@mrabarnett.plus.com>
Message-ID: <loom.20090924T022449-588@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:
> 
> It's called a 'subscript' because conventional mathematical notation
> uses subscripting. Newbies might be acquainted with the term 'index'
> from books, where the 'value' is non-numeric. It's a bit unfortunate
> that dicts have keys+value instead of index+value! 

Well, "index" for me points to the mathematical notion of an index, which AFAIK
is always numeric. So it would be a mistake to use that term for dicts.

Regards

Antoine.



From guido at python.org  Thu Sep 24 02:31:34 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 23 Sep 2009 17:31:34 -0700
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <loom.20090924T022449-588@post.gmane.org>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com> 
	<1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<200909240933.09773.steve@pearwood.info>
	<87tyyt8anw.fsf@benfinney.id.au> 
	<4ABABB46.2040105@mrabarnett.plus.com>
	<loom.20090924T022449-588@post.gmane.org>
Message-ID: <ca471dc20909231731t468fe93eic410cf667d737ac7@mail.gmail.com>

On Wed, Sep 23, 2009 at 5:25 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> MRAB <python <at> mrabarnett.plus.com> writes:
>>
>> It's called a 'subscript' because conventional mathematical notation
>> uses subscripting. Newbies might be acquainted with the term 'index'
>> from books, where the 'value' is non-numeric. It's a bit unfortunate
>> that dicts have keys+value instead of index+value!
>
> Well, "index" for me points to the mathematical notion of an index, which AFAIK
> is always numeric. So it would be a mistake to use that term for dicts.

'Key' and 'index' refer to semantics. 'Subscript' refers to syntax.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Thu Sep 24 02:34:40 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 24 Sep 2009 12:34:40 +1200
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <4AB9D865.1070701@simplistix.co.uk>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<4AB9D865.1070701@simplistix.co.uk>
Message-ID: <4ABABEA0.8040802@canterbury.ac.nz>

Chris Withers wrote:

> But this isn't coercion! :-)

The raisin is probably hysterical. Once upon a time, Python
had a __coerce__ special method that took care of things
like this in a different (and inferior) way. It gradually
got replaced by the current scheme, but nobody changed the
section heading in the docs.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Sep 24 02:44:33 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 24 Sep 2009 12:44:33 +1200
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <4ABA0FE9.7020401@gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<4AB9A3A9.1090800@canterbury.ac.nz> <4ABA0FE9.7020401@gmail.com>
Message-ID: <4ABAC0F1.3080603@canterbury.ac.nz>

Nick Coghlan wrote:

> The interpreter doesn't promise to call those slots with "self" first -
> self will be the second argument in the "rop" case.

I know. My question is: How does it know whether a subclass
"has overridden __rop__" when there is no concept of an
__rop__ method distinct from the __op__ method?

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Sep 24 03:33:43 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 24 Sep 2009 13:33:43 +1200
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <87tyyt8anw.fsf@benfinney.id.au>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<200909240933.09773.steve@pearwood.info>
	<87tyyt8anw.fsf@benfinney.id.au>
Message-ID: <4ABACC77.2060601@canterbury.ac.nz>

Ben Finney wrote:

> It's a single operation as far as the parser is concerned, so there
> needs to be a single term for it. That term is ?subscript?.

How about something like "does not support the [] operation".

This refers directly to the syntax involved, rather than
using a typographical term that is at best a metaphor when
applied to a language written linearly.

-- 
Greg

From ncoghlan at gmail.com  Thu Sep 24 03:48:06 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 24 Sep 2009 11:48:06 +1000
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <4ABAB95D.1020906@trueblade.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>		<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>		<1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com>		<bbaeab100909231412s58b45906tb6da48b2ef643717@mail.gmail.com>		<1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com>		<4ABA98A6.7060803@voidspace.org.uk>		<1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com>		<4ABAACF9.60607@voidspace.org.uk>	<1afaf6160909231622o5f677b48yde371b2d8a8147f4@mail.gmail.com>	<4ABAAE02.1060800@voidspace.org.uk>
	<4ABAB95D.1020906@trueblade.com>
Message-ID: <4ABACFD6.3080401@gmail.com>

Eric Smith wrote:
>>  From Trent on the snakebite mailing list. Too late for me to look it
>> up though; an exercise I leave to the reader.
> 
> http://groups.google.com/group/snakebite-list/browse_thread/thread/d08642261f2cc502

Hmm, I thought I was subscribed to the snakebite list... guess I will
have to check my settings.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From pje at telecommunity.com  Thu Sep 24 03:57:14 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 23 Sep 2009 21:57:14 -0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.co
 m>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<94bdd2610909231526l62c20d1cub6f2dff9d7d0480b@mail.gmail.com>
	<94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com>
	<94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.com>
Message-ID: <20090924015718.B11343A4079@sparrow.telecommunity.com>

At 12:40 AM 9/24/2009 +0200, Tarek Ziad? wrote:
>Come on phillip, no one is "plotting" against you.

Perhaps this is a language issue.  When I said, "if Tarek is too busy 
projecting hidden plots onto everything I say and do," I meant that 
you were acting as if I were plotting against *you*, not the other 
way around.  (For example, you described one of my proposals as 
"suspicious", in a context that made it appear you were concerned it 
would undermine your plans for Distribute.)

However, I find it "suspicious" myself, that, rather than actually 
address *any* of the substantial issues I brought up, you chose to 
re-argue points I'm not even disputing.

For example, is it really necessary to make *every* post of yours 
that mentions me include an essay on how long it's been since the 
last setuptools release?  As I said below, I don't see how that's 
remotely relevant to the value of my contributions...  but somehow 
you find a way to bring it up constantly.

Do you feel so guilty about forking that you need to continually 
re-justify yourself?  If you're doing it on my account, do please 
stop.  AFAIR, I haven't said a negative thing about your fork since 
it got off the ground, and have in fact said many positive things 
about it.  Indeed, the only negative thing I would currently say 
about it, is that your characterization of it as a "friendly" fork is 
not consistent with your public behavior and demeanor towards me.

Once again, I'd like for the badgering to stop.  Thanks.

At 12:40 AM 9/24/2009 +0200, Tarek Ziad? wrote:
>You didn't maintain setuptools for a year while people where begging 
>you to do bug fixes. You blessed Ian and Jim to take over but they 
>are too busy to do it . I even sent them a mail on my side to try to 
>convince them.
>
>So we asked you to bless someone else that was active (not in 
>particular me as your mail seem to say) but you did not. So we 
>forked. And people were pissed off at you. (Which I am sorry about)
>
>If a project is not maintained and if the maintainer does not open 
>it to other maintainers, that s the way to go .
>
>And the fact that I took the lead of that fork doesn't mean I am 
>offended because you did not bless me to maintain setuptools. It 
>just means that I want to move forward and have a working tool for python 3.
>
>So let me make it clear that when you say "Tarek appears" it is 
>something I have never said but rather something you are thinking 
>like being the truth.
>
>Now for the Distribute work , your patches are very welcome. It s a 
>community project.
>
>>On Sep 23, 2009 10:47 PM, "P.J. Eby" 
>><<mailto:pje at telecommunity.com>pje at telecommunity.com> wrote:
>>
>>At 07:00 PM 9/23/2009 +0200, Tarek Ziad? wrote: > > While it's 
>>great to have Philipp being part of o...
>>Here's what actually happened, if anyone cares.  Tarek and friends 
>>announced a fork of setuptools.  I reviewed the work and saw that 
>>-- for the most part -- I was happy with it, and opined as how I 
>>might be willing to bless the the "package inquisition" team as 
>>official maintainers of the 0.6 branch of setuptools, so that I 
>>could work on the fun bits I've long planned for 0.7, but never 
>>felt free to start on while there was so much still needing to be done on 0.6.
>>
>>However, just as I mentioned this, and suggested an option for what 
>>I could do that would be helpful to his Distribute 0.7 project as 
>>well as various other tools (e.g. implementing some of Jim Fulton's 
>>long-requested features for better modularization of setuptools), 
>>Tarek accused me of somehow trying to undermine his plans.
>>
>>In addition, it appears Tarek was also offended by my earlier 
>>statement that there were only a few people in the Python community 
>>who had *already* earned my implicit trust to not only hack on 
>>setuptools unsupervised, but also to take over its *future* 
>>direction and BDFL-ship.  (For example, Jim Fulton and Ian Bicking.)
>>
>>Tarek, however, appears to have taken this to mean that I 
>>personally thought he was an incompetent programmer or something 
>>(when I actually had no opinion one way or the other), and ever 
>>since he has taken to levelling potshots like the above at me on a 
>>semi-regular basis.
>>
>>I've tried to ignore this and play nice, because he is actually 
>>working on this stuff and I am not.  But it's hard for me to 
>>actually give any help in practice, if Tarek is too busy projecting 
>>hidden plots onto everything I say and do.
>>
>>If you read Tarek's distutils-sig posts, it appears my 
>>already-existing trust in Ian and Jim was not only a personal 
>>insult to Tarek, but also a plot to ensure that nobody with any 
>>time to do so would ever work on setuptools, just as my excitement 
>>about working on setuptools again was a plot to steal thunder from his fork.
>>
>>All I want is for good stuff to happen for setuptools users and 
>>Python users in general, so I don't think all the suspicion and 
>>backbiting is merited.  I certainly don't appreciate it, and I 
>>would like it to stop.  It also isn't even relevant to the thread, 
>>since my lack of work on setuptools says exactly zero about the 
>>merits or lack thereof of Tarek's proposals for the distutils!
>>
>>Hell, I *support* the bulk of Tarek's setup.cfg proposal, and don't 
>>even object to him Pronouncing it or cutting off the 
>>discussion!  My only issue on Python-Dev was his inaccurate 
>>implication that it was a SIG consensus rather than a pronouncement 
>>on it.  There is and was no need for any of this to get personal, 
>>and I have continually strived to keep my posts here and 
>>distutils-sig civil, even when I didn't feel like being civil in 
>>response to Tarek's jabs.  I have in fact bent over backwards to be 
>>*nice* to Tarek, because he seemed so damn sensitive about 
>>everything.  Apparently, however, this does not actually help things.  :-(


From guido at python.org  Thu Sep 24 04:02:43 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 23 Sep 2009 19:02:43 -0700
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
	if both 	object have implementations
In-Reply-To: <4ABAC0F1.3080603@canterbury.ac.nz>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com> 
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> 
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com> 
	<4AB9A3A9.1090800@canterbury.ac.nz> <4ABA0FE9.7020401@gmail.com> 
	<4ABAC0F1.3080603@canterbury.ac.nz>
Message-ID: <ca471dc20909231902g59891978va89b2eff8b14a2c6@mail.gmail.com>

On Wed, Sep 23, 2009 at 5:44 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
>
>> The interpreter doesn't promise to call those slots with "self" first -
>> self will be the second argument in the "rop" case.
>
> I know. My question is: How does it know whether a subclass
> "has overridden __rop__" when there is no concept of an
> __rop__ method distinct from the __op__ method?

This is a constraint on types implemented in C -- the same definition
is required to apply to __op__ and __rop__.

The datetime module actually has a slight problem here because it
wants to override datetime - timedelta but not timedelta - datetime
(since the latter would have to return a negative time, which is
meaningless). It solves this easily by doing a type check on each
argument and returning Py_NotImplemented for the unimplemented
combinations -- see the various _subtract functions in
Modules/datetimemodule.c. Returning NotImplemented is close enough to
not actually implementing the method.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Sep 24 04:04:18 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 24 Sep 2009 12:04:18 +1000
Subject: [Python-Dev] unsubscriptable vs object does not support indexing
In-Reply-To: <bbaeab100909231500o761fb3c9l901dfe06bb713273@mail.gmail.com>
References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com>	<4AB9735B.4000102@mrabarnett.plus.com>	<Pine.LNX.4.64.0909222234260.18193@kimball.webabinitio.net>
	<bbaeab100909231053s3a458745m836f302aa85eabd4@mail.gmail.com>
	<4ABA905C.2040807@egenix.com> <4ABA90C4.2040401@voidspace.org.uk>
	<bbaeab100909231426q4bdb4a47n372f1dc5ae27bee2@mail.gmail.com>
	<4ABA97F2.6040803@voidspace.org.uk>
	<bbaeab100909231500o761fb3c9l901dfe06bb713273@mail.gmail.com>
Message-ID: <4ABAD3A2.7050303@gmail.com>

Brett Cannon wrote:
> So +0 on your specific improvment and +1 for trying to think about
> newbies when writing exception messages.

The __eq__/__hash__ messages are somewhat arcane because the problem
they're describing is somewhat arcane.

Michael's suggested improvement also isn't quite right (and I believe
indicates that we're susceptible to issuing false alarms in this case).
You really only need to override __hash__ if you want your object to
remain hashable, but we currently issue the warning for any object that
defines __eq__ without defining __hash__.

Perhaps the implementation of that warning should be improved so that it
is only issued if the default __hash__  implementation is actually
invoked on an affected object? (the downside is that depending on the
implementation technique the extra checks may slow down hashing under
"python -3", but I guess that kind of the undesirable impact is the
reason we have all those warnings switched off by default)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Thu Sep 24 04:05:15 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 23 Sep 2009 19:05:15 -0700
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <20090924015718.B11343A4079@sparrow.telecommunity.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> 
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> 
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> 
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<94bdd2610909231526l62c20d1cub6f2dff9d7d0480b@mail.gmail.com> 
	<94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com> 
	<94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.com> 
	<20090924015718.B11343A4079@sparrow.telecommunity.com>
Message-ID: <ca471dc20909231905q27a3b1demf9678d35149df72@mail.gmail.com>

On Wed, Sep 23, 2009 at 6:57 PM, P.J. Eby <pje at telecommunity.com> wrote:
> Once again, I'd like for the badgering to stop. ?Thanks.

Tarek already agreed to that ("Fine with me. Let's move forward"). I
hope you will stop badgering him too.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Sep 24 04:07:03 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 24 Sep 2009 12:07:03 +1000
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both 	object have implementations
In-Reply-To: <5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>	<4AB8E136.8040706@simplistix.co.uk>	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>	<19129.39295.970536.916767@montanaro.dyndns.org>	<1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com>
Message-ID: <4ABAD447.408@gmail.com>

Mark Dickinson wrote:
> On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland <dinov at microsoft.com> wrote:
>> We are going to start contributing tests back real soon now.  I'm not sure
>> that these are the best tests to contribute as they require a version of
>> Python to compare against rather than being nice and stand alone.  But I'm
>> sure we have other tests which cover this as well just not as exhaustively.
>> We could also possibly check in the baseline file and then CPython could
>> compare it's self to previous versions but it'd probably be a pretty
>> big file - so it probably shouldn't be included in the standard install
>> in the tests directory.
> 
> How big is big?  For comparison, CPython's Lib/test/decimaltestdata
> directory alone is already over 4Mb, so maybe size isn't an issue?

For big test files, there's also the option of including download logic
in the test to go retrieve it from a known URL (I believe we already do
that for some of the codec tests).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Sep 24 04:16:01 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 24 Sep 2009 12:16:01 +1000
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <loom.20090923T191242-680@post.gmane.org>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<4AB8D910.8050601@simplistix.co.uk>	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>	<4AB92ADE.9060700@simplistix.co.uk>	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<loom.20090923T191242-680@post.gmane.org>
Message-ID: <4ABAD661.3020103@gmail.com>

Antoine Pitrou wrote:
> As far as I'm concerned, anything which looks intuitive enough (e.g. ini-style)
> and not overly complicated is fine. The details of the syntax aren't really
> important as long as they make sense, and don't get in the way.

One small comment before all this goes back to the distutils list: the
abbreviation "py" shouldn't be used to refer to the Python interpreter
as it is somewhat ambiguous due to the existence of the independent "py"
package [1]. The longer "python_version" has a genuine advantage in
clarity here.

Cheers,
Nick.

[1] http://codespeak.net/py/dist/index.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Sep 24 04:26:50 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 24 Sep 2009 12:26:50 +1000
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <loom.20090923T191242-680@post.gmane.org>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<4AB8D910.8050601@simplistix.co.uk>	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>	<4AB92ADE.9060700@simplistix.co.uk>	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<loom.20090923T191242-680@post.gmane.org>
Message-ID: <4ABAD8EA.9060202@gmail.com>

Antoine Pitrou wrote:
> As far as I'm concerned, anything which looks intuitive enough (e.g. ini-style)
> and not overly complicated is fine. The details of the syntax aren't really
> important as long as they make sense, and don't get in the way.

One small comment before all this goes back to the distutils list: the
abbreviation "py" shouldn't be used to refer to the Python interpreter
as it is somewhat ambiguous due to the existence of the independent "py"
package [1]. The longer "python_version" has a genuine advantage in
clarity here.

Cheers,
Nick.

[1] http://codespeak.net/py/dist/index.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From pje at telecommunity.com  Thu Sep 24 05:36:13 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 23 Sep 2009 23:36:13 -0400
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <ca471dc20909231905q27a3b1demf9678d35149df72@mail.gmail.com
 >
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<94bdd2610909231526l62c20d1cub6f2dff9d7d0480b@mail.gmail.com>
	<94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com>
	<94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.com>
	<20090924015718.B11343A4079@sparrow.telecommunity.com>
	<ca471dc20909231905q27a3b1demf9678d35149df72@mail.gmail.com>
Message-ID: <20090924033618.6AFE33A4079@sparrow.telecommunity.com>

At 07:05 PM 9/23/2009 -0700, Guido van Rossum wrote:
>On Wed, Sep 23, 2009 at 6:57 PM, P.J. Eby <pje at telecommunity.com> wrote:
> > Once again, I'd like for the badgering to stop.  Thanks.
>
>Tarek already agreed to that ("Fine with me. Let's move forward"). I
>hope you will stop badgering him too.

The only reason I said "once again" above instead of, "One last 
time", is because I didn't want to be misconstrued as making some 
sort of threat.  However, I can assure you, the above is the very 
last time I will make that request.

In any case, I hadn't yet seen the message of Tarek's you allude to 
above, in large part because it wasn't actually *addressed* to 
me.  It went out only to Python-Dev, whereas the message calling me 
paranoid went straight to my inbox.

I doubt that it was intentional on Tarek's part (checking the thread 
shows that Antoine's the one who first cut the cc: list), but the 
fact remains that I hadn't seen it when I sent my 
message.  Otherwise, I'd have replied in *that* thread, rather than this one.


From david.lyon at preisshare.net  Thu Sep 24 06:23:31 2009
From: david.lyon at preisshare.net (David Lyon)
Date: Thu, 24 Sep 2009 00:23:31 -0400
Subject: [Python-Dev] thinking about 2.7 / buildbots / testing
In-Reply-To: <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain>
Message-ID: <ce460522eead8d5fe610ee5ca4adbcaa@preisshare.net>

On Wed, 23 Sep 2009 15:13:55 -0000, exarkun at twistedmatrix.com wrote:
> Quite a few years of experience with a distributed team of build slave 
> managers has shown me that by far the most reliable way to keep slaves 
> online is to have them managed by a dedicated team.  This team doesn't 
> need to be small, but since finding dedicated people can sometimes be 
> challenging, I think small teams are the most likely outcome (possibly 
> resulting in a team of one).  

> ..

> Casual volunteers generally just won't keep up with these tasks.

True.

> I suggest finding someone who's seriously interested in the quality of 
> CPython and giving them the responsibility of keeping things operating 
> properly.  This includes paying attention to the status of slaves, 
> cajoling hardware operators into bringing hosts back online and fixing 
> network issues, and finding replacements of the appropriate type 
> (hardware/software platform) when a slave host is permanently lost.

Well, I'm a system administrator now, and a casual developer.

Yes, you need somebody who watches machine control panels and can
type emails on python lists at the same time.

> I would also personally recommend that this person first (well, after 
> tracking down all the slave operators and convincing them to bring their 
> slaves back online) acquire shell access to all of the slave machines so 
> that the owners of the slave hosts themselves no longer need to be the 
> gating factor for most issues.

Depends on where the machines are. There are good tools do check all
automatically. Nagios is one.

Anyway, this would suite my work schedule for the next 12 months.

Do we already have the machines? or do they need to be acquired?

David




From ctb at msu.edu  Thu Sep 24 06:40:27 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 23 Sep 2009 21:40:27 -0700
Subject: [Python-Dev] thinking about 2.7 / buildbots / testing
In-Reply-To: <ce460522eead8d5fe610ee5ca4adbcaa@preisshare.net>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain>
	<ce460522eead8d5fe610ee5ca4adbcaa@preisshare.net>
Message-ID: <20090924044027.GB22595@idyll.org>

On Thu, Sep 24, 2009 at 12:23:31AM -0400, David Lyon wrote:
> On Wed, 23 Sep 2009 15:13:55 -0000, exarkun at twistedmatrix.com wrote:
> 
> > I would also personally recommend that this person first (well, after 
> > tracking down all the slave operators and convincing them to bring their 
> > slaves back online) acquire shell access to all of the slave machines so 
> > that the owners of the slave hosts themselves no longer need to be the 
> > gating factor for most issues.
> 
> Depends on where the machines are. There are good tools do check all
> automatically. Nagios is one.
> 
> Anyway, this would suite my work schedule for the next 12 months.
> 
> Do we already have the machines? or do they need to be acquired?

Hi David,

as Brett leaked, I am developing software towards making this buildbot
dilemma easier to deal with.  It's not really ready for primetime yet;
I'll send you a link off-list.

I am also (physically) hosting Snakebite, which Trent Nelson will be
tackling over the next few months.

And, finally, I now have Windows VMs (XP, Vista, 7) and a Mac OS X box
that I can dedicate to Python purposes.  I even have a student that will
be monitoring those boxes.  I am happy to give you access to these
machines if you want to set up buildbot on them; buildbot is compatible
with my own endeavors, so that would actually help me out ;)

I will be buying more/better/bigger Linux hardware over the next few
months, too, and I am happy to slice that off to provide more VMs --
basically whatever x86-based platforms people think are needed.

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From mal at egenix.com  Thu Sep 24 10:55:24 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 24 Sep 2009 10:55:24 +0200
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <d096608c6a93af28ff7d5907f9f99b06@preisshare.net>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<4AB9D2FC.6040705@egenix.com>
	<d096608c6a93af28ff7d5907f9f99b06@preisshare.net>
Message-ID: <4ABB33FC.5080105@egenix.com>

David Lyon wrote:
> On Wed, 23 Sep 2009 09:49:16 +0200, "M.-A. Lemburg" <mal at egenix.com> wrote:
> 
>> While it's a good idea to put up some form of meta-data
>> into an index, I wonder why you are using setup.cfg
>> for this.
>>
>> setup.cfg has traditionally been used to configure distutils,
>> not to define meta-data. As such you wouldn't want to
>> put such a configuration file up on PyPI.
>>
>> Wouldn't it be better use a new file for this (with the
>> same syntax), e.g. metadata.cfg ?! This would then just
>> contain the meta data that needs to be published.
> 
> Forgive me if I answer this from being on the distutils-ml
> but it is easy to answer and might save somebody else time.
> 
> Currently, the metadata is stored within setup.py and the
> biggest issue with that is maintaining the version number.
> For instance, getting the same version number to go into the
> documentation files, etc. So consensus was that keeping that 
> information in setup.py is somewhat cumbersome and putting
> it somewhere else would be better. Tarek suggested setup.cfg
> and it seems to make the most sense.

So the idea is to make the setup() parameters settable via
the setup.cfg as well ?! (+1 on that) Or are you suggesting
that the setup.cfg file be the *only* place to set the version
number ? (I'd be -1 on that, because you often need to determine the
version number dynamically, e.g. when doing snapshot releases)

In my experience, it's usually better to place all the
setup() keyword parameters into a separate module
which can then be loaded by other tools as well - see e.g.
egenix-mx-base for what I mean:

http://www.egenix.com/products/python/mxBase/#Download

The setup.py module itself then just loads this configuration
module and uses the values from it to fire off distutils.

> By moving the metadata values into a configParser format file, 
> it would mean that external scripts could more easily 
> access/update the version number as required.
> 
> Of course, setup.cfg wouldn't get uploaded. Nobody would
> want to do that.
> 
> But distutils would create the .PKG_INFO file and metadata
> from configuration, and not from hardcoded string values
> within setup.py as it does now.

I lost you there... if you have the setup.py file available,
where is the problem in running "python setup.py somecommand"
to extract the metadata into some other file ?

As far as I understood Tarek's summary, the idea was to
have a separate file with just the metadata to upload
to PyPI in order to have tools working on the PyPI
index extract that metadata without having to download
the whole installation package.

That's a very good idea.

The only problem I saw was the name of the file that Tarek
chose and whether or not he was actually referring to the
setup.cfg used for configuring distutils or not.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 24 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Thu Sep 24 13:13:58 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 24 Sep 2009 11:13:58 +0000 (UTC)
Subject: [Python-Dev] [OT] cutting the cc list
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
	<94bdd2610909231526l62c20d1cub6f2dff9d7d0480b@mail.gmail.com>
	<94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com>
	<94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.com>
	<20090924015718.B11343A4079@sparrow.telecommunity.com>
	<ca471dc20909231905q27a3b1demf9678d35149df72@mail.gmail.com>
	<ca471dc20909231905q27a3b1demf9678d35149df72@mail.gmail.com >
	<20090924033618.6AFE33A4079@sparrow.telecommunity.com>
Message-ID: <loom.20090924T131110-68@post.gmane.org>

P.J. Eby <pje <at> telecommunity.com> writes:
> 
> I doubt that it was intentional on Tarek's part (checking the thread 
> shows that Antoine's the one who first cut the cc: list), but the 
> fact remains that I hadn't seen it when I sent my 
> message.

Just FYI, there are two reasons that I cut the CC list:
- I usually do it deliberately, when replying from e-mail, because I don't want
people to get duplicates, nor to get an e-mail not coming from the list which
might then not match with their own ML filters (based on whatever headers the ML
machine adds to e-mails)
- In the python-dev case, I reply through gmane, and gmane doesn't do any CC, it
posts only to the list

cheers

Antoine.



From rdmurray at bitdance.com  Thu Sep 24 15:08:50 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 24 Sep 2009 09:08:50 -0400 (EDT)
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both  object have implementations
In-Reply-To: <4ABAD447.408@gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
	<1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com>
	<4ABAD447.408@gmail.com>
Message-ID: <Pine.LNX.4.64.0909240906260.18193@kimball.webabinitio.net>

On Thu, 24 Sep 2009 at 12:07, Nick Coghlan wrote:
> Mark Dickinson wrote:
>> On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland <dinov at microsoft.com> wrote:
>>> We are going to start contributing tests back real soon now.  I'm not sure
>>> that these are the best tests to contribute as they require a version of
>>> Python to compare against rather than being nice and stand alone.  But I'm
>>> sure we have other tests which cover this as well just not as exhaustively.
>>> We could also possibly check in the baseline file and then CPython could
>>> compare it's self to previous versions but it'd probably be a pretty
>>> big file - so it probably shouldn't be included in the standard install
>>> in the tests directory.
>>
>> How big is big?  For comparison, CPython's Lib/test/decimaltestdata
>> directory alone is already over 4Mb, so maybe size isn't an issue?
>
> For big test files, there's also the option of including download logic
> in the test to go retrieve it from a known URL (I believe we already do
> that for some of the codec tests).

We do.  There's even support code in test.support for handling such
downloads relatively cleanly (and cleaning up afterward, though that
last was only added recently).

--David

From solipsis at pitrou.net  Thu Sep 24 15:49:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 24 Sep 2009 13:49:35 +0000 (UTC)
Subject: [Python-Dev] thinking about 2.7 / buildbots / testing
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain>
	<ce460522eead8d5fe610ee5ca4adbcaa@preisshare.net>
Message-ID: <h9ftdf$pqh$1@ger.gmane.org>

Le Thu, 24 Sep 2009 00:23:31 -0400, David Lyon a ?crit?:
> 
> Depends on where the machines are. There are good tools do check all
> automatically. Nagios is one.
> 
> Anyway, this would suite my work schedule for the next 12 months.
> 
> Do we already have the machines? or do they need to be acquired?

I have (root) access to a powerful x86-64 Debian machine that I might 
give you non-root access to, if it's enough. I'll have to ask the owner 
of the machine, however. Tell me if that can be useful.

Regards

Antoine.


From rdmurray at bitdance.com  Thu Sep 24 16:14:46 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 24 Sep 2009 10:14:46 -0400 (EDT)
Subject: [Python-Dev] thinking about 2.7 / buildbots / testing
In-Reply-To: <h9ftdf$pqh$1@ger.gmane.org>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain>
	<ce460522eead8d5fe610ee5ca4adbcaa@preisshare.net>
	<h9ftdf$pqh$1@ger.gmane.org>
Message-ID: <Pine.LNX.4.64.0909241005170.18193@kimball.webabinitio.net>

On Thu, 24 Sep 2009 at 13:49, Antoine Pitrou wrote:
> Le Thu, 24 Sep 2009 00:23:31 -0400, David Lyon a ??crit??:
>>
>> Depends on where the machines are. There are good tools do check all
>> automatically. Nagios is one.
>>
>> Anyway, this would suite my work schedule for the next 12 months.
>>
>> Do we already have the machines? or do they need to be acquired?
>
> I have (root) access to a powerful x86-64 Debian machine that I might
> give you non-root access to, if it's enough. I'll have to ask the owner
> of the machine, however. Tell me if that can be useful.

I've been a sysadmin for years (among other things).  If Jean-Paul's
suggestion of forming a small team to manage this is followed, I
volunteer to be part of that team.  And if Snakebite isn't enough I
have a well-connected platform I could run Linux buildbots on in
vserver virthosts, and a less-well-connected platform that could run
KVM based stuff.

--David (R. David Murray)

From solipsis at pitrou.net  Thu Sep 24 19:16:36 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 24 Sep 2009 17:16:36 +0000 (UTC)
Subject: [Python-Dev] thinking about 2.7 / buildbots / testing
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain>
	<ce460522eead8d5fe610ee5ca4adbcaa@preisshare.net>
	<h9ftdf$pqh$1@ger.gmane.org>
	<Pine.LNX.4.64.0909241005170.18193@kimball.webabinitio.net>
Message-ID: <loom.20090924T191537-918@post.gmane.org>

R. David Murray <rdmurray <at> bitdance.com> writes:
> 
> And if Snakebite isn't enough I
> have a well-connected platform I could run Linux buildbots on in
> vserver virthosts, and a less-well-connected platform that could run
> KVM based stuff.

I suppose Snakebite will be enough, but only once it is ready :)
We still need some buildbots (or any equivalent) in the meantime...

Regards

Antoine.



From martin at v.loewis.de  Fri Sep 25 01:25:12 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 25 Sep 2009 01:25:12 +0200
Subject: [Python-Dev] thinking about 2.7 / buildbots / testing
In-Reply-To: <ce460522eead8d5fe610ee5ca4adbcaa@preisshare.net>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>	<20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain>
	<ce460522eead8d5fe610ee5ca4adbcaa@preisshare.net>
Message-ID: <4ABBFFD8.8070707@v.loewis.de>


> Do we already have the machines? or do they need to be acquired?

Yes, the machines are all there, see

http://www.python.org/dev/buildbot/all/

Regards,
Martin

From martin at v.loewis.de  Fri Sep 25 01:27:52 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 25 Sep 2009 01:27:52 +0200
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
Message-ID: <4ABC0078.8030509@v.loewis.de>

>> Additionally, I'm very apprehensive about doing any kind of release
>> without the buildbots running. Does anyone know when they might be up?

When I (or somebody else) contacts all the slave operators and asks them
to restart the buildbot slaves.

Regards,
Martin

From exarkun at twistedmatrix.com  Fri Sep 25 02:06:54 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Fri, 25 Sep 2009 00:06:54 -0000
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <4ABC0078.8030509@v.loewis.de>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>
	<4ABC0078.8030509@v.loewis.de>
Message-ID: <20090925000654.1533.1151834611.divmod.xquotient.332@localhost.localdomain>

On 24 Sep, 11:27 pm, martin at v.loewis.de wrote:
>>>Additionally, I'm very apprehensive about doing any kind of release
>>>without the buildbots running. Does anyone know when they might be 
>>>up?
>
>When I (or somebody else) contacts all the slave operators and asks 
>them
>to restart the buildbot slaves.

Does this mean you're taking responsibility for this task?  Or are you 
looking for a volunteer?

Jean-Paul

From martin at v.loewis.de  Fri Sep 25 02:38:18 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 25 Sep 2009 02:38:18 +0200
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <20090925000654.1533.1151834611.divmod.xquotient.332@localhost.localdomain>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>	<bbaeab100909231103q5793c400q36d4add734a73005@mail.gmail.com>	<4ABC0078.8030509@v.loewis.de>
	<20090925000654.1533.1151834611.divmod.xquotient.332@localhost.localdomain>
Message-ID: <4ABC10FA.2040900@v.loewis.de>

>> When I (or somebody else) contacts all the slave operators and asks them
>> to restart the buildbot slaves.
> 
> Does this mean you're taking responsibility for this task?  Or are you
> looking for a volunteer?

I had been running this buildbot installation ever since I created it,
and will happily continue to take responsibility for it.

Regards,
Martin

From weebinn at gmail.com  Wed Sep 23 05:47:29 2009
From: weebinn at gmail.com (Wee Binn)
Date: Wed, 23 Sep 2009 11:47:29 +0800
Subject: [Python-Dev] surprised to "++" and "--"
Message-ID: <A2522CEB70274F08BE08F40983B0D8E3@graduate>

Hi,
    I know that there is no "++" or "--" operator in python, but if "var++" or something like that in my code(you know, most of C/C++ coders may like this),there is nothing wrong reported and program goes on just like expected!! 
    This is obscure, maybe a bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090923/231dd75b/attachment.htm>

From matthew at matthewwilkes.co.uk  Fri Sep 25 12:58:46 2009
From: matthew at matthewwilkes.co.uk (Matthew Wilkes)
Date: Fri, 25 Sep 2009 11:58:46 +0100
Subject: [Python-Dev] surprised to "++" and "--"
In-Reply-To: <A2522CEB70274F08BE08F40983B0D8E3@graduate>
References: <A2522CEB70274F08BE08F40983B0D8E3@graduate>
Message-ID: <22F36940-FCC8-49AB-A148-220EA3655E48@matthewwilkes.co.uk>

>     I know that there is no "++" or "--" operator in python, but if  
> "var++" or something like that in my code(you know, most of C/C++  
> coders may like this),there is nothing wrong reported and program  
> goes on just like expected!!
>     This is obscure, maybe a bug.

Hi,

Firstly, this list is for the development of Python, not with Python,  
questions about problems you're having should generally go to the  
users' list.  Bug reports should go to the bug tracker at http://bugs.python.org/

However, in this particular case, there's no point submitting it; you  
have made a mistake somewhere.  As you say, there is no ++ or -- unary  
postfix operator, but this DOES raise a SyntaxError:

> >>> var = 1
> >>> var++
>   File "<stdin>", line 1
>     var++
>         ^
> SyntaxError: invalid syntax


The prefix form is valid, as + and - are both valid prefix unary  
operators:

> >>> ++var
> 1
> >>> --var
> 1

Which are equivalent to:

> >>> +(+1)
> 1
> >>> -(-1)
> 1
>

If you were to try this with something that didn't implement __neg__  
and __pos__, such as strings, you'd get:

> >>> ++var
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: bad operand type for unary +

Hope this clarifies things for you,

Matthew

From facundobatista at gmail.com  Fri Sep 25 15:13:07 2009
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 25 Sep 2009 10:13:07 -0300
Subject: [Python-Dev] surprised to "++" and "--"
In-Reply-To: <A2522CEB70274F08BE08F40983B0D8E3@graduate>
References: <A2522CEB70274F08BE08F40983B0D8E3@graduate>
Message-ID: <e04bdf310909250613v7f79fb46v959fa296cb6d2162@mail.gmail.com>

2009/9/23 Wee Binn <weebinn at gmail.com>:

> ????I know that there is no "++" or "--" operator in python, but if "var++"
> or something like that in my code(you know, most of C/C++ coders may like
> this),there is nothing wrong reported and program goes on just like
> expected!!
> ????This is obscure, maybe a bug.

You can not write "var++":

>>> var = 5
>>> var++
   File "<ipython console>", line 1
     var++
          ^
SyntaxError: invalid syntax
>>>

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From status at bugs.python.org  Fri Sep 25 18:07:26 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 25 Sep 2009 18:07:26 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090925160726.09CAA78638@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (09/18/09 - 09/25/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2410 open (+34) / 16406 closed (+20) / 18816 total (+54)

Open issues with patches:   955

Average duration of open issues: 667 days.
Median duration of open issues: 423 days.

Open Issues Breakdown
   open  2373 (+31)
pending    36 ( +3)

Issues Created Or Reopened (54)
_______________________________

Socket error when launching IDLE                                 09/18/09
       http://bugs.python.org/issue6941    created  08jpurcell                    
                                                                               

email.generator.Generator memory consumption                     09/18/09
       http://bugs.python.org/issue6942    created  rpatterson                    
                                                                               

setup.py fails to find headers of system libffi                  09/19/09
       http://bugs.python.org/issue6943    created  Arfrever                      
       patch, easy                                                             

socket.getnameinfo raises SystemError on bogus input             09/19/09
CLOSED http://bugs.python.org/issue6944    created  marienz                       
       patch                                                                   

pprint.pprint does not pprint unsortable dicts in Python 3       09/19/09
CLOSED http://bugs.python.org/issue6945    created  aronacher                     
                                                                               

Document: Missing link to datetime.datetime                      09/19/09
CLOSED http://bugs.python.org/issue6946    created  ocean-city                    
                                                                               

Fix distutils test on windows (SO extension)                     09/19/09
CLOSED http://bugs.python.org/issue6947    created  ocean-city                    
       patch                                                                   

list                                                             09/19/09
CLOSED http://bugs.python.org/issue6948    created  nutregina                     
                                                                               

Support Berkeley DB 4.8                                          09/19/09
       http://bugs.python.org/issue6949    created  Arfrever                      
       patch                                                                   

online documentation error: PyObject* PyByteArray_Resize(PyObjec 09/19/09
CLOSED http://bugs.python.org/issue6950    created  kaizhu                        
                                                                               

OS X IDLE has two Preferences menus on 2.6.x with Tk > 8.4.7     09/20/09
CLOSED http://bugs.python.org/issue6951    created  ned.deily                     
       easy                                                                    

deprecated conversion from string constant to char *             09/20/09
       http://bugs.python.org/issue6952    created  barry-scott                   
                                                                               

readline documenation needs work                                 09/20/09
       http://bugs.python.org/issue6953    created  ronaldoussoren                
                                                                               

DISTUTILS_DEBUG causes stack trace, really simple fix            09/20/09
CLOSED http://bugs.python.org/issue6954    created  nedbat                        
                                                                               

Max() Not Working Properly                                       09/21/09
CLOSED http://bugs.python.org/issue6955    created  lovelygoo2                    
                                                                               

Test creation in unittest.TestProgram should be done in one plac 09/21/09
       http://bugs.python.org/issue6956    created  michael.foord                 
                                                                               

Extension modules fail to build on OS X 10.6 using python.org 2. 09/21/09
       http://bugs.python.org/issue6957    created  ned.deily                     
                                                                               

Add Python command line flags to configure logging               09/21/09
       http://bugs.python.org/issue6958    created  theller                       
       patch                                                                   

OS X 10.6 / Snow Leopard: building 2.6 maintenance release fails 09/21/09
CLOSED http://bugs.python.org/issue6959    created  cscscs                        
                                                                               

test_telnetlib gives spurious output                             09/21/09
       http://bugs.python.org/issue6960    created  pitrou                        
                                                                               

test_distutils failure                                           09/21/09
CLOSED http://bugs.python.org/issue6961    created  pitrou                        
                                                                               

traceback.format_exception_only does not return SyntaxError caro 09/21/09
       http://bugs.python.org/issue6962    created  thewtex                       
                                                                               

Add worker process lifetime to multiprocessing.Pool - patch incl 09/22/09
       http://bugs.python.org/issue6963    created  charlesc                      
       patch                                                                   

import new fails                                                 09/22/09
CLOSED http://bugs.python.org/issue6964    created  illume                        
                                                                               

tmpnam should not be used if tempfile or mkstemp are available   09/22/09
       http://bugs.python.org/issue6965    created  djc                           
                                                                               

Ability to refer to arguments in TestCase.fail* methods          09/22/09
       http://bugs.python.org/issue6966    created  tuben                         
                                                                               

codec windows1256 should be windows windows-1256                 09/22/09
CLOSED http://bugs.python.org/issue6967    created  DerSascha                     
       patch                                                                   

numpy extensions to distutils... are a source of improvements fo 09/22/09
       http://bugs.python.org/issue6968    created  illume                        
                                                                               

configparser                                                     09/22/09
CLOSED http://bugs.python.org/issue6969    created  exe                           
                                                                               

Redundant calls made to comparison methods.                      09/22/09
       http://bugs.python.org/issue6970    created  mark.dickinson                
       needs review                                                            

Add the SIO_KEEPALIVE_VALS option to socket.ioctl                09/22/09
CLOSED http://bugs.python.org/issue6971    created  krisvale                      
       patch, patch, easy                                                      

zipfile.ZipFile overwrites files outside destination path        09/22/09
       http://bugs.python.org/issue6972    created  schmir                        
                                                                               

subprocess.Popen.send_signal doesn't check whether the process h 09/22/09
       http://bugs.python.org/issue6973    created  milko.krachounov              
                                                                               

test_posix getcwd test leaves tmp  dir                           09/23/09
       http://bugs.python.org/issue6974    created  iandekit                      
                                                                               

symlinks incorrectly resolved on Linux                           09/23/09
       http://bugs.python.org/issue6975    created  swarecki                      
                                                                               

getcwd hangs and leaks mem on Solaris <= 9 in very long file nam 09/23/09
       http://bugs.python.org/issue6976    created  iandekit                      
                                                                               

Getopt documentation ambiguity                                   09/23/09
       http://bugs.python.org/issue6977    created  SilentGhost                   
                                                                               

compiler.transformer dict key bug d[1,] = 1                      09/23/09
       http://bugs.python.org/issue6978    created  kees                          
       patch                                                                   

Passing output arguments to an ActiveX application               09/23/09
CLOSED http://bugs.python.org/issue6979    created  benji13                       
                                                                               

fix ctypes build failure on armel-linux-gnueabi with -mfloat-abi 09/23/09
       http://bugs.python.org/issue6980    created  doko                          
                                                                               

locale.getdefaultlocale() envvars default code and documentation 09/23/09
       http://bugs.python.org/issue6981    created  scop                          
                                                                               

make clean does not remove pickle files                          09/23/09
CLOSED http://bugs.python.org/issue6982    created  egreen                        
       patch                                                                   

Add specific get_platform() for freebsd                          09/24/09
       http://bugs.python.org/issue6983    created  stefw                         
       patch                                                                   

typo in Unicode HOWTO                                            09/24/09
CLOSED http://bugs.python.org/issue6984    created  Yinon                         
                                                                               

range() fails with long integers                                 09/24/09
CLOSED http://bugs.python.org/issue6985    created  kszawala                      
                                                                               

_json crash on scanner/encoder initialization error              09/24/09
       http://bugs.python.org/issue6986    created  haypo                         
       patch                                                                   

Idle not start                                                   09/24/09
CLOSED http://bugs.python.org/issue6987    created  dorina_n2005                  
                                                                               

shlex.split() converts unicode input to UCS-4 output             09/24/09
       http://bugs.python.org/issue6988    created  fenner                        
                                                                               

Compile error  for Python-2.6.2 on Solaris                       09/24/09
       http://bugs.python.org/issue6989    created  ashish                        
                                                                               

threading.local subclasses don't cleanup their state and it gets 09/25/09
       http://bugs.python.org/issue6990    created  pjenvey                       
       patch, needs review                                                     

logging encoding failes some situation                           09/25/09
       http://bugs.python.org/issue6991    created  naoki                         
       patch                                                                   

multiple authors in setup.by                                     09/25/09
       http://bugs.python.org/issue6992    created  techtonik                     
                                                                               

importing of "time" module is terribly slow                      09/25/09
       http://bugs.python.org/issue6993    created  ndyankov                      
                                                                               

enumerate dosctring has a typo                                   09/25/09
       http://bugs.python.org/issue6994    created  nessita                       
                                                                               



Issues Now Closed (41)
______________________

Passing 'None' if argtype is set to POINTER(...) doesn't always   283 days
       http://bugs.python.org/issue4606    theller                       
       patch                                                                   

Structure sub-subclass does not initialize with base class posit  237 days
       http://bugs.python.org/issue5042    theller                       
       patch                                                                   

urllib ctypes error on Mac OS X Server 10.5                       200 days
       http://bugs.python.org/issue5413    ronaldoussoren                
       patch                                                                   

2.6.2 Python Manuals CHM file seems broken                        159 days
       http://bugs.python.org/issue5764    dx617                         
                                                                               

os.popen causes illegal seek on AIX in Python 3.1rc               106 days
       http://bugs.python.org/issue6236    nestor                        
       patch                                                                   

Upgrading xml.etree to ElementTree 1.3                            104 days
       http://bugs.python.org/issue6272    senthil.kumaran               
                                                                               

encode and decode should accept 'errors' as a keyword argument     93 days
       http://bugs.python.org/issue6300    benjamin.peterson             
       patch, easy                                                             

solaris/aix: Py_Initialize: can't initialize sys standard stream   87 days
       http://bugs.python.org/issue6348    pitrou                        
                                                                               

ctypes.PyThreadState_SetAsyncExc does not work on linux 64bit ma   49 days
       http://bugs.python.org/issue6613    theller                       
                                                                               

time.strftime does unnecessary range check                         19 days
       http://bugs.python.org/issue6823    brett.cannon                  
       patch, easy                                                             

urllib.urlopen crashes in a thread on Snow Leopard                 13 days
       http://bugs.python.org/issue6851    ronaldoussoren                
       patch, 26backport                                                       

Support system readline on OS X 10.6                               15 days
       http://bugs.python.org/issue6872    mark.dickinson                
       patch                                                                   

enable compilation of readline module on Mac OS X 10.5 and 10.6    12 days
       http://bugs.python.org/issue6877    zvezdan                       
       patch, 26backport, needs review                                         

incorrect signature in doc for PyByteArray_Resize                   9 days
       http://bugs.python.org/issue6881    ezio.melotti                  
                                                                               

pdb documentation shows running as script using "python" not "py    7 days
       http://bugs.python.org/issue6885    georg.brandl                  
                                                                               

ctypes compilation error on SnowLeopard                             5 days
       http://bugs.python.org/issue6918    ronaldoussoren                
                                                                               

Doc for locals and vars                                             3 days
       http://bugs.python.org/issue6925    tjreedy                       
                                                                               

[PATCH] PyUnicode_DecodeUTF16 docs wrong (byteorder param)          1 days
       http://bugs.python.org/issue6930    georg.brandl                  
       patch                                                                   

[PATCH] postflight.framework script (from the Mac OS X .dmg inst    3 days
       http://bugs.python.org/issue6934    ronaldoussoren                
       patch                                                                   

multiprocessing lock on OS X Snow Leopard dumps core                2 days
       http://bugs.python.org/issue6937    ronaldoussoren                
                                                                               

cannot compute sizeof wchar_t on OS X 10.5.7, Python 2.6.2          0 days
       http://bugs.python.org/issue6940    rajarshi                      
                                                                               

socket.getnameinfo raises SystemError on bogus input                0 days
       http://bugs.python.org/issue6944    georg.brandl                  
       patch                                                                   

pprint.pprint does not pprint unsortable dicts in Python 3          1 days
       http://bugs.python.org/issue6945    aronacher                     
                                                                               

Document: Missing link to datetime.datetime                         0 days
       http://bugs.python.org/issue6946    georg.brandl                  
                                                                               

Fix distutils test on windows (SO extension)                        0 days
       http://bugs.python.org/issue6947    tarek                         
       patch                                                                   

list                                                                0 days
       http://bugs.python.org/issue6948    mark.dickinson                
                                                                               

online documentation error: PyObject* PyByteArray_Resize(PyObjec    0 days
       http://bugs.python.org/issue6950    ezio.melotti                  
                                                                               

OS X IDLE has two Preferences menus on 2.6.x with Tk > 8.4.7        1 days
       http://bugs.python.org/issue6951    ronaldoussoren                
       easy                                                                    

DISTUTILS_DEBUG causes stack trace, really simple fix               1 days
       http://bugs.python.org/issue6954    tarek                         
                                                                               

Max() Not Working Properly                                          0 days
       http://bugs.python.org/issue6955    benjamin.peterson             
                                                                               

OS X 10.6 / Snow Leopard: building 2.6 maintenance release fails    1 days
       http://bugs.python.org/issue6959    cscscs                        
                                                                               

test_distutils failure                                              1 days
       http://bugs.python.org/issue6961    tarek                         
                                                                               

import new fails                                                    1 days
       http://bugs.python.org/issue6964    georg.brandl                  
                                                                               

codec windows1256 should be windows windows-1256                    0 days
       http://bugs.python.org/issue6967    georg.brandl                  
       patch                                                                   

configparser                                                        0 days
       http://bugs.python.org/issue6969    georg.brandl                  
                                                                               

Add the SIO_KEEPALIVE_VALS option to socket.ioctl                   3 days
       http://bugs.python.org/issue6971    krisvale                      
       patch, patch, easy                                                      

Passing output arguments to an ActiveX application                  0 days
       http://bugs.python.org/issue6979    mark.dickinson                
                                                                               

make clean does not remove pickle files                             1 days
       http://bugs.python.org/issue6982    mark.dickinson                
       patch                                                                   

typo in Unicode HOWTO                                               0 days
       http://bugs.python.org/issue6984    georg.brandl                  
                                                                               

range() fails with long integers                                    0 days
       http://bugs.python.org/issue6985    brett.cannon                  
                                                                               

Idle not start                                                      0 days
       http://bugs.python.org/issue6987    amaury.forgeotdarc            
                                                                               



Top Issues Most Discussed (10)
______________________________

 19 Add Python command line flags to configure logging                 4 days
open    http://bugs.python.org/issue6958   

 12 enable compilation of readline module on Mac OS X 10.5 and 10.6   12 days
closed  http://bugs.python.org/issue6877   

  9 improve xrange.__contains__                                      785 days
open    http://bugs.python.org/issue1766304

  8 IDLE 2.6.1 locks up on Mac OS 10.6                                17 days
open    http://bugs.python.org/issue6864   

  7 shlex.split() converts unicode input to UCS-4 output               1 days
open    http://bugs.python.org/issue6988   

  7 Add specific get_platform() for freebsd                            2 days
open    http://bugs.python.org/issue6983   

  7 Add gamma function, error functions and other C99 math.h functi  437 days
open    http://bugs.python.org/issue3366   

  6 make clean does not remove pickle files                            1 days
closed  http://bugs.python.org/issue6982   

  5 range() fails with long integers                                   0 days
closed  http://bugs.python.org/issue6985   

  5 deprecated conversion from string constant to char *               5 days
open    http://bugs.python.org/issue6952   




From waqas805 at hotmail.com  Fri Sep 25 20:28:55 2009
From: waqas805 at hotmail.com (waqas ahmad)
Date: Fri, 25 Sep 2009 20:28:55 +0200
Subject: [Python-Dev] help required
Message-ID: <BAY111-W1161B0BF044923728BF24987D90@phx.gbl>


Hi, 

 

I dont know it is the right place to post this question. I need help to change one search code line . can you help me please.
 
here is my search method code:
 
search=re.compile("^#acl InternationalGroup.*\n", re.M).search(pagetext)
    if search:
        ret=search.group()



here i am searching for "#acl InternationalGroup" in the pageText and when it true is then give me search group.
 
 
I want to change this for following requirement:
 
I want to search  for "#acl InternationalGroup" and  "CatInternational" in the pageText.
when "#acl InternationalGroup" is not there but "CatInternational" is there. then return me search group.

I shall be thankful to you for any help.
 
Best Regards, 
Waqas




 

 		 	   		  
_________________________________________________________________
Invite your mail contacts to join your friends list with Windows Live Spaces. It's easy!
http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090925/c5632fe3/attachment.htm>

From fuzzyman at voidspace.org.uk  Fri Sep 25 21:30:01 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 25 Sep 2009 20:30:01 +0100
Subject: [Python-Dev] help required
In-Reply-To: <BAY111-W1161B0BF044923728BF24987D90@phx.gbl>
References: <BAY111-W1161B0BF044923728BF24987D90@phx.gbl>
Message-ID: <4ABD1A39.3070906@voidspace.org.uk>

waqas ahmad wrote:
> Hi,
>  
> I dont know it is the right place to post this question. I need help 
> to change one search code line . can you help me please.
>  

Hello Waqas,

This is a list for the development *of* Python, not for development with 
Python. More appropriate lists, where hopefully there will be lots of 
people willing to help you, are python-list and python-tutor:

    http://mail.python.org/mailman/listinfo/python-list
    http://mail.python.org/mailman/listinfo/tutor

Or the google groups gateway for the Python-list (comp.lang.python):

    http://groups.google.com/group/comp.lang.python/topics

All the best,

Michael Foord

> here is my search method code:
>  
> search=re.compile("^#acl InternationalGroup.*\n", re.M).search(pagetext)
>     if search:
>         ret=search.group()
>
>
> here i am searching for "#acl InternationalGroup" in the pageText and 
> when it true is then give me search group.
>  
>  
> I want to change this for following requirement:
>  
> I want to search  for "#acl InternationalGroup" and  
> "CatInternational" in the pageText.
> when "#acl InternationalGroup" is not there but "CatInternational" is 
> there. then return me search group.
>
> I shall be thankful to you for any help.
>  
> Best Regards,
> Waqas
>
>
>
>  
>
>
>
> ------------------------------------------------------------------------
> Invite your mail contacts to join your friends list with Windows Live 
> Spaces. It's easy! Try it! 
> <http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us> 
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From python at mrabarnett.plus.com  Fri Sep 25 21:36:44 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 25 Sep 2009 20:36:44 +0100
Subject: [Python-Dev] help required
In-Reply-To: <BAY111-W1161B0BF044923728BF24987D90@phx.gbl>
References: <BAY111-W1161B0BF044923728BF24987D90@phx.gbl>
Message-ID: <4ABD1BCC.6080902@mrabarnett.plus.com>

waqas ahmad wrote:
> Hi,
>  
> I dont know it is the right place to post this question. I need help to 
> change one search code line . can you help me please.
>  
> here is my search method code:
>  
> search=re.compile("^#acl InternationalGroup.*\n", re.M).search(pagetext)
>     if search:
>         ret=search.group()
> 
> 
> here i am searching for "#acl InternationalGroup" in the pageText and 
> when it true is then give me search group.
>  
>  
> I want to change this for following requirement:
>  
> I want to search  for "#acl InternationalGroup" and  "CatInternational" 
> in the pageText.
> when "#acl InternationalGroup" is not there but "CatInternational" is 
> there. then return me search group.
> 
> I shall be thankful to you for any help.
>  
I'm not clear whether you want the search to succeed if pageText
contains either, or if pageText contains "CatInternational" but not
"#acl InternationalGroup".

Whichever you want, you could use 2 simple separate regular expressions
or 1 more complicated regular expression.

Search for either:

search = re.compile("^#acl InternationalGroup.*\n", re.M).search(pagetext)
if not search:
     search = re.compile("^CatInternational.*\n", re.M).search(pagetext)
if search:
     ret = search.group()

Search for one but not the other:

search = re.compile("^CatInternational.*\n", re.M).search(pagetext)
if search:
     search_2 = re.compile("^#acl InternationalGroup.*\n", 
re.M).search(pagetext)
     if not search_2:
         ret = search.group()


From barry at python.org  Fri Sep 25 22:14:56 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 25 Sep 2009 16:14:56 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <25524412.post@talk.nabble.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-C736AE.09230710092009@ger.gmane.org>
	<F10025B7-8AEA-4FD2-88C4-231F8BD77458@mac.com>
	<B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>
	<25524412.post@talk.nabble.com>
Message-ID: <ADCF1B7F-B679-41F7-90A5-15733F4F736F@python.org>

On Sep 19, 2009, at 2:51 PM, qwavel wrote:

> What about this bug:
> http://bugs.python.org/issue3890
> It appears to me that the SSL module is broken in the 2.6.x line on  
> all
> platforms in one of its most common uses (non-blocking).  It also  
> seems that
> the problem and solution are well understood, so the solution would  
> make a
> good candidate for 2.6.3?

I made this a release blocker, but reserve the right to bump it down  
again.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090925/d1d2d90b/attachment.pgp>

From jnoller at gmail.com  Fri Sep 25 22:18:06 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 25 Sep 2009 15:18:06 -0500
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <ADCF1B7F-B679-41F7-90A5-15733F4F736F@python.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-C736AE.09230710092009@ger.gmane.org>
	<F10025B7-8AEA-4FD2-88C4-231F8BD77458@mac.com>
	<B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>
	<25524412.post@talk.nabble.com>
	<ADCF1B7F-B679-41F7-90A5-15733F4F736F@python.org>
Message-ID: <4222a8490909251318h604038b7o45444d300417f317@mail.gmail.com>

On Fri, Sep 25, 2009 at 3:14 PM, Barry Warsaw <barry at python.org> wrote:
> On Sep 19, 2009, at 2:51 PM, qwavel wrote:
>
>> What about this bug:
>> http://bugs.python.org/issue3890
>> It appears to me that the SSL module is broken in the 2.6.x line on all
>> platforms in one of its most common uses (non-blocking). ?It also seems
>> that
>> the problem and solution are well understood, so the solution would make a
>> good candidate for 2.6.3?
>
> I made this a release blocker, but reserve the right to bump it down again.
>
> -Barry
>

Barry - this is your call, but I think
http://bugs.python.org/issue6990 should be a rel blocker too.

jesse

From db3l.net at gmail.com  Fri Sep 25 22:19:57 2009
From: db3l.net at gmail.com (David Bolen)
Date: Fri, 25 Sep 2009 16:19:57 -0400
Subject: [Python-Dev] thinking about 2.7
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>
	<20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain>
Message-ID: <m2d45edbf6.fsf@valheru.db3l.homeip.net>

exarkun at twistedmatrix.com writes:

> It's easy for someone to volunteer to set up a new slave.  It's even
> easy to make sure it keeps running for 6 months.  But it's not as easy
> to keep it running indefinitely.  This isn't about the software
> involved (at least not entirely).  It's about someone paying attention
> to whether the slave restarts on reboots, and about paying attention
> to whether the slave host has lots its network connection, or been
> decomissioned, or whether a system upgrade disabled the slave, or
> whatever other random administrative-like tasks are necessary to keep
> things running.  Casual volunteers generally just won't keep up with
> these tasks.

One suggestion from my perspective as a buildbot maintainer who, IMHO,
has maintained generally stable and fairly well monitored buildbots
(XP-4 and FreeBSD) for a while now....

I don't know if it's possible but it would be great if the buildbot
master could send an email to a buildbot maintainer if it thinks a
buildbot is down.  There have actually been times in the past when
from the perspective of my buildbot things were just dandy, but the
master showed it as down.  Stopping and restarting the buildbot,
forcing it to reconnect, cleaned things up, but I have to monitor the
web status page which is a manual process.

And since it's the master's perspective that really matters when it
comes to a buildbot being available, getting status from the master
would be very helpful.

-- David


From barry at python.org  Fri Sep 25 22:23:22 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 25 Sep 2009 16:23:22 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <4222a8490909251318h604038b7o45444d300417f317@mail.gmail.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-C736AE.09230710092009@ger.gmane.org>
	<F10025B7-8AEA-4FD2-88C4-231F8BD77458@mac.com>
	<B9761462-C294-4A1A-9CC1-7E80FE36EEF5@python.org>
	<25524412.post@talk.nabble.com>
	<ADCF1B7F-B679-41F7-90A5-15733F4F736F@python.org>
	<4222a8490909251318h604038b7o45444d300417f317@mail.gmail.com>
Message-ID: <923DCCFF-3475-4A04-AC46-6362D40E3F7E@python.org>

On Sep 25, 2009, at 4:18 PM, Jesse Noller wrote:

> Barry - this is your call, but I think
> http://bugs.python.org/issue6990 should be a rel blocker too.

Done.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090925/1a7a6beb/attachment.pgp>

From martin at v.loewis.de  Sat Sep 26 01:03:52 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 26 Sep 2009 01:03:52 +0200
Subject: [Python-Dev] thinking about 2.7
In-Reply-To: <m2d45edbf6.fsf@valheru.db3l.homeip.net>
References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com>	<20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain>
	<m2d45edbf6.fsf@valheru.db3l.homeip.net>
Message-ID: <4ABD4C58.6030405@v.loewis.de>

> I don't know if it's possible but it would be great if the buildbot
> master could send an email to a buildbot maintainer if it thinks a
> buildbot is down.

I think it should be possible to detect that. I wouldn't recommend
immediate reaction, but rather a delayed reaction (say, 12h). Buildbot
is very well capable of tracking timeouts of many hours.

I could put that on my list of things to do, but if you beat me to
it, that would be even better.

One problem is that the buildbot configuration currently doesn't
have email address of all slave operators, but that would be possible
to add.

> And since it's the master's perspective that really matters when it
> comes to a buildbot being available, getting status from the master
> would be very helpful.

This sounds very reasonable. I'll ask on the buildbot list; maybe
somebody already has that implemented.

Regards,
Martin

From rob.cliffe at btinternet.com  Fri Sep 25 12:37:27 2009
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Fri, 25 Sep 2009 11:37:27 +0100
Subject: [Python-Dev] surprised to "++" and "--"
References: <A2522CEB70274F08BE08F40983B0D8E3@graduate>
Message-ID: <999354BF3D184B7CA51D6E0512729F9F@robslaptop>

I don't think this is quite true.

    ++var  and --var
are legal albeit redundant expressions that equal var

    var++ and var--
cause a SyntaxError, as they should.

Rob


  ----- Original Message ----- 
  From: Wee Binn 
  To: python-dev at python.org 
  Sent: Wednesday, September 23, 2009 4:47 AM
  Subject: [Python-Dev] surprised to "++" and "--"


  Hi,
      I know that there is no "++" or "--" operator in python, but if "var++" or something like that in my code(you know, most of C/C++ coders may like this),there is nothing wrong reported and program goes on just like expected!! 
      This is obscure, maybe a bug.


------------------------------------------------------------------------------


  _______________________________________________
  Python-Dev mailing list
  Python-Dev at python.org
  http://mail.python.org/mailman/listinfo/python-dev
  Unsubscribe: http://mail.python.org/mailman/options/python-dev/rob.cliffe%40btinternet.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090925/a8bfb05b/attachment.htm>

From barry at barrys-emacs.org  Sat Sep 26 15:23:14 2009
From: barry at barrys-emacs.org (Barry Scott)
Date: Sat, 26 Sep 2009 14:23:14 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AAFDA0D.5040602@scottdial.com>
	<Pine.LNX.4.64.0909151424210.6303@kimball.webabinitio.net>
	<loom.20090915T203952-716@post.gmane.org>
	<Pine.LNX.4.64.0909151501140.17020@kimball.webabinitio.net>
	<loom.20090915T211440-93@post.gmane.org>
	<Pine.LNX.4.64.0909151525410.17020@kimball.webabinitio.net>
	<1253044711.5678.56.camel@localhost>
	<Pine.LNX.4.64.0909151851190.17020@kimball.webabinitio.net>
	<4AB02471.7000701@scottdial.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
Message-ID: <D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>


On 16 Sep 2009, at 01:05, Peter Moody wrote:

> On Tue, Sep 15, 2009 at 4:34 PM, Scott Dial
> <scott+python-dev at scottdial.com> wrote:
>> R. David Murray wrote:
>>> On Tue, 15 Sep 2009 at 21:58, Antoine Pitrou wrote:
>>>> Le mardi 15 septembre 2009 ? 15:48 -0400, R. David Murray a ?crit :
>>>>> However, I do not think
>>>>> that the proposed API should accept, eg, IPv4Network 
>>>>> ('192.168.1.1/24')
>>>>> as valid.  That's just too confusing and error prone.
>>>>
>>>> Indeed, it should throw some kind of ValueError instead.
>>>
>>> Peter, what do you think?
>
> I disagree. It seems overly pedantic to throw a valueerror on this.
> IPy does (used to do?) this and it's one of the primary reasons I
> wrote ipaddr.

I've seen user interfaces accept 192.168.1.1/24 as a short cut
to set the ipaddr and netmask on an interface.

For that use being able to parse that string into an IP Address and
a Net Mask is what they want.

If this is a common idiom having a utility function that returned
an IPv4Address('192.168.1.1') and IPv4Network('192.168.0.0')
would be useful.

If someone really thinks that '192.168.1.1/16' is a network any
good UI should detect it is invalid report that to the user.

If you think this is a decision that the only the caller can decide
why not add a keyword param to control the parsing?

IPv4Network( '192.168.1.1/16', syntax='loose' )

(I'm sure syntax='loose' is not right but you get the idea.)

Barry


From barry at barrys-emacs.org  Sat Sep 26 14:46:00 2009
From: barry at barrys-emacs.org (Barry Scott)
Date: Sat, 26 Sep 2009 13:46:00 +0100
Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk
Message-ID: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>

I'm working with http://svn.python.org/projects/python/trunk on Mac OS  
X 10.6.1
using Apples xcode gcc 4.2.1.

When I run the following commands:

	./configure --enable-framework --with-universal-archs=32-bit | tee  
build.config.log
	make clean all | tee build.make.log

I end up with a x86_64 Python image.

No matter what I use for archs its always the same.

I would expect to see -arch arg to GCC but it is not there.

	export CFLAG="-arch i386"

did not work either.

Am I doing something wrong or is this broken on trunk?

Barry


From martin at v.loewis.de  Sat Sep 26 19:44:45 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 26 Sep 2009 19:44:45 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <h8sd2n$n1g$1@ger.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB0C7A9.3030001@gmail.com>	<4AB15FD3.5040804@canterbury.ac.nz>	<Pine.LNX.4.64.0909161808480.17020@kimball.webabinitio.net>	<4AB17B29.8050708@canterbury.ac.nz>	<20090917003027.3AC245AC0D6@longblack.object-craft.com.au>	<8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com>	<20090917013219.6A3895AC0D6@longblack.object-craft.com.au>	<8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com>	<20090917032126.66F896006DE@longblack.object-craft.com.au>	<8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com>	<4AB1B2A8.9050204@scottdial.com>
	<h8sd2n$n1g$1@ger.gmane.org>
Message-ID: <4ABE530D.8090609@v.loewis.de>

>  - Masks are also 32- (128-) bit integers, which happen to have the
>    property that their leftmost N bits are all zero and the rest are all
>    one.

As a side note, I would be in favor of dropping the concept of a mask
from the library, and only support a prefix length.

IPv6 doesn't support masks at all, and even for IPv4, I think there are
conventions (if not RFCs) against using them in a way that does not
correspond to a prefix length.

Regards,
Martin

From martin at v.loewis.de  Sat Sep 26 19:59:25 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 26 Sep 2009 19:59:25 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090917004138.2177C5AC0D6@longblack.object-craft.com.au>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4AB03909.3090608@trueblade.com>	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>	<200909162053.00354.steve@pearwood.info>	<4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>	<eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>
	<20090917004138.2177C5AC0D6@longblack.object-craft.com.au>
Message-ID: <4ABE567D.6080808@v.loewis.de>

> I don't think the RFCs forbid the zero address being used


RFC 1122 does: "IP addresses are not permitted to have the value 0 or -1
for any of the <Host-number>, <Network-number>, or <Subnet-
number> fields (except in the special cases listed above)."

RFC 3021 modifies this requirement, allowing the zero address to
be used for an 31-bit prefix.

Regards,
Martin

From martin at v.loewis.de  Sat Sep 26 20:19:16 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 26 Sep 2009 20:19:16 +0200
Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for	the
 Python Standard Library
In-Reply-To: <20090916090849.GH12810@ruber.office.udmvt.ru>
References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com>	<h6gjku$t2n$1@ger.gmane.org>
	<4A8C0267.5020408@wildenhain.de>	<8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com>	<Pine.LNX.4.64.0908191148530.30683@kimball.webabinitio.net>	<8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com>	<4A8C4264.2040708@trueblade.com>	<d9047f780908191128p12107a1exc20dabc34ddff5de@mail.gmail.com>
	<20090916090849.GH12810@ruber.office.udmvt.ru>
Message-ID: <4ABE5B24.1090003@v.loewis.de>

>>> I think using .network and .broadcast are pretty well understood to be the
>>> [0] and [-1] of the network address block. I don't think we want to start
>>> creating new terms or access patterns here.
>>>
>>> +1 on leaving .network and .broadcast as-is (including returning a
>>> IPvXAddress object).
>>>
>> -1.  I think 'network.number' or 'network.zero' is a lot clearer than
>> 'network.network'.  Maybe '.broadcast' would be okay, as long as it *can* be
>> adjusted for those unusual, or maybe even only hypothetical, networks where
>> it is not the [-1].
> Real life example: network with a /31 mask.
> There are only two hosts: 0 and 1
> first host configures the other's host as broadcast address and vice versa.
> NOTE - broadcasts are different here!

This is RFC 3021. IIUC, it does not support directed broadcast; only
link-local broadcast can be used on that link.

So ISTM that .broadcast should raise an exception on a /31 network. Any
installation that configures the partner as the broadcast address is
broken (somebody correct me if I'm wrong).

> Another real life examples include /32 networks on PPP. Just a point-to-point.
> No need for broadcasts and networks, a host just have one IP address and
> send all traffic via point-to-point link, no addressing is required there.
> This is a working dialup configuration, it works for me, it works for you, probably.
> It is not weird, it is common, it is used for PPPoE, for ADSL, for dialup.

So where is that defined?

Regards,
Martin

From drkjam at gmail.com  Sat Sep 26 21:07:21 2009
From: drkjam at gmail.com (DrKJam)
Date: Sat, 26 Sep 2009 20:07:21 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4ABE567D.6080808@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>
	<20090917004138.2177C5AC0D6@longblack.object-craft.com.au>
	<4ABE567D.6080808@v.loewis.de>
Message-ID: <538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com>

2009/9/26 "Martin v. L?wis" <martin at v.loewis.de>

> > I don't think the RFCs forbid the zero address being used
>
>
> RFC 1122 does: "IP addresses are not permitted to have the value 0 or -1
> for any of the <Host-number>, <Network-number>, or <Subnet-
> number> fields (except in the special cases listed above)."
>
> The current version of the PEP and reference implementation do not mention
or deal with IPv4 classful addressing (A, B, C, D and E). It would be good
to know if any of this (admittedly older yet no less important)
functionality is going to be supported. If the library is to concentrate
solely on classless addressing (i.e. CIDR) please can this be stated in
future revisions of the PEP.


> RFC 3021 modifies this requirement, allowing the zero address to
> be used for an 31-bit prefix.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/drkjam%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090926/4de590f6/attachment.htm>

From daniel at stutzbachenterprises.com  Sat Sep 26 22:14:24 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sat, 26 Sep 2009 15:14:24 -0500
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB03909.3090608@trueblade.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>
	<20090917004138.2177C5AC0D6@longblack.object-craft.com.au>
	<4ABE567D.6080808@v.loewis.de>
	<538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com>
Message-ID: <eae285400909261314t6a00ecf7p2194ef7bdb0bb822@mail.gmail.com>

On Sat, Sep 26, 2009 at 2:07 PM, DrKJam <drkjam at gmail.com> wrote:

> The current version of the PEP and reference implementation do not mention
> or deal with IPv4 classful addressing (A, B, C, D and E). It would be good
> to know if any of this (admittedly older yet no less important)
> functionality is going to be supported. If the library is to concentrate
> solely on classless addressing (i.e. CIDR) please can this be stated in
> future revisions of the PEP.
>

Classful addressing was deprecated more than 15 years ago!

Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such
scenarios are no longer operationally relevant."

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
"The plumage don't enter into it. It's stone dead."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090926/993f6bfe/attachment-0001.htm>

From drkjam at gmail.com  Sat Sep 26 23:57:00 2009
From: drkjam at gmail.com (DrKJam)
Date: Sat, 26 Sep 2009 22:57:00 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <eae285400909261314t6a00ecf7p2194ef7bdb0bb822@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>
	<20090917004138.2177C5AC0D6@longblack.object-craft.com.au>
	<4ABE567D.6080808@v.loewis.de>
	<538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com>
	<eae285400909261314t6a00ecf7p2194ef7bdb0bb822@mail.gmail.com>
Message-ID: <538a660a0909261457n1a956bc0i1524e58c4e48be4e@mail.gmail.com>

2009/9/26 Daniel Stutzbach <daniel at stutzbachenterprises.com>

> On Sat, Sep 26, 2009 at 2:07 PM, DrKJam <drkjam at gmail.com> wrote:
>
>> The current version of the PEP and reference implementation do not mention
>> or deal with IPv4 classful addressing (A, B, C, D and E). It would be good
>> to know if any of this (admittedly older yet no less important)
>> functionality is going to be supported. If the library is to concentrate
>> solely on classless addressing (i.e. CIDR) please can this be stated in
>> future revisions of the PEP.
>>
>
> Classful addressing was deprecated more than 15 years ago!
>
> Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such
> scenarios are no longer operationally relevant."


Interesting as evidence of classful IPv4 behaviour seems fairly pervasive in
current IP stacks and software that supports IPv4 addressing (e.g.
PostgreSQL inet and cidr data types).

Here's an excerpt from the 'ifconfig' man page (taken from an Ubuntu 9.04
install) :-
^^^^^^^^^^^^^^^^^^^^^^^
...
netmask addr

Set the IP network mask for this interface. This value defaults to the usual
class A, B or C network mask (as derived from the interface IP address). but
it can be set to any value.
...
^^^^^^^^^^^^^^^^^^^^^^^

The point being that you can't always assume /32 implicitly for all use
cases.

Here is how IP addresses without an explicit prefix or netmask are currently
handled :-

>>> import ipaddr
>>> ipaddr.IPv4Network('10.0.0.1')
IPv4Network('10.0.0.1/32')

It may not be something we want to support (you could always force the end
user to specify a prefix or netmask explicitly). This is fair enough, but
let's indicate that it's a considered choice. Somewhere in the PEP text is
might be a good place for treatment of this topic.

> --
> Daniel Stutzbach, Ph.D.
> President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
> "The plumage don't enter into it. It's stone dead."
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090926/45e00103/attachment.htm>

From daniel at stutzbachenterprises.com  Sun Sep 27 01:27:34 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sat, 26 Sep 2009 18:27:34 -0500
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <538a660a0909261457n1a956bc0i1524e58c4e48be4e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com>
	<4AB15FD3.5040804@canterbury.ac.nz>
	<eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>
	<20090917004138.2177C5AC0D6@longblack.object-craft.com.au>
	<4ABE567D.6080808@v.loewis.de>
	<538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com>
	<eae285400909261314t6a00ecf7p2194ef7bdb0bb822@mail.gmail.com>
	<538a660a0909261457n1a956bc0i1524e58c4e48be4e@mail.gmail.com>
Message-ID: <eae285400909261627u6b67e94o6e5962fc8713c763@mail.gmail.com>

On Sat, Sep 26, 2009 at 4:57 PM, DrKJam <drkjam at gmail.com> wrote:

> 2009/9/26 Daniel Stutzbach <daniel at stutzbachenterprises.com>
>
>> On Sat, Sep 26, 2009 at 2:07 PM, DrKJam <drkjam at gmail.com> wrote:
>>
>>> The current version of the PEP and reference implementation do not
>>> mention or deal with IPv4 classful addressing (A, B, C, D and E). It would
>>> be good to know if any of this (admittedly older yet no less important)
>>> functionality is going to be supported. If the library is to concentrate
>>> solely on classless addressing (i.e. CIDR) please can this be stated in
>>> future revisions of the PEP.
>>>
>>
>> Classful addressing was deprecated more than 15 years ago!
>>
>> Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such
>> scenarios are no longer operationally relevant."
>
>
> Interesting as evidence of classful IPv4 behaviour seems fairly pervasive
> in current IP stacks and software that supports IPv4 addressing (e.g.
> PostgreSQL inet and cidr data types).
>

PostgreSQL's inet type stores a host address and optionally a network mask.
It's cidr type stores a network address and mask.  Neither of them are
classful.


> Here's an excerpt from the 'ifconfig' man page (taken from an Ubuntu 9.04
> install) :-
> netmask addr
>
> Set the IP network mask for this interface. This value defaults to the
> usual class A, B or C network mask (as derived from the interface IP
> address). but it can be set to any value.
>

Since the network mask can be set to any legal value, that means it's stored
as CIDR.  It falls back on the classful network mask as a default.
Undoubtedly that was useful when ifconfig was updated for CIDR.  There were
still many classful networks and it needed to retain backward
compatibility.  After all, ifconfig is around 25 years old, and CIDR is only
16.

Today, all IP networks are classless (CIDR), and a system administrator
would have to do extra work to figure out if the default value happened to
be just right for the network they're setting up.


>  The point being that you can't always assume /32 implicitly for all use
> cases.
>

Certainly, which is why the user can explicitly set the netmask.


> Here is how IP addresses without an explicit prefix or netmask are
> currently handled :-
>
> >>> import ipaddr
> >>> ipaddr.IPv4Network('10.0.0.1')
> IPv4Network('10.0.0.1/32')
>
> It may not be something we want to support (you could always force the end
> user to specify a prefix or netmask explicitly). This is fair enough, but
> let's indicate that it's a considered choice. Somewhere in the PEP text is
> might be a good place for treatment of this topic.
>

+1 on forcing the user to specify a netmask explicitly and documenting it.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090926/4d5d5d38/attachment.htm>

From peter at hda3.com  Sun Sep 27 02:28:49 2009
From: peter at hda3.com (Peter Moody)
Date: Sat, 26 Sep 2009 17:28:49 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <eae285400909261627u6b67e94o6e5962fc8713c763@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz>
	<eae285400909161531q777895e5x4b5c524b50566b93@mail.gmail.com>
	<20090917004138.2177C5AC0D6@longblack.object-craft.com.au>
	<4ABE567D.6080808@v.loewis.de>
	<538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com>
	<eae285400909261314t6a00ecf7p2194ef7bdb0bb822@mail.gmail.com>
	<538a660a0909261457n1a956bc0i1524e58c4e48be4e@mail.gmail.com>
	<eae285400909261627u6b67e94o6e5962fc8713c763@mail.gmail.com>
Message-ID: <8517e9350909261728q2f15ccd6g37103277ebba4b31@mail.gmail.com>

On Sat, Sep 26, 2009 at 4:27 PM, Daniel Stutzbach
<daniel at stutzbachenterprises.com> wrote:
> On Sat, Sep 26, 2009 at 4:57 PM, DrKJam <drkjam at gmail.com> wrote:
>>
>> 2009/9/26 Daniel Stutzbach <daniel at stutzbachenterprises.com>
>>>
>>> On Sat, Sep 26, 2009 at 2:07 PM, DrKJam <drkjam at gmail.com> wrote:
>>>>
>>>> The current version of the PEP and reference implementation do not
>>>> mention or deal with IPv4 classful addressing (A, B, C, D and E). It would
>>>> be good to know if any of this (admittedly older yet no less important)
>>>> functionality is going to be supported. If the library is to concentrate
>>>> solely on classless addressing (i.e. CIDR) please can this be stated in
>>>> future revisions of the PEP.

I'll call this out in the PEP if necessary.  As mentioned elsewhere,
classful addressing is so long deprecated that I would expect it to
cause confusion if it was assumed.

>>> Classful addressing was deprecated more than 15 years ago!
>>>
>>> Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such
>>> scenarios are no longer operationally relevant."
>>
>> Interesting as evidence of classful IPv4 behaviour seems fairly pervasive
>> in current IP stacks and software that supports IPv4 addressing (e.g.
>> PostgreSQL inet and cidr data types).
>
> PostgreSQL's inet type stores a host address and optionally a network mask.
> It's cidr type stores a network address and mask.? Neither of them are
> classful.
>
>>
>> Here's an excerpt from the 'ifconfig' man page (taken from an Ubuntu 9.04
>> install) :-
>> netmask addr
>>
>> Set the IP network mask for this interface. This value defaults to the
>> usual class A, B or C network mask (as derived from the interface IP
>> address). but it can be set to any value.
>
> Since the network mask can be set to any legal value, that means it's stored
> as CIDR.? It falls back on the classful network mask as a default.
> Undoubtedly that was useful when ifconfig was updated for CIDR.? There were
> still many classful networks and it needed to retain backward
> compatibility.? After all, ifconfig is around 25 years old, and CIDR is only
> 16.
>
> Today, all IP networks are classless (CIDR), and a system administrator
> would have to do extra work to figure out if the default value happened to
> be just right for the network they're setting up.
>
>>
>> ?The point being that you can't always assume /32 implicitly for all use
>> cases.
>
> Certainly, which is why the user can explicitly set the netmask.
>
>>
>> Here is how IP addresses without an explicit prefix or netmask are
>> currently handled :-
>>
>> >>> import ipaddr
>> >>> ipaddr.IPv4Network('10.0.0.1')
>> IPv4Network('10.0.0.1/32')
>>
>> It may not be something we want to support (you could always force the end
>> user to specify a prefix or netmask explicitly). This is fair enough, but
>> let's indicate that it's a considered choice. Somewhere in the PEP text is
>> might be a good place for treatment of this topic.
>
> +1 on forcing the user to specify a netmask explicitly and documenting it.

-(1/0) forcing a user to include a prefixlen when creating a network
object doesn't actually serve any useful purpose. when dealing with
network objects, it's very useful (and common) to assume a /32 prefix
length when none is supplied.

Cheers,
/peter

> --
> Daniel Stutzbach, Ph.D.
> President, Stutzbach Enterprises, LLC
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>
>

From steve at pearwood.info  Sun Sep 27 03:08:32 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 27 Sep 2009 11:08:32 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4ABE530D.8090609@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<h8sd2n$n1g$1@ger.gmane.org> <4ABE530D.8090609@v.loewis.de>
Message-ID: <200909271108.32456.steve@pearwood.info>

On Sun, 27 Sep 2009 03:44:45 am Martin v. L?wis wrote:
> >  - Masks are also 32- (128-) bit integers, which happen to have the
> >    property that their leftmost N bits are all zero and the rest
> > are all one.
>
> As a side note, I would be in favor of dropping the concept of a mask
> from the library, and only support a prefix length.

-1

> IPv6 doesn't support masks at all, and even for IPv4, I think there
> are conventions (if not RFCs) against using them in a way that does
> not correspond to a prefix length.

Then the module should only support netmasks of the form 
(say) '255.255.255.224' (equivalent to "/27"), and reject those 
like "255.3.255.255". It currently accepts them.

Many applications still display netmasks in dot-quad form, and I would 
be terribly annoyed if I had to count the bits myself before passing it 
to IPv4Address.



-- 
Steven D'Aprano

From steve at pearwood.info  Sun Sep 27 04:05:18 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 27 Sep 2009 12:05:18 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>
Message-ID: <200909271205.18488.steve@pearwood.info>

On Sat, 26 Sep 2009 11:23:14 pm Barry Scott wrote:

> I've seen user interfaces accept 192.168.1.1/24 as a short cut
> to set the ipaddr and netmask on an interface.
>
> For that use being able to parse that string into an IP Address and
> a Net Mask is what they want.

I think you're at least the second person who has requested this 
functionality, or something similar.


> If this is a common idiom having a utility function that returned
> an IPv4Address('192.168.1.1') and IPv4Network('192.168.0.0')
> would be useful.
>
> If someone really thinks that '192.168.1.1/16' is a network any
> good UI should detect it is invalid report that to the user.

Currently the ipaddr module accepts that without complaint:

>>> ipaddr.IPv4Network('192.168.1.1/16')
IPv4Network('192.168.1.1/16')

The current behaviour is confusing to me. For example:

>>> netw1 = ipaddr.IPv4Network('192.168.1.1/24')
>>> netw2 = ipaddr.IPv4Network('192.168.1.0/24')
>>> netw1 == netw2
False
>>> list(netw1) == list(netw2)
True

Two networks, containing exactly the same range of addresses, but they 
don't compare equal. I'm not convinced that netw1 should even be 
allowed, but if it is, surely it should be turned into canonical form 
netw2? E.g. I would expect this:

>>> ipaddr.IPv4Network('192.168.1.1/24')
IPv4Network('192.168.1.0/24')

but that's not what it does.


-- 
Steven D'Aprano

From nad at acm.org  Sun Sep 27 04:12:44 2009
From: nad at acm.org (Ned Deily)
Date: Sat, 26 Sep 2009 19:12:44 -0700
Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on
	trunk
References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>
Message-ID: <nad-A69561.19124426092009@news.gmane.org>

In article <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A at barrys-emacs.org>,
 Barry Scott <barry at barrys-emacs.org> wrote:
> I'm working with http://svn.python.org/projects/python/trunk on Mac OS  
> X 10.6.1
> using Apples xcode gcc 4.2.1.
> 
> When I run the following commands:
> 
> 	./configure --enable-framework --with-universal-archs=32-bit | tee  
> build.config.log
> 	make clean all | tee build.make.log
> 
> I end up with a x86_64 Python image.
> 
> No matter what I use for archs its always the same.
> 
> I would expect to see -arch arg to GCC but it is not there.
> 
> 	export CFLAG="-arch i386"
> 
> did not work either.
> 
> Am I doing something wrong or is this broken on trunk?

You need to add the enable-universalsdk parameter to configure:

  ... --enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk

Be aware, though, that universal build support on 10.6 is a bit of a 
work in progress as there are still some interesting unexplained 
universal build issues when building on Snow Leopard (see, for instance, 
the comments in http://bugs.python.org/issue6957).  At the moment, the 
focus is on getting 2.6.3 out the door and the standard installer for 
that will be built on 10.5.

-- 
 Ned Deily,
 nad at acm.org


From peter at hda3.com  Sun Sep 27 04:22:08 2009
From: peter at hda3.com (Peter Moody)
Date: Sat, 26 Sep 2009 19:22:08 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909271205.18488.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>
	<200909271205.18488.steve@pearwood.info>
Message-ID: <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>

I again invite interested parties to continue this discussion on
ipaddr-py-dev at googlegroups.com.  we're pushing 250 messages on PEP
3144 at this point; well beyond what most folks would call a "long
open-ended discussion".

anyway:

> The current behaviour is confusing to me. For example:
>
>>>> netw1 = ipaddr.IPv4Network('192.168.1.1/24')
>>>> netw2 = ipaddr.IPv4Network('192.168.1.0/24')
>>>> netw1 == netw2
> False
>>>> list(netw1) == list(netw2)
> True

I think you're intentionally confusing yourself here.  192.168.1.1
clearly doesn't equal 192.168.1.0, so I don't know why you'd expect
192.168.1.1/24 to equal 192.168.1.0/24.

> Two networks, containing exactly the same range of addresses, but they
> don't compare equal. I'm not convinced that netw1 should even be
> allowed, but if it is, surely it should be turned into canonical form
> netw2? E.g. I would expect this:
>
>>>> ipaddr.IPv4Network('192.168.1.1/24')
> IPv4Network('192.168.1.0/24')
>
> but that's not what it does.

now you're confusing me as before you've asked for this to either:

 1. Raise an exception
 2. Return two objects.

your unconfusing (to you) example does neither one of those.


> --
> Steven D'Aprano
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From guido at python.org  Sun Sep 27 04:25:32 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 26 Sep 2009 19:25:32 -0700
Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the
	Python Standard Library
In-Reply-To: <4ABE5B24.1090003@v.loewis.de>
References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com> 
	<h6gjku$t2n$1@ger.gmane.org> <4A8C0267.5020408@wildenhain.de> 
	<8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com> 
	<Pine.LNX.4.64.0908191148530.30683@kimball.webabinitio.net> 
	<8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com> 
	<4A8C4264.2040708@trueblade.com>
	<d9047f780908191128p12107a1exc20dabc34ddff5de@mail.gmail.com> 
	<20090916090849.GH12810@ruber.office.udmvt.ru>
	<4ABE5B24.1090003@v.loewis.de>
Message-ID: <ca471dc20909261925o2b24d36fjeda61b89be58d3e0@mail.gmail.com>

I realize I'm late to this party, but this is just a naming issue, right?

For any network, there are two special addresses, one with the last
bits all zeros, one with the last bits all ones. We can call them A
and B, or network and broadcast, or zeros and ones, or whatever we
care. But their definitions are two well-defined functions for all
networks (assuming a network is defined as an IP address and a number
of bits), even if in an extreme case the functions return the same
value.

What is actually configured on a particular host to be the broadcast
address is a separate issue, even if *by convention* in most cases it
is given by one of the above functions -- the network object doesn't
care, the configuration object is something else (and outside the
scope of this PEP).

IMO "real life examples" don't matter for the definitions of the
functions, and I would personally be happy to name them network and
broadcast, since I know their definitions and their typical uses and
these match pretty closely. The expectation should be clearly set that
these are pure functions though and do not imply knowledge of the
configuration of any given host.

--Guido

On Sat, Sep 26, 2009 at 11:19 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>>> I think using .network and .broadcast are pretty well understood to be the
>>>> [0] and [-1] of the network address block. I don't think we want to start
>>>> creating new terms or access patterns here.
>>>>
>>>> +1 on leaving .network and .broadcast as-is (including returning a
>>>> IPvXAddress object).
>>>>
>>> -1. ?I think 'network.number' or 'network.zero' is a lot clearer than
>>> 'network.network'. ?Maybe '.broadcast' would be okay, as long as it *can* be
>>> adjusted for those unusual, or maybe even only hypothetical, networks where
>>> it is not the [-1].
>> Real life example: network with a /31 mask.
>> There are only two hosts: 0 and 1
>> first host configures the other's host as broadcast address and vice versa.
>> NOTE - broadcasts are different here!
>
> This is RFC 3021. IIUC, it does not support directed broadcast; only
> link-local broadcast can be used on that link.
>
> So ISTM that .broadcast should raise an exception on a /31 network. Any
> installation that configures the partner as the broadcast address is
> broken (somebody correct me if I'm wrong).
>
>> Another real life examples include /32 networks on PPP. Just a point-to-point.
>> No need for broadcasts and networks, a host just have one IP address and
>> send all traffic via point-to-point link, no addressing is required there.
>> This is a working dialup configuration, it works for me, it works for you, probably.
>> It is not weird, it is common, it is used for PPPoE, for ADSL, for dialup.
>
> So where is that defined?
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Sun Sep 27 05:50:08 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 26 Sep 2009 20:50:08 -0700
Subject: [Python-Dev] LAST CHANCE: PyCon 2010: Call for Proposals
Message-ID: <20090927035008.GA28270@panix.com>

Just four more days to propose a presentation!

Call for proposals -- PyCon 2010 -- <http://us.pycon.org/2010/>
===============================================================

Due date: October 1st, 2009

Want to showcase your skills as a Python Hacker? Want to have
hundreds of people see your talk on the subject of your choice? Have some
hot button issue you think the community needs to address, or have some
package, code or project you simply love talking about? Want to launch
your master plan to take over the world with python?

PyCon is your platform for getting the word out and teaching something
new to hundreds of people, face to face.

Previous PyCon conferences have had a broad range of presentations,
from reports on academic and commercial projects, tutorials on a broad
range of subjects and case studies. All conference speakers are volunteers
and come from a myriad of backgrounds. Some are new speakers, some
are old speakers. Everyone is welcome so bring your passion and your
code! We're looking to you to help us top the previous years of success
PyCon has had.

PyCon 2010 is looking for proposals to fill the formal presentation tracks.
The PyCon conference days will be February 19-22, 2010 in Atlanta,
Georgia, preceded by the tutorial days (February 17-18), and followed
by four days of development sprints (February 22-25).

Online proposal submission is open now! Proposals  will be accepted
through October 1st, with acceptance notifications coming out on
November 15th. For the detailed call for proposals, please see:

 <http://us.pycon.org/2010/conference/proposals/>

For videos of talks from previous years - check out:

<http://pycon.blip.tv>

We look forward to seeing you in Atlanta!
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Look, it's your affair if you want to play with five people, but don't
go calling it doubles."  --John Cleese anticipates Usenet

From ncoghlan at gmail.com  Sun Sep 27 07:38:37 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 27 Sep 2009 15:38:37 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>	<200909271205.18488.steve@pearwood.info>
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
Message-ID: <4ABEFA5D.40506@gmail.com>

Peter Moody wrote:
> I again invite interested parties to continue this discussion on
> ipaddr-py-dev at googlegroups.com.  we're pushing 250 messages on PEP
> 3144 at this point; well beyond what most folks would call a "long
> open-ended discussion".
> 
> anyway:
> 
>> The current behaviour is confusing to me. For example:
>>
>>>>> netw1 = ipaddr.IPv4Network('192.168.1.1/24')
>>>>> netw2 = ipaddr.IPv4Network('192.168.1.0/24')
>>>>> netw1 == netw2
>> False
>>>>> list(netw1) == list(netw2)
>> True
> 
> I think you're intentionally confusing yourself here.  192.168.1.1
> clearly doesn't equal 192.168.1.0, so I don't know why you'd expect
> 192.168.1.1/24 to equal 192.168.1.0/24.

No, this is exactly the "dual purpose" nature of IPNetwork that people
have been complaining about: 192.168.1.1 & 255.255.255.0 == 192.168.1.0
& 255.255.255.0, so the two objects describe the exact same network, but
the network objects don't currently respect their own netmask, instead
remembering which particular host happened to be used to derive the
network ID.

"Arbitrary network host with associated netmask" and "IP Network" are
different concepts, but the library currently uses a single class
interface (IPNetwork) for both of them.

What we're asking for is for the "ip" attribute to be removed from IP
Network, with the constructor either forgetting the host IP that was
used to derive the network address, or else only accepting the correct
network address in the constructor and providing a separate helper
function or class method to create an IPNetwork object from an arbitrary
hostname.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From peter at hda3.com  Sun Sep 27 07:54:21 2009
From: peter at hda3.com (Peter Moody)
Date: Sat, 26 Sep 2009 22:54:21 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4ABEFA5D.40506@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>
	<200909271205.18488.steve@pearwood.info>
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
Message-ID: <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>

On Sat, Sep 26, 2009 at 10:38 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Peter Moody wrote:
>> I again invite interested parties to continue this discussion on
>> ipaddr-py-dev at googlegroups.com. ?we're pushing 250 messages on PEP
>> 3144 at this point; well beyond what most folks would call a "long
>> open-ended discussion".
>>
>> anyway:
>>
>>> The current behaviour is confusing to me. For example:
>>>
>>>>>> netw1 = ipaddr.IPv4Network('192.168.1.1/24')
>>>>>> netw2 = ipaddr.IPv4Network('192.168.1.0/24')
>>>>>> netw1 == netw2
>>> False
>>>>>> list(netw1) == list(netw2)
>>> True
>>
>> I think you're intentionally confusing yourself here. ?192.168.1.1
>> clearly doesn't equal 192.168.1.0, so I don't know why you'd expect
>> 192.168.1.1/24 to equal 192.168.1.0/24.
>
> No, this is exactly the "dual purpose" nature of IPNetwork that people
> have been complaining about: 192.168.1.1 & 255.255.255.0 == 192.168.1.0
> & 255.255.255.0, so the two objects describe the exact same network, but
> the network objects don't currently respect their own netmask, instead
> remembering which particular host happened to be used to derive the
> network ID.
>
> "Arbitrary network host with associated netmask" and "IP Network" are
> different concepts, but the library currently uses a single class
> interface (IPNetwork) for both of them.
>
> What we're asking for is for the "ip" attribute to be removed from IP
> Network, with the constructor either forgetting the host IP that was
> used to derive the network address, or else only accepting the correct
> network address in the constructor and providing a separate helper
> function or class method to create an IPNetwork object from an arbitrary
> hostname.

what you want is strict=True, just checked in. I've been meaning to
send an update to the PEP explaining why this choice was made. I'm
hoping this will ameliorate your confusion.

Cheers,
/peter

> Cheers,
> Nick.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
>

From ncoghlan at gmail.com  Sun Sep 27 08:08:30 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 27 Sep 2009 16:08:30 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>	
	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>	
	<200909271205.18488.steve@pearwood.info>	
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>	
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
Message-ID: <4ABF015E.5060002@gmail.com>

Peter Moody wrote:
> what you want is strict=True, just checked in. I've been meaning to
> send an update to the PEP explaining why this choice was made. I'm
> hoping this will ameliorate your confusion.

If the ip property remains and equality is still broken as Steven
describes, then no, it won't really alleviate my concerns.

I've also previously registered my objections to using a boolean flag
for this instead of a separate method.

However, I'll wait and see what the PEP update says before commenting
further.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Sun Sep 27 08:56:40 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 27 Sep 2009 08:56:40 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909271108.32456.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<h8sd2n$n1g$1@ger.gmane.org>
	<4ABE530D.8090609@v.loewis.de>
	<200909271108.32456.steve@pearwood.info>
Message-ID: <4ABF0CA8.2030408@v.loewis.de>

>> As a side note, I would be in favor of dropping the concept of a mask
>> from the library, and only support a prefix length.
> 
> -1
> 
>> IPv6 doesn't support masks at all, and even for IPv4, I think there
>> are conventions (if not RFCs) against using them in a way that does
>> not correspond to a prefix length.
> 
> Then the module should only support netmasks of the form 
> (say) '255.255.255.224' (equivalent to "/27"), and reject those 
> like "255.3.255.255". It currently accepts them.
> 
> Many applications still display netmasks in dot-quad form, and I would 
> be terribly annoyed if I had to count the bits myself before passing it 
> to IPv4Address.

I wouldn't ask for that: it should certainly be possible to supply
masks. However, I would want to reject masks that don't correspond to
a prefix, and have only the prefix length in the internal representation.

Regards,
Martin

From martin at v.loewis.de  Sun Sep 27 09:08:39 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 27 Sep 2009 09:08:39 +0200
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <h8tsb9$bkq$1@ger.gmane.org>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>
	<h8tsb9$bkq$1@ger.gmane.org>
Message-ID: <4ABF0F77.8050209@v.loewis.de>

> One thing I'd like to see in the list are real names of committers,
> not tracker names.  Of course, people looking for people to assign a bug
> to should not have to search for the tracker name, so I'd like to make
> another request (that Brett already made when we switched trackers):
> 
>   Could we *please* have tracker names that match the committer names?

I'm opposed. I never type my committer name - I don't even know what it
is (martin.v.loewis, martin.vonloewis, martin.von.loewis, something
else?). In the tracker, I have to type my account name, so I need to
remember. I have been using "loewis" as my account name in the net for
nearly two decades now, so I have little interest in starting to have
different account names, now.

> (This doesn't even need to be done by the individual users, I would
> volunteer to rename all committer accounts and notify them by email that
> their name changed.  This will also be coordinated with the new names
> used for Mercurial commits, if a change will be made.)

IIUC, there will be relatively little control over Mercurial committer
names. Every user will have to configure ~/.hgrc, which should list a
single name/email address pair, to be used for all Mercurial
repositories. So I put "Martin v. L?wis <martin at v.loewis.de>" into
.hgrc, which is my name and my email address. It's not the same as the
tracker name or the subversion committer name; it can't possibly be
(since it comes from an entirely different namespace).

Regards,
Martin

From barry at barrys-emacs.org  Sat Sep 26 15:34:27 2009
From: barry at barrys-emacs.org (Barry Scott)
Date: Sat, 26 Sep 2009 14:34:27 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
Message-ID: <A7970A2A-4463-471B-AE28-321581DD778D@barrys-emacs.org>


On 14 Sep 2009, at 17:44, Peter Moody wrote:

> Folks, Guido,
>
> I believe PEP 3144 is ready for your review.  When you get a chance,
> can you take a look/make a pronouncement?
>


 >>> x=ipaddr.IPv4Network( '192.168.1.1/240.255.0.0' )
 >>> x
IPv4Network('192.168.1.1/16')

You can decide on which bug this shows.

1) parse must raise exception on bad input
2) repr does not show input

Barry


From dickinsm at gmail.com  Sun Sep 27 09:21:16 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 27 Sep 2009 08:21:16 +0100
Subject: [Python-Dev] Python build question (fixing pymath.c).
Message-ID: <5c6f2a5d0909270021n20438083yaf3f854e8dd96c6d@mail.gmail.com>

Hello all,

I'm looking for advice on how to clean up an ugliness (one
that I'm at least partly responsible for) in the Python build setup.

Here's the problem:  some of the exported functions (e.g. atanh,
log1p) in the Python/pymath.c file aren't needed by the Python core
at all; they're exported solely for use in the math and cmath modules.
Even worse, these exported functions have no '_Py' or 'Py' prefix.

Since I'm currently working on adding some oft-requested
functions to the math module (gamma, lgamma, erf, ...) it seemed
like a good time to clean this up.

So I've now got a file Modules/math_support.c that contains
some functions needed by both mathmodule.c and
cmathmodule.c, as well as a couple of functions only
currently needed by the math module.  How should I incorporate
this file into the build?

One obvious solution seems to be to build an extra math_support.so
file that's then used by both mathmodule.so and cmathmodule.so.
Is this a sensible solution?  Are there better ways?

A complication to bear in mind is that some users may want to
alter Modules/Setup.dist so that either the math module and/or
the cmath module is included in the core Python executable.

Cluelessly yours,

Mark

From peter at hda3.com  Sun Sep 27 09:34:42 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 00:34:42 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <A7970A2A-4463-471B-AE28-321581DD778D@barrys-emacs.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<A7970A2A-4463-471B-AE28-321581DD778D@barrys-emacs.org>
Message-ID: <8517e9350909270034k76f88da3pe1bdf52c576001bb@mail.gmail.com>

On Sat, Sep 26, 2009 at 6:34 AM, Barry Scott <barry at barrys-emacs.org> wrote:
>
> On 14 Sep 2009, at 17:44, Peter Moody wrote:
>
>> Folks, Guido,
>>
>> I believe PEP 3144 is ready for your review. ?When you get a chance,
>> can you take a look/make a pronouncement?
>>
>
>
>>>> x=ipaddr.IPv4Network( '192.168.1.1/240.255.0.0' )
>>>> x
> IPv4Network('192.168.1.1/16')
>
> You can decide on which bug this shows.
>
> 1) parse must raise exception on bad input
> 2) repr does not show input
>
> Barry

thank you, Barry, for your continued interest. Since it may have been
lost in the 200+ messages on this topic, issues with the current
implementation can be filed at:

  http://code.google.com/p/ipaddr-py/issues/entry

In case you missed it, I'm trying to move discussion of the PEP over
to ipaddr-py-dev at googlegroups.com b/c pep-0001 is fairly explicit
about trying to avoid long open-ended discussions. I'm pretty sure bug
reports like this at the end of 200+ messages qualifies as long and
open-ended.

Cheers,
/peter

>

From ncoghlan at gmail.com  Sun Sep 27 09:39:38 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 27 Sep 2009 17:39:38 +1000
Subject: [Python-Dev] Misc/maintainers.rst
In-Reply-To: <4ABF0F77.8050209@v.loewis.de>
References: <Pine.LNX.4.64.0909171231180.17020@kimball.webabinitio.net>	<h8tsb9$bkq$1@ger.gmane.org>
	<4ABF0F77.8050209@v.loewis.de>
Message-ID: <4ABF16BA.4020306@gmail.com>

Martin v. L?wis wrote:
>> One thing I'd like to see in the list are real names of committers,
>> not tracker names.  Of course, people looking for people to assign a bug
>> to should not have to search for the tracker name, so I'd like to make
>> another request (that Brett already made when we switched trackers):
>>
>>   Could we *please* have tracker names that match the committer names?
> 
> I'm opposed. I never type my committer name - I don't even know what it
> is (martin.v.loewis, martin.vonloewis, martin.von.loewis, something
> else?). In the tracker, I have to type my account name, so I need to
> remember. I have been using "loewis" as my account name in the net for
> nearly two decades now, so I have little interest in starting to have
> different account names, now.

Same - my tracker is my normal online account name (ncoghlan), while my
committer name is my full name (nick.coghlan).

Having roundup store an extra field with committer names (and letting
people see both names in the user lists on the tracker) seems like a
better option than asking us to change our account names.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Sun Sep 27 09:41:28 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 27 Sep 2009 09:41:28 +0200
Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the
 Python Standard Library
In-Reply-To: <ca471dc20909261925o2b24d36fjeda61b89be58d3e0@mail.gmail.com>
References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com>
	<h6gjku$t2n$1@ger.gmane.org> <4A8C0267.5020408@wildenhain.de>
	<8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com>
	<Pine.LNX.4.64.0908191148530.30683@kimball.webabinitio.net>
	<8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com>
	<4A8C4264.2040708@trueblade.com>
	<d9047f780908191128p12107a1exc20dabc34ddff5de@mail.gmail.com>
	<20090916090849.GH12810@ruber.office.udmvt.ru>
	<4ABE5B24.1090003@v.loewis.de>
	<ca471dc20909261925o2b24d36fjeda61b89be58d3e0@mail.gmail.com>
Message-ID: <4ABF1728.9@v.loewis.de>

Guido van Rossum wrote:
> I realize I'm late to this party, but this is just a naming issue, right?

Not really (AFAIU).

> For any network, there are two special addresses, one with the last
> bits all zeros, one with the last bits all ones. We can call them A
> and B, or network and broadcast, or zeros and ones, or whatever we
> care. But their definitions are two well-defined functions for all
> networks (assuming a network is defined as an IP address and a number
> of bits), even if in an extreme case the functions return the same
> value.

For the broadcast address, it's different, since you might also use
it in programming (i.e. when sending broadcasts).

Now, it seems that people will typically use undirected broadcasts
(INADDR_BROADCAST). However, I assume there are also applications
for directed broadcasts; in this case, it would matter to know the
broadcast address of a network.

> What is actually configured on a particular host to be the broadcast
> address is a separate issue, even if *by convention* in most cases it
> is given by one of the above functions -- the network object doesn't
> care, the configuration object is something else (and outside the
> scope of this PEP).

It's a little bit stronger than convention; there are RFCs saying
that the all-ones (-1) address has to be the broadcast address.

This is like saying that it is out of scope of the socket protocol
whether TCP is the IP protocol 6, and that individual hosts may do
it differently - yes, they may, but then everything stops working.

> IMO "real life examples" don't matter for the definitions of the
> functions

That I agree with.

> and I would personally be happy to name them network and
> broadcast, since I know their definitions and their typical uses and
> these match pretty closely. The expectation should be clearly set that
> these are pure functions though and do not imply knowledge of the
> configuration of any given host.

That I also agree with.

However, there is this issue of /31 networks, where an 8-year-old
RFC specifies how to do broadcasts on these. I think RFCs should
be considered wherever applicable.

Regards,
Martin


From brett at python.org  Sun Sep 27 09:48:13 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 27 Sep 2009 00:48:13 -0700
Subject: [Python-Dev] Python build question (fixing pymath.c).
In-Reply-To: <5c6f2a5d0909270021n20438083yaf3f854e8dd96c6d@mail.gmail.com>
References: <5c6f2a5d0909270021n20438083yaf3f854e8dd96c6d@mail.gmail.com>
Message-ID: <bbaeab100909270048q5cc8f389we9319f1001c624b6@mail.gmail.com>

On Sun, Sep 27, 2009 at 00:21, Mark Dickinson <dickinsm at gmail.com> wrote:
> Hello all,
>
> I'm looking for advice on how to clean up an ugliness (one
> that I'm at least partly responsible for) in the Python build setup.
>
> Here's the problem: ?some of the exported functions (e.g. atanh,
> log1p) in the Python/pymath.c file aren't needed by the Python core
> at all; they're exported solely for use in the math and cmath modules.
> Even worse, these exported functions have no '_Py' or 'Py' prefix.
>
> Since I'm currently working on adding some oft-requested
> functions to the math module (gamma, lgamma, erf, ...) it seemed
> like a good time to clean this up.
>
> So I've now got a file Modules/math_support.c that contains
> some functions needed by both mathmodule.c and
> cmathmodule.c, as well as a couple of functions only
> currently needed by the math module. ?How should I incorporate
> this file into the build?
>
> One obvious solution seems to be to build an extra math_support.so
> file that's then used by both mathmodule.so and cmathmodule.so.
> Is this a sensible solution? ?Are there better ways?
>

Are you planning on exposing any of this outside of those two modules?
If not then I would change the name to _math.so

> A complication to bear in mind is that some users may want to
> alter Modules/Setup.dist so that either the math module and/or
> the cmath module is included in the core Python executable.

If you are mucking with Modules.Setup.dist you better know how to
figure out that those two modules depend on another .so.

-Brett

From peter at hda3.com  Sun Sep 27 09:49:41 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 00:49:41 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4ABF015E.5060002@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>
	<200909271205.18488.steve@pearwood.info>
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
Message-ID: <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>

On Sat, Sep 26, 2009 at 11:08 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Peter Moody wrote:
>> what you want is strict=True, just checked in. I've been meaning to
>> send an update to the PEP explaining why this choice was made. I'm
>> hoping this will ameliorate your confusion.
>
> If the ip property remains and equality is still broken as Steven
> describes, then no, it won't really alleviate my concerns.
>
> I've also previously registered my objections to using a boolean flag
> for this instead of a separate method.
>
> However, I'll wait and see what the PEP update says before commenting
> further.

I've mailed the PEP patch in, hopefully it will be checked in and
updated shortly.

To be explicit though, unless I'm drastically misreading what you're
asking for, I consider the design you're proposing to be broken from a
usability perspective (striving for academic/pedantic correctness over
accepted practice). I have no intention of adding any sort of
AddressWithNetmask classes or making other fundamental design changes
you've suggested. If what the community requires is the library you've
described, then ipaddr is not that library.

Cheers,
/peter

> Cheers,
> Nick.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
>

From chambon.pascal at gmail.com  Sun Sep 27 09:58:51 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Sun, 27 Sep 2009 09:58:51 +0200
Subject: [Python-Dev] Fuzziness in io module specs -	PEP update
 proposition V2
In-Reply-To: <4AB67894.5020903@mrabarnett.plus.com>
References: <4AB3DCC9.5050209@gmail.com>	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<200909201007.11923.steve@pearwood.info>	<4AB58159.3040303@canterbury.ac.nz>	<4AB5D6CB.6050604@gmail.com>	<4AB5E8DD.6050204@canterbury.ac.nz>	<4AB5FA81.5060408@gmail.com>	<eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com>	<4AB6590E.2070800@gmail.com>
	<4AB67894.5020903@mrabarnett.plus.com>
Message-ID: <4ABF1B3B.4080106@gmail.com>

Hello

Below is a corrected version of the PEP update, adding the start/end 
indexes proposition and fixing functions signatures. Does anyone 
disagree with these specifications ? Or can we consider it as a target 
for the next versions of the io module ?
I would have no problem to implement this behaviour in my own pure 
python FileIO system, however if someone is willing to patch the _fileio 
implementation, it'd save a lot of time - I most probably won't have the 
means to setup a C compilation environment under windows and linux, and 
properly update/test this, before January (when I get freelance...).

I launch another thread on other to-be-discussed IO points B-)

Regards,
Pascal

================ PEP UPDATE for new I/O system - v2 ===========

**Truncate and file pointer semantics**

Rationale :

The current implementation of truncate() always move the file pointer to 
the new end of file.

This behaviour is interesting for compatibility, if the file has been 
reduced and the file pointer is now past its end, since some platforms 
might require 0 <= filepointer <= filesize.

However, there are several arguments against this semantic:

    * Most common standards (posix, win32?) allow the file pointer to be
      past the end of file, and define the behaviour of other stream
      methods in this case
    * In many cases, moving the filepointer when truncating has no
      reasons to happen (if we?re extending the file, or reducing it
      without going beneath the file pointer)
    * Making 0 <= filepointer <= filesize a global rule of the python IO
      module doesn?t seems possible, since it would require
      modifications of the semantic of other methods (eg. seek() should
      raise exceptions or silently disobey when asked to move the
      filepointer past the end of file), and lead to incoherent
      situations when concurrently accessing files without locking (what
      if another process truncates to 0 bytes the file you?re writing ?)

So here is the proposed semantic, which matches established conventions:

*IOBase.truncate(n: int = None) -> int*

Resizes the file to the size specified by the positive integer n, or by 
the current filepointer position if n is None.

The file must be opened with write permissions.

If the file was previously larger than size, the extra data is discarded.
If the file was previously shorter than size, its size is increased, and
the extended area appears as if it were zero-filled.

In any case, the file pointer is left unchanged, and may point beyond
the end of file.

Note: trying to read past the end of file returns an empty string, and
trying to write past the end of file extends it by zero-ing the gap. On
rare platforms which don't support file pointers to be beyond the end of
file, all these behaviours shall be faked thanks to internal storage of
the "wanted" file pointer position (silently extending the file, if
necessary, when a write operation occurs).

 

*Propositions of doc update*

*RawIOBase*.read(n: int) -> bytes

Read up to n bytes from the object and return them. Fewer than n bytes
may be returned if the operating system call returns fewer than n bytes.
If 0 bytes are returned, and n was not 0, this indicates end of file. If
the object is in non-blocking mode and no bytes are available, the call
returns None.


*RawIOBase*.readinto(b: bytearray, [start: int = None], [end: int = 
None]) -> int

start and end are used as slice indexes, so that the bytearray taken 
into account is actually range = b[start:end] (or b[start:], b[:end] or 
b[:], depending on the arguments which are not None).

Read up to len(range) bytes from the object and store them in b, returning
the number of bytes read. Like .read, fewer than len(range) bytes may be
read, and 0 indicates end of file if len(range) is not 0.
None is returned if a non-blocking object has no bytes available. The 
length of b is never changed.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090927/49657bb8/attachment-0001.htm>

From dickinsm at gmail.com  Sun Sep 27 10:00:02 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 27 Sep 2009 09:00:02 +0100
Subject: [Python-Dev] Python build question (fixing pymath.c).
In-Reply-To: <bbaeab100909270048q5cc8f389we9319f1001c624b6@mail.gmail.com>
References: <5c6f2a5d0909270021n20438083yaf3f854e8dd96c6d@mail.gmail.com>
	<bbaeab100909270048q5cc8f389we9319f1001c624b6@mail.gmail.com>
Message-ID: <5c6f2a5d0909270100m7199a1f4r84f3be9e62455509@mail.gmail.com>

On Sun, Sep 27, 2009 at 8:48 AM, Brett Cannon <brett at python.org> wrote:
> On Sun, Sep 27, 2009 at 00:21, Mark Dickinson <dickinsm at gmail.com> wrote:
[...]
>> So I've now got a file Modules/math_support.c that contains
>> some functions needed by both mathmodule.c and
>> cmathmodule.c, as well as a couple of functions only
>> currently needed by the math module. ?How should I incorporate
>> this file into the build?
>>
>> One obvious solution seems to be to build an extra math_support.so
>> file that's then used by both mathmodule.so and cmathmodule.so.
>> Is this a sensible solution? ?Are there better ways?
>>
>
> Are you planning on exposing any of this outside of those two modules?

No.

> If not then I would change the name to _math.so

That makes sense.

>> A complication to bear in mind is that some users may want to
>> alter Modules/Setup.dist so that either the math module and/or
>> the cmath module is included in the core Python executable.
>
> If you are mucking with Modules.Setup.dist you better know how to
> figure out that those two modules depend on another .so.

Sure.  I'll at least add a comment pointing out the dependence, though.

Thanks,

Mark

From barry at barrys-emacs.org  Sun Sep 27 10:07:47 2009
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 27 Sep 2009 09:07:47 +0100
Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on
	trunk
In-Reply-To: <nad-A69561.19124426092009@news.gmane.org>
References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>
	<nad-A69561.19124426092009@news.gmane.org>
Message-ID: <3400FFB0-8A1D-4C16-9D7A-8664924CCF14@barrys-emacs.org>


On 27 Sep 2009, at 03:12, Ned Deily wrote:

> In article <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A at barrys-emacs.org>,
> Barry Scott <barry at barrys-emacs.org> wrote:
>> I'm working with http://svn.python.org/projects/python/trunk on Mac  
>> OS
>> X 10.6.1
>> using Apples xcode gcc 4.2.1.
>>
>> When I run the following commands:
>>
>> 	./configure --enable-framework --with-universal-archs=32-bit | tee
>> build.config.log
>> 	make clean all | tee build.make.log
>>
>> I end up with a x86_64 Python image.
>>
>> No matter what I use for archs its always the same.
>>
>> I would expect to see -arch arg to GCC but it is not there.
>>
>> 	export CFLAG="-arch i386"

I should have used CFLAGS...

>>
>> did not work either.
>>
>> Am I doing something wrong or is this broken on trunk?
>
> You need to add the enable-universalsdk parameter to configure:
>
>  ... --enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk

Thanks that worked with the 10.5 sdk.

>
> Be aware, though, that universal build support on 10.6 is a bit of a
> work in progress as there are still some interesting unexplained
> universal build issues when building on Snow Leopard (see, for  
> instance,
> the comments in http://bugs.python.org/issue6957).  At the moment, the
> focus is on getting 2.6.3 out the door and the standard installer for
> that will be built on 10.5.

O.k.

Barry


From chambon.pascal at gmail.com  Sun Sep 27 10:20:23 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Sun, 27 Sep 2009 10:20:23 +0200
Subject: [Python-Dev] IO module precisions and exception hierarchy
Message-ID: <4ABF2047.7050501@gmail.com>

Found in current io PEP :
Q: Do we want to mandate in the specification that switching between 
reading and writing on a read-write object implies a .flush()? Or is 
that an implementation convenience that users should not rely on?
-> it seems that the only important matter is : file pointer positions 
and bytes/characters read should always be the ones that the user 
expects, as if there
were no buffering. So flushing or not may stay a non-mandatory 
behaviour, as long as the buffered streams ensures this data integrity.
Eg. If a user opens a file in r/w mode, writes two bytes in it (which 
stay buffered), and then reads 2 bytes, the two bytes read should be 
those on range [2:4] of course, even though the file pointer would, due 
to python buffering, still be at index 0.


Q from me : What happens in read/write text files, when overwriting a 
three-bytes character with a single-byte character ? Or at the contrary, 
when a single chinese character overrides 3 ASCII characters in an UTF8 
file ? Is there any system designed to avoid this data corruption ? Or 
should TextIO classes forbid read+write streams ?


IO Exceptions :
Currently, the situation is kind of fuzzy around EnvironmentError 
subclasses.
* OSError represents errors notified by the OS via errno.h error codes 
(as mirrored in the python "errno" module).
errno.h errors (less than 125 error codes) seem to represent the whole 
of *nix system errors. However, Windows has many more system errors 
(15000+). So windows errors, when they can't be mapped to one of the 
errno errors are raises as "WindowsError" instances (a subclass of 
OSError), with the special attribute "winerror" indicating that win32 
error code.
* IOError are "errors raised because of I/O problems", but they use 
errno codes, like OSError.

Thus, at the moment IOErrors rather have the semantic of "particular 
case of OSError", and it's kind of confusing to have them remain in 
their own separate tree... Furthermore, OSErrors are often used where 
IOErrors would perfectly fit, eg. in low level I/O functions of the OS 
module.
Since OSErrors and IOErrors are slightly mixed up when we deal with IO 
operations, maybe the easiest way to make it clearer would be to push to 
their limits already existing designs.

- the os module should only raise OSErrors, whatever the os operation 
involved (maybe it's already the case in CPython, isn't it ?)
- the io module should only raise IOErrors and its subclasses, so that 
davs can easily take measures depending on the cause of the io failure 
(except 1 OSError exception, it's already the case in _fileio)
- other modules refering to i/o might maybe keep their current (fuzzy) 
behaviour, since they're more platform specific, and should in the end 
be replaced by a crossplatform solution (at least I'd love it to happen)

Until there, there would be no real benefits for the user, compared to 
catching EnvironmentErrors as most probably do. But the sweet thing 
would be to offer a concise but meaningfull IOError hierarchy, so that 
we can easily handle most specific errors gracefully (having a disk full 
is not the same level of gravity as simply having another process 
locking your target file).

Here is a very rough beginning of IOError hierarchy. I'd liek to have 
people's opinion on the relevance of these, as well as on what other 
exceptions should be distinguished from basic IOErrors.

IOError
  +-InvalidStreamError  (eg. we try to write on a stream opened in 
readonly mode)
  +-LockingError
  +-PermissionError (mostly *nix chmod stuffs)
  +-FileNotFoundError
  +-DiskFullError
  +-MaxFileSizeError (maybe hard to implement, happens when we exceed 
4Gb on fat32 and stuffs...)
  +-InvalidFileNameError (filepath max lengths, or "? / : " characters 
in a windows file name...)

Regards,
Pascal




From drkjam at gmail.com  Sun Sep 27 11:06:02 2009
From: drkjam at gmail.com (David Moss)
Date: Sun, 27 Sep 2009 10:06:02 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4ABF0CA8.2030408@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<h8sd2n$n1g$1@ger.gmane.org> <4ABE530D.8090609@v.loewis.de>
	<200909271108.32456.steve@pearwood.info>
	<4ABF0CA8.2030408@v.loewis.de>
Message-ID: <9118C602-C5D3-49A9-B07F-CE61723D4D6A@gmail.com>



Dave M.

On 27 Sep 2009, at 07:56, "Martin v. L?wis" <martin at v.loewis.de> wrote:

>>> As a side note, I would be in favor of dropping the concept of a  
>>> mask
>>> from the library, and only support a prefix length.
>>
>> -1
>>
>>> IPv6 doesn't support masks at all, and even for IPv4, I think there
>>> are conventions (if not RFCs) against using them in a way that does
>>> not correspond to a prefix length.
>>
>> Then the module should only support netmasks of the form
>> (say) '255.255.255.224' (equivalent to "/27"), and reject those
>> like "255.3.255.255". It currently accepts them.
>>
>> Many applications still display netmasks in dot-quad form, and I  
>> would
>> be terribly annoyed if I had to count the bits myself before  
>> passing it
>> to IPv4Address.
>
> I wouldn't ask for that: it should certainly be possible to supply
> masks. However, I would want to reject masks that don't correspond to
> a prefix, and have only the prefix length in the internal  
> representation.
>
+1 on rejection of netmasks without direct CIDR prefix equivalents.  
AFAIK Cisco routers accept them but I don't see how they would be  
useful in practice (unless someone can demonstrate their need for this).

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/drkjam%40gmail.com

From greg.ewing at canterbury.ac.nz  Sun Sep 27 13:16:38 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 27 Sep 2009 23:16:38 +1200
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4ABF2047.7050501@gmail.com>
References: <4ABF2047.7050501@gmail.com>
Message-ID: <4ABF4996.6000900@canterbury.ac.nz>

Pascal Chambon wrote:

> -> it seems that the only important matter is : file pointer positions 
> and bytes/characters read should always be the ones that the user 
> expects, as if there were no buffering.

That sounds right to me.

> Q from me : What happens in read/write text files, when overwriting a 
> three-bytes character with a single-byte character ?

I think you deserve whatever you get. If you want to be able
to overwrite things that accurately, you should be dealing
with the stream at the byte level.

> Here is a very rough beginning of IOError hierarchy.

>  +-InvalidFileNameError (filepath max lengths, or "? / : " characters in 
> a windows file name...)

This might be a bit too precise. Unix just has EINVAL, which
covers any kind of invalid parameter, not just file names.

-- 
Greg


From solipsis at pitrou.net  Sun Sep 27 13:23:43 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 11:23:43 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>
	<200909271205.18488.steve@pearwood.info>
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
Message-ID: <loom.20090927T131544-612@post.gmane.org>

Peter Moody <peter <at> hda3.com> writes:
> 
> To be explicit though, unless I'm drastically misreading what you're
> asking for, I consider the design you're proposing to be broken from a
> usability perspective (striving for academic/pedantic correctness over
> accepted practice).

It is certainly not academic correctness, it is concern over the
understandibility of the library. The behaviour you are proposing is confusing,
and even if you think it is more practical (which I doubt it is), it should
still be avoided IMO. While "192.168.1.2/24" may be an accepted notation in some
situations, it is not a notation for a *network* but for an (address, mask)
pair, or an (address, network) pair.

There was a proposal to have a separate parse_address_and_mask method which
would return a (Address, Network) tuple, I still don't know why you don't seem
to consider it seriously, rather than trying to make the Network class a kind of
all-in-one type conflating different concepts.

Regards

Antoine.



From fuzzyman at voidspace.org.uk  Sun Sep 27 13:31:46 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 27 Sep 2009 12:31:46 +0100
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4ABF2047.7050501@gmail.com>
References: <4ABF2047.7050501@gmail.com>
Message-ID: <4ABF4D22.4090201@voidspace.org.uk>

Pascal Chambon wrote:
> Found in current io PEP :
> Q: Do we want to mandate in the specification that switching between 
> reading and writing on a read-write object implies a .flush()? Or is 
> that an implementation convenience that users should not rely on?
> -> it seems that the only important matter is : file pointer positions 
> and bytes/characters read should always be the ones that the user 
> expects, as if there
> were no buffering. So flushing or not may stay a non-mandatory 
> behaviour, as long as the buffered streams ensures this data integrity.
> Eg. If a user opens a file in r/w mode, writes two bytes in it (which 
> stay buffered), and then reads 2 bytes, the two bytes read should be 
> those on range [2:4] of course, even though the file pointer would, 
> due to python buffering, still be at index 0.
>
>
> Q from me : What happens in read/write text files, when overwriting a 
> three-bytes character with a single-byte character ? Or at the 
> contrary, when a single chinese character overrides 3 ASCII characters 
> in an UTF8 file ? Is there any system designed to avoid this data 
> corruption ? Or should TextIO classes forbid read+write streams ?
>
>
> IO Exceptions :
> Currently, the situation is kind of fuzzy around EnvironmentError 
> subclasses.
> * OSError represents errors notified by the OS via errno.h error codes 
> (as mirrored in the python "errno" module).
> errno.h errors (less than 125 error codes) seem to represent the whole 
> of *nix system errors. However, Windows has many more system errors 
> (15000+). So windows errors, when they can't be mapped to one of the 
> errno errors are raises as "WindowsError" instances (a subclass of 
> OSError), with the special attribute "winerror" indicating that win32 
> error code.
> * IOError are "errors raised because of I/O problems", but they use 
> errno codes, like OSError.
>
> Thus, at the moment IOErrors rather have the semantic of "particular 
> case of OSError", and it's kind of confusing to have them remain in 
> their own separate tree... Furthermore, OSErrors are often used where 
> IOErrors would perfectly fit, eg. in low level I/O functions of the OS 
> module.
> Since OSErrors and IOErrors are slightly mixed up when we deal with IO 
> operations, maybe the easiest way to make it clearer would be to push 
> to their limits already existing designs.
>
> - the os module should only raise OSErrors, whatever the os operation 
> involved (maybe it's already the case in CPython, isn't it ?)
> - the io module should only raise IOErrors and its subclasses, so that 
> davs can easily take measures depending on the cause of the io failure 
> (except 1 OSError exception, it's already the case in _fileio)
> - other modules refering to i/o might maybe keep their current (fuzzy) 
> behaviour, since they're more platform specific, and should in the end 
> be replaced by a crossplatform solution (at least I'd love it to happen)
>
> Until there, there would be no real benefits for the user, compared to 
> catching EnvironmentErrors as most probably do. But the sweet thing 
> would be to offer a concise but meaningfull IOError hierarchy, so that 
> we can easily handle most specific errors gracefully (having a disk 
> full is not the same level of gravity as simply having another process 
> locking your target file).
>
> Here is a very rough beginning of IOError hierarchy. I'd liek to have 
> people's opinion on the relevance of these, as well as on what other 
> exceptions should be distinguished from basic IOErrors.
>
> IOError
>  +-InvalidStreamError  (eg. we try to write on a stream opened in 
> readonly mode)
>  +-LockingError
>  +-PermissionError (mostly *nix chmod stuffs)
>  +-FileNotFoundError
>  +-DiskFullError
>  +-MaxFileSizeError (maybe hard to implement, happens when we exceed 
> 4Gb on fat32 and stuffs...)
>  +-InvalidFileNameError (filepath max lengths, or "? / : " characters 
> in a windows file name...)

Personally I'd love to see a richer set of exceptions for IO errors, so 
long as they can be implemented for all supported platforms and no 
information (err number from the os) is lost.

I've been implementing a fake 'file' type [1] for Silverlight which does 
IO operations using local browser storage. The use case is for an online 
Python tutorial running in the browser [2]. Whilst implementing the 
exception behaviour (writing to a file open in read mode, etc) I 
considered improving the exception messages as they are very poor - but 
decided that being similar to CPython was more important.

Michael

[1] 
http://code.google.com/p/trypython/source/browse/trunk/trypython/app/storage.py 
and 
http://code.google.com/p/trypython/source/browse/trunk/trypython/app/tests/test_storage.py
[2] http://www.trypython.org/

>
> Regards,
> Pascal
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From solipsis at pitrou.net  Sun Sep 27 13:33:03 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 11:33:03 +0000 (UTC)
Subject: [Python-Dev] Fuzziness in io module specs -	PEP update
	proposition V2
References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com>
	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>
	<200909201007.11923.steve@pearwood.info>
	<4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com>
	<4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com>
	<eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com>
	<4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com>
	<4ABF1B3B.4080106@gmail.com>
Message-ID: <h9nihf$9f4$1@ger.gmane.org>


Hello,

> So here is the proposed semantic, which matches established conventions:
> 
> *IOBase.truncate(n: int = None) -> int*
[...]

I still don't think there is a sufficient benefit in breaking 
compatibility. If you want the file pointer to remain the same, you can 
save it first and restore it afterwards manually.

> *Propositions of doc update*

Please open tracker issues for these kinds of suggestions.

Regards

Antoine.


From solipsis at pitrou.net  Sun Sep 27 13:49:06 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 11:49:06 +0000 (UTC)
Subject: [Python-Dev] IO module precisions and exception hierarchy
References: <4ABF2047.7050501@gmail.com>
Message-ID: <h9njfi$9f4$2@ger.gmane.org>

Le Sun, 27 Sep 2009 10:20:23 +0200, Pascal Chambon a ?crit?:
> Q: Do we want to mandate in the specification that switching between
> reading and writing on a read-write object implies a .flush()?

It doesn't and shouldn't be mandated in the specification, IMO. An 
implementation should be free to optimize out those "implicit flush() 
calls" for performance reasons.

> Eg. If a user opens a file in r/w mode, writes two bytes in it (which
> stay buffered), and then reads 2 bytes, the two bytes read should be
> those on range [2:4] of course, even though the file pointer would, due
> to python buffering, still be at index 0.

Actually the raw file pointer would be at index N, where N is at least 
the buffer size (say 4096).

However, it is not specified how the raw file pointer behaves when using 
a Buffered{Reader, Writer, Random} wrapper over it. The buffered object 
is free to do what it wants with the raw stream until flush() is called 
(or the file is closed, which calls flush() implicitly).
Even after flush() is called, the raw file pointer can still be at 
whatever place, but the raw file contents are consistent with the view 
given by the buffered object.

> Q from me : What happens in read/write text files, when overwriting a
> three-bytes character with a single-byte character ? Or at the contrary,
> when a single chinese character overrides 3 ASCII characters in an UTF8
> file ? Is there any system designed to avoid this data corruption ? Or
> should TextIO classes forbid read+write streams ?

What happens isn't specified, but in practice (with the current 
implementation) the overwriting will happen at the byte level, without 
any check for correctness at the character level.

Actually, read+write text streams are implemented quite crudely, and 
little testing is done of them. The reason, as you discovered, is that 
the semantics are too weak, and it is not obvious how stronger semantics 
could look like. People wanting to do sophisticated random reads+writes 
over a text file should probably handle the encoding themselves and 
access the file at the binary level.

> Here is a very rough beginning of IOError hierarchy. I'd liek to have
> people's opinion on the relevance of these, as well as on what other
> exceptions should be distinguished from basic IOErrors.

This deserves its own PEP IMO, although I'm not sure it would be accepted 
(ISTR the idea of a detailed IO exception hierarchy was already refused 
in the past).

Regards

Antoine.



From python at mrabarnett.plus.com  Sun Sep 27 16:51:34 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 27 Sep 2009 15:51:34 +0100
Subject: [Python-Dev] Fuzziness in io module specs -	PEP update
 proposition V2
In-Reply-To: <4ABF1B3B.4080106@gmail.com>
References: <4AB3DCC9.5050209@gmail.com>	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<200909201007.11923.steve@pearwood.info>	<4AB58159.3040303@canterbury.ac.nz>	<4AB5D6CB.6050604@gmail.com>	<4AB5E8DD.6050204@canterbury.ac.nz>	<4AB5FA81.5060408@gmail.com>	<eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com>	<4AB6590E.2070800@gmail.com>	<4AB67894.5020903@mrabarnett.plus.com>
	<4ABF1B3B.4080106@gmail.com>
Message-ID: <4ABF7BF6.5070903@mrabarnett.plus.com>

Pascal Chambon wrote:
> Hello
> 
> Below is a corrected version of the PEP update, adding the start/end 
> indexes proposition and fixing functions signatures. Does anyone 
> disagree with these specifications ? Or can we consider it as a target 
> for the next versions of the io module ?
> I would have no problem to implement this behaviour in my own pure 
> python FileIO system, however if someone is willing to patch the _fileio 
> implementation, it'd save a lot of time - I most probably won't have the 
> means to setup a C compilation environment under windows and linux, and 
> properly update/test this, before January (when I get freelance...).
> 
> I launch another thread on other to-be-discussed IO points B-)
> 
> Regards,
> Pascal
> 
> ================ PEP UPDATE for new I/O system - v2 ===========
> 
> **Truncate and file pointer semantics**
> 
> Rationale :
> 
> The current implementation of truncate() always move the file pointer to 
> the new end of file.
> 
> This behaviour is interesting for compatibility, if the file has been 
> reduced and the file pointer is now past its end, since some platforms 
> might require 0 <= filepointer <= filesize.
> 
> However, there are several arguments against this semantic:
> 
>     * Most common standards (posix, win32?) allow the file pointer to be
>       past the end of file, and define the behaviour of other stream
>       methods in this case
>     * In many cases, moving the filepointer when truncating has no
>       reasons to happen (if we?re extending the file, or reducing it
>       without going beneath the file pointer)
>     * Making 0 <= filepointer <= filesize a global rule of the python IO
>       module doesn?t seems possible, since it would require
>       modifications of the semantic of other methods (eg. seek() should
>       raise exceptions or silently disobey when asked to move the
>       filepointer past the end of file), and lead to incoherent
>       situations when concurrently accessing files without locking (what
>       if another process truncates to 0 bytes the file you?re writing ?)
> 
> So here is the proposed semantic, which matches established conventions:
> 
> *IOBase.truncate(n: int = None) -> int*
> 
> Resizes the file to the size specified by the positive integer n, or by 
> the current filepointer position if n is None.
> 
> The file must be opened with write permissions.
> 
> If the file was previously larger than size, the extra data is discarded.
> If the file was previously shorter than size, its size is increased, and
> the extended area appears as if it were zero-filled.
> 
Instead of "than size", perhaps "than n".

> In any case, the file pointer is left unchanged, and may point beyond
> the end of file.
> 
> Note: trying to read past the end of file returns an empty string, and
> trying to write past the end of file extends it by zero-ing the gap. On
> rare platforms which don't support file pointers to be beyond the end of
> file, all these behaviours shall be faked thanks to internal storage of
> the "wanted" file pointer position (silently extending the file, if
> necessary, when a write operation occurs).
> 
>  
> 
> *Propositions of doc update*
> 
> *RawIOBase*.read(n: int) -> bytes
> 
> Read up to n bytes from the object and return them. Fewer than n bytes
> may be returned if the operating system call returns fewer than n bytes.
> If 0 bytes are returned, and n was not 0, this indicates end of file. If
> the object is in non-blocking mode and no bytes are available, the call
> returns None.
> 
> 
> *RawIOBase*.readinto(b: bytearray, [start: int = None], [end: int = 
> None]) -> int
> 
> start and end are used as slice indexes, so that the bytearray taken 
> into account is actually range = b[start:end] (or b[start:], b[:end] or 
> b[:], depending on the arguments which are not None).
> 
> Read up to len(range) bytes from the object and store them in b, returning
> the number of bytes read. Like .read, fewer than len(range) bytes may be
> read, and 0 indicates end of file if len(range) is not 0.
> None is returned if a non-blocking object has no bytes available. The 
> length of b is never changed.
> 
Should an exception be raised if start and/or end are out of range?

From stephen at xemacs.org  Sun Sep 27 17:09:44 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 28 Sep 2009 00:09:44 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090927T131544-612@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>
	<200909271205.18488.steve@pearwood.info>
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
Message-ID: <878wg0mnk7.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > There was a proposal to have a separate parse_address_and_mask
 > method which would return a (Address, Network) tuple, I still don't
 > know why you don't seem to consider it seriously, rather than
 > trying to make the Network class a kind of all-in-one type
 > conflating different concepts.

Because he thinks about the problem space differently from you.
Specifically, AFAICS he does not (maybe even "can't") see a reason to
distinguish an AddressWithMask from a Network.  If so, it's a
reasonable API choice for him to choose not to have separate classes
to represent such similar concepts.  That's all there is to it.

I personally do not have a problem with that, except that you
apparently can't grasp his way of thinking, and he apparently can't
grasp yours.  I'm -1 on PEP 3144 primarily because of this
communications gap.

Secondarily because I agree that it's unnatural that a Network
instance can have an arbitrary distinguished address other than those
defined in the RFCs (the network and broadcast addresses), especially
since it matters to equality comparisons.  (But I personally would use
ipaddr if it were in the stdlib despite that.)

From python at mrabarnett.plus.com  Sun Sep 27 17:14:35 2009
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 27 Sep 2009 16:14:35 +0100
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4ABF2047.7050501@gmail.com>
References: <4ABF2047.7050501@gmail.com>
Message-ID: <4ABF815B.2010501@mrabarnett.plus.com>

Pascal Chambon wrote:
 > Found in current io PEP :
 > Q: Do we want to mandate in the specification that switching between
 > reading and writing on a read-write object implies a .flush()? Or is
 > that an implementation convenience that users should not rely on?
 > -> it seems that the only important matter is : file pointer
 > positions and bytes/characters read should always be the ones that
 > the user expects, as if there were no buffering. So flushing or not
 > may stay a non-mandatory behaviour, as long as the buffered streams
 > ensures this data integrity. Eg. If a user opens a file in r/w mode,
 > writes two bytes in it (which stay buffered), and then reads 2 bytes,
 > the two bytes read should be those on range [2:4] of course, even
 > though the file pointer would, due to python buffering, still be at
 > index 0.
 >
+1
 >
 > Q from me : What happens in read/write text files, when overwriting a
 > three-bytes character with a single-byte character ? Or at the
 > contrary, when a single chinese character overrides 3 ASCII characters
 > in an UTF8 file ? Is there any system designed to avoid this data
 > corruption ? Or should TextIO classes forbid read+write streams ?
 >
If the characters are always the same number of bytes) then overwriting
could be possible; that would, however, make the behaviour more
complicated (sometimes you can, sometimes you can't), so it's probably
just simpler to forbid read+write text streams.


From solipsis at pitrou.net  Sun Sep 27 17:32:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 15:32:14 +0000 (UTC)
Subject: [Python-Dev] IO module precisions and exception hierarchy
References: <4ABF2047.7050501@gmail.com> <4ABF815B.2010501@mrabarnett.plus.com>
Message-ID: <loom.20090927T172825-519@post.gmane.org>

MRAB <python <at> mrabarnett.plus.com> writes:
> 
> If the characters are always the same number of bytes) then overwriting
> could be possible; that would, however, make the behaviour more
> complicated (sometimes you can, sometimes you can't), so it's probably
> just simpler to forbid read+write text streams.

Forbidding them would be gratuitous. There are perfectly safe uses, for example
if all writes are appends, or if you use a fixed-size encoding.
(and besides, it would break backwards compatibility)

Regards

Antoine.



From foom at fuhm.net  Sun Sep 27 17:43:00 2009
From: foom at fuhm.net (James Y Knight)
Date: Sun, 27 Sep 2009 11:43:00 -0400
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4ABF2047.7050501@gmail.com>
References: <4ABF2047.7050501@gmail.com>
Message-ID: <E4A61F3D-0551-4FA7-B6B2-F3B63168B537@fuhm.net>

On Sep 27, 2009, at 4:20 AM, Pascal Chambon wrote:
> Thus, at the moment IOErrors rather have the semantic of "particular  
> case of OSError", and it's kind of confusing to have them remain in  
> their own separate tree... Furthermore, OSErrors are often used  
> where IOErrors would perfectly fit, eg. in low level I/O functions  
> of the OS module.
> Since OSErrors and IOErrors are slightly mixed up when we deal with  
> IO operations, maybe the easiest way to make it clearer would be to  
> push to their limits already existing designs.

How about just making IOError = OSError, and introducing your proposed  
subclasses? Does the usage of IOError vs OSError have *any* useful  
semantics?

James

From solipsis at pitrou.net  Sun Sep 27 17:52:38 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 17:52:38 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <878wg0mnk7.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>
	<200909271205.18488.steve@pearwood.info>
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<878wg0mnk7.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <1254066758.5532.40.camel@localhost>


> I personally do not have a problem with that, except that you
> apparently can't grasp his way of thinking, and he apparently can't
> grasp yours.

If I was the only one disagreeing it wouldn't be that annoying (except
for me :-)).

> I'm -1 on PEP 3144 primarily because of this
> communications gap.

I must say I have started to lean in that direction, too.
If it is not possible to produce a result (a PEP) that solves all major
design issues, it will produce the kind of little-liked modules that
people have recently complained about in an stdlib-sig thread.

Regards

Antoine.



From guido at python.org  Sun Sep 27 18:37:40 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 27 Sep 2009 09:37:40 -0700
Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the
	Python Standard Library
In-Reply-To: <4ABF1728.9@v.loewis.de>
References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com> 
	<8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com> 
	<Pine.LNX.4.64.0908191148530.30683@kimball.webabinitio.net> 
	<8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com> 
	<4A8C4264.2040708@trueblade.com>
	<d9047f780908191128p12107a1exc20dabc34ddff5de@mail.gmail.com> 
	<20090916090849.GH12810@ruber.office.udmvt.ru>
	<4ABE5B24.1090003@v.loewis.de> 
	<ca471dc20909261925o2b24d36fjeda61b89be58d3e0@mail.gmail.com> 
	<4ABF1728.9@v.loewis.de>
Message-ID: <ca471dc20909270937w45b1eed4n169fbe1ade7cb572@mail.gmail.com>

On Sun, Sep 27, 2009 at 12:41 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Guido van Rossum wrote:
>> I realize I'm late to this party, but this is just a naming issue, right?
>
> Not really (AFAIU).
>
>> For any network, there are two special addresses, one with the last
>> bits all zeros, one with the last bits all ones. We can call them A
>> and B, or network and broadcast, or zeros and ones, or whatever we
>> care. But their definitions are two well-defined functions for all
>> networks (assuming a network is defined as an IP address and a number
>> of bits), even if in an extreme case the functions return the same
>> value.
>
> For the broadcast address, it's different, since you might also use
> it in programming (i.e. when sending broadcasts).

So there is no need to look up the broadcast address in the
configuration? Don't you have to look up the rest of the network
configuration too? (Or otherwise how do you know your network address
and the value of /N?)

> Now, it seems that people will typically use undirected broadcasts
> (INADDR_BROADCAST). However, I assume there are also applications
> for directed broadcasts; in this case, it would matter to know the
> broadcast address of a network.

I don't know, and it seems you are not familiar with an actual case
either. I would assume that apps doing such low-level work would have
direct access to the complete network configuration, of which the
broadcast address to use would be a separate argument. The ipaddr
library seems most useful for doing manipulations and computations on
addresses and networks that may bear no relationship to the current
host's configuration -- e.g. off-line verification of firewall
configurations, or whatever.

>> What is actually configured on a particular host to be the broadcast
>> address is a separate issue, even if *by convention* in most cases it
>> is given by one of the above functions -- the network object doesn't
>> care, the configuration object is something else (and outside the
>> scope of this PEP).
>
> It's a little bit stronger than convention; there are RFCs saying
> that the all-ones (-1) address has to be the broadcast address.

Sure, but what is the status of those RFCs? (Plenty of RFCs are
unimplemented or superseded by others etc.)

In any case we seem to agree that 'broadcast' is a fine name for the
'all ones' address so I don't want to argue this point any further. :)

> This is like saying that it is out of scope of the socket protocol
> whether TCP is the IP protocol 6, and that individual hosts may do
> it differently - yes, they may, but then everything stops working.

Well, except that if al hosts on a closed network were to define their
alternate value in the appropriate header, then everything would work
(within that closed network!), even Python (which gets the TCP value
from the system header too).

But I don't think that that is relevant to ipaddr, which has to be
able to make decisions about arbitrary IP addresses and networks
without having access to the hosts or their configurations. It can
only look at the syntax.

>> IMO "real life examples" don't matter for the definitions of the
>> functions
>
> That I agree with.
>
>> and I would personally be happy to name them network and
>> broadcast, since I know their definitions and their typical uses and
>> these match pretty closely. The expectation should be clearly set that
>> these are pure functions though and do not imply knowledge of the
>> configuration of any given host.
>
> That I also agree with.
>
> However, there is this issue of /31 networks, where an 8-year-old
> RFC specifies how to do broadcasts on these. I think RFCs should
> be considered wherever applicable.

Considered wherever applicable, yes. I'm assuming you're talking about
RFC 3021 -- which says "standards track" but I can't find whether it
is actually a standard or how widespread its implementation is. It
might well be obsolete by now, or widely ignored, or irrelevant
outside a few drivers. Another question is whether it *always* applies
when a /31 network is used. If not, then we're back to the situation
where we can't know the configuration, and declaring the RFC
applicable in all cases doesn't make it so.

RFC 3021 adds an odd wart to the "all ones" and "all zeros" functions
(or perhaps only to "all ones"?). If after further examination of the
facts we find that it should be honored, then what do we do for
.network and .broadcast on a /32 "network"? Is that an error or should
it just return the one IP address of which the "network" consists?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From peter at hda3.com  Sun Sep 27 18:47:27 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 09:47:27 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090927T131544-612@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>
	<200909271205.18488.steve@pearwood.info>
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
Message-ID: <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>

On Sun, Sep 27, 2009 at 4:23 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Peter Moody <peter <at> hda3.com> writes:
>>
>> To be explicit though, unless I'm drastically misreading what you're
>> asking for, I consider the design you're proposing to be broken from a
>> usability perspective (striving for academic/pedantic correctness over
>> accepted practice).
>
> It is certainly not academic correctness, it is concern over the
> understandibility of the library. The behaviour you are proposing is confusing,
> and even if you think it is more practical (which I doubt it is), it should
> still be avoided IMO. While "192.168.1.2/24" may be an accepted notation in some
> situations, it is not a notation for a *network* but for an (address, mask)
> pair, or an (address, network) pair.
>
> There was a proposal to have a separate parse_address_and_mask method which
> would return a (Address, Network) tuple, I still don't know why you don't seem
> to consider it seriously, rather than trying to make the Network class a kind of
> all-in-one type conflating different concepts.

The reason (aside from the name) that I'm not going to include this in
ipaddr is that it would require the user to deal with two objects when
one would suffice. It's similar to getting two return values from
float().

 integer, fraction = float('1.25')

crazy, right?

Finally, to Stephen's point about seeing the other side of the
argument, I wrote this offlist a week ago:

  I *understand* what you're saying, I *understand* that 192.168.1.1/24
  isn't a network, 192.168.1.0/24 is a network, and 192.168.1.1 is an ip
  address on that network and it has an associated netmask that helps
  describe its network, etc. etc. what I'm saying is that, enforcing
  that restriction by introducing 2 new classes (ipv4 is not comparable
  with ipv6, I've actually got a good quote from Vint Cerf about this,
  so any AddressWithNetmask (?!?) class would have to be duplicated for
  both addr types) or something else as of yet described, is not useful.
  it is in fact, detrimental (optionally enforcing that restriction via
  an argument to the Network constructors is, however, useful)

Cheers,
/peter

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From solipsis at pitrou.net  Sun Sep 27 19:22:41 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 17:22:41 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com>
	<D13DB604-903B-4014-93E7-0D1F8B6112C8@barrys-emacs.org>
	<200909271205.18488.steve@pearwood.info>
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
Message-ID: <loom.20090927T185114-628@post.gmane.org>

Peter Moody <peter <at> hda3.com> writes:
> 
> The reason (aside from the name) that I'm not going to include this in
> ipaddr is that it would require the user to deal with two objects when
> one would suffice.

You make it sound like it's a burden, but "dealing" with two objects is not
something extraordinary or bizarre, it happens all the time when you do network
programming: e.g. ('127.0.0.1', 80).

(or would you argue that Address objects should have an optional distinguishing
port number, for "convenience" reasons?)

If you need the two objects, whether you access them through separate variables
(`network` and `host`) or through attribute access (`network` and
`network.host`) doesn't really matter. You can even pretend the tuple is an
atomic object and store it as-is if you want. I'm not sure what annoys you here.

> It's similar to getting two return values from
> float().

I don't see how it's similar, since a float is a clearly defined atomic entity.
Most float operations become meaningless if you consider the integral and the
fractional part separately.

Regards

Antoine.



From steve at pearwood.info  Sun Sep 27 19:30:29 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 28 Sep 2009 03:30:29 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
Message-ID: <200909280330.30643.steve@pearwood.info>

On Mon, 28 Sep 2009 02:47:27 am Peter Moody wrote:

> > There was a proposal to have a separate parse_address_and_mask
> > method which would return a (Address, Network) tuple, I still don't
> > know why you don't seem to consider it seriously, rather than
> > trying to make the Network class a kind of all-in-one type
> > conflating different concepts.
>
> The reason (aside from the name) that I'm not going to include this
> in ipaddr is that it would require the user to deal with two objects
> when one would suffice. It's similar to getting two return values 
> from float().
>
>  integer, fraction = float('1.25')
>
> crazy, right?

Not if you want a separate integer and fraction object. There are plenty 
of use-cases for such things, and the math module includes a function, 
modf(), that does precisely that.. 


> Finally, to Stephen's point about seeing the other side of the
> argument, I wrote this offlist a week ago:
>
>   I *understand* what you're saying, I *understand* that
> 192.168.1.1/24 isn't a network, 

But you still want to treat it as one.

Could you explain what benefit there is for allowing the user to create 
network objects that don't represent networks? Is there a use-case 
where these networks-that-aren't-networks are something other than a 
typo? Under what circumstances would I want to specify a network as 
192.168.1.1/24 instead of 192.168.1.0/24?



-- 
Steven D'Aprano

From peter at hda3.com  Sun Sep 27 19:53:27 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 10:53:27 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909280330.30643.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
Message-ID: <8517e9350909271053n174ee3dy9173ecf73e3dd39b@mail.gmail.com>

On Sun, Sep 27, 2009 at 10:30 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Mon, 28 Sep 2009 02:47:27 am Peter Moody wrote:
>
>> > There was a proposal to have a separate parse_address_and_mask
>> > method which would return a (Address, Network) tuple, I still don't
>> > know why you don't seem to consider it seriously, rather than
>> > trying to make the Network class a kind of all-in-one type
>> > conflating different concepts.
>>
>> The reason (aside from the name) that I'm not going to include this
>> in ipaddr is that it would require the user to deal with two objects
>> when one would suffice. It's similar to getting two return values
>> from float().
>>
>> ?integer, fraction = float('1.25')
>>
>> crazy, right?
>
> Not if you want a separate integer and fraction object. There are plenty
> of use-cases for such things, and the math module includes a function,
> modf(), that does precisely that..
>
>
>> Finally, to Stephen's point about seeing the other side of the
>> argument, I wrote this offlist a week ago:
>>
>> ? I *understand* what you're saying, I *understand* that
>> 192.168.1.1/24 isn't a network,
>
> But you still want to treat it as one.
>
> Could you explain what benefit there is for allowing the user to create
> network objects that don't represent networks? Is there a use-case
> where these networks-that-aren't-networks are something other than a
> typo? Under what circumstances would I want to specify a network as
> 192.168.1.1/24 instead of 192.168.1.0/24?

this is pretty ridiculous. if in the last two months you haven't seen
a single use case, then you haven't been paying attention.


> --
> Steven D'Aprano
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From peter at hda3.com  Sun Sep 27 20:04:04 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 11:04:04 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090927T185114-628@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909271205.18488.steve@pearwood.info>
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
Message-ID: <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>

On Sun, Sep 27, 2009 at 10:22 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Peter Moody <peter <at> hda3.com> writes:
>>
>> The reason (aside from the name) that I'm not going to include this in
>> ipaddr is that it would require the user to deal with two objects when
>> one would suffice.
>
> You make it sound like it's a burden, but "dealing" with two objects is not
> something extraordinary or bizarre, it happens all the time when you do network
> programming: e.g. ('127.0.0.1', 80).
>
> (or would you argue that Address objects should have an optional distinguishing
> port number, for "convenience" reasons?)

I'm not sure what you're talking about, I've never argued to add layer
4 information to ipaddr.

> If you need the two objects, whether you access them through separate variables
> (`network` and `host`) or through attribute access (`network` and
> `network.host`) doesn't really matter. You can even pretend the tuple is an
> atomic object and store it as-is if you want. I'm not sure what annoys you here.

this is tautological. if you need two objects, then two objects is
what you need. I'm saying that you don't need two objects for this,
that common accepted practice is to represent this as one.  Note,
there is *nothing* stopping you currently from using two objects for
this, an IPAddress object and a strict=True IPNetwork object.

I'm not sure what annoys you about this.

>> It's similar to getting two return values from
>> float().
>
> I don't see how it's similar, since a float is a clearly defined atomic entity.
> Most float operations become meaningless if you consider the integral and the
> fractional part separately.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From solipsis at pitrou.net  Sun Sep 27 20:17:34 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 18:17:34 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909271205.18488.steve@pearwood.info>
	<8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
Message-ID: <loom.20090927T201052-343@post.gmane.org>

Peter Moody <peter <at> hda3.com> writes:
> 
> > (or would you argue that Address objects should have an optional 
distinguishing
> > port number, for "convenience" reasons?)
> 
> I'm not sure what you're talking about, I've never argued to add layer
> 4 information to ipaddr.

It was an analogy, just like your "float" analogy.

> I'm not sure what annoys you about this.

I have already explained it: what annoys me is that it makes `Network` a hybrid
object conflating two independent concepts, and makes the library less
understandable as a result.

Others have already pointed out that it makes operations like equality
confusing. 

Regards

Antoine.



From peter at hda3.com  Sun Sep 27 21:18:58 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 12:18:58 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090927T201052-343@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
Message-ID: <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>

On Sun, Sep 27, 2009 at 11:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Peter Moody <peter <at> hda3.com> writes:
>>
>> > (or would you argue that Address objects should have an optional
> distinguishing
>> > port number, for "convenience" reasons?)
>>
>> I'm not sure what you're talking about, I've never argued to add layer
>> 4 information to ipaddr.
>
> It was an analogy, just like your "float" analogy.
>
>> I'm not sure what annoys you about this.
>
> I have already explained it: what annoys me is that it makes `Network` a hybrid
> object conflating two independent concepts, and makes the library less
> understandable as a result.

I understand that this is your assertion.

Here's the thing, your use case is already supported. IPv?Network
objects can only be created of network addresses, exceptions are
raised if all the host bits aren't 0, you can create and only deal
with IPv4Address('192.168.1.1') and IPv4Network('192.168.1.0/24',
strict=True) etc etc. That's not how I (nor many other network
administrators) would use it, but it's doable. what you're claiming is
that my use case is invalid.

that's what I claim is broken.

There have been folks on both sides claiming that this design is both
fundamentally confusing and fundamentally sound.  The confused have
obviously been more vocal and less willing to compromise. So, the
basic design of the library stands as it is, minor implementation and
documentation bugs not withstanding.  I'm not going to make ipaddr
less useful (strictly removing functionality), more bulky and
confusing (adding more confusingly named classes and methods) or
otherwise break the library in a vain attempt to have it included in
the stdlib.

Cheers,
/peter

> Others have already pointed out that it makes operations like equality
> confusing.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From foom at fuhm.net  Sun Sep 27 21:40:22 2009
From: foom at fuhm.net (James Y Knight)
Date: Sun, 27 Sep 2009 15:40:22 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4ABEFA5D.40506@gmail.com>
	<8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
Message-ID: <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>


On Sep 27, 2009, at 3:18 PM, Peter Moody wrote:

> administrators) would use it, but it's doable. what you're claiming is
> that my use case is invalid.
>
> that's what I claim is broken.

He's claiming your solution to address your use case is confusing, not  
that the use case is invalid.

> I'm not going to make ipaddr
> less useful (strictly removing functionality), more bulky and
> confusing (adding more confusingly named classes and methods) or
> otherwise break the library in a vain attempt to have it included in
> the stdlib.

If I understand correctly, the proposal for addressing the issue is to  
make two rather simple changes:
1) if strict=False, mask off the bits described by the netmask when  
creating an IPNetwork, such that the host bits are always 0.
2) add a single new function:

def parse_net_and_addr(s):
   return (IPNetwork(s), IPAddress(s.split('/')[0]))

James

From martin at v.loewis.de  Sun Sep 27 21:50:09 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 27 Sep 2009 21:50:09 +0200
Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the
 Python Standard Library
In-Reply-To: <ca471dc20909270937w45b1eed4n169fbe1ade7cb572@mail.gmail.com>
References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com>
	<8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com>
	<Pine.LNX.4.64.0908191148530.30683@kimball.webabinitio.net>
	<8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com>
	<4A8C4264.2040708@trueblade.com>
	<d9047f780908191128p12107a1exc20dabc34ddff5de@mail.gmail.com>
	<20090916090849.GH12810@ruber.office.udmvt.ru>
	<4ABE5B24.1090003@v.loewis.de>
	<ca471dc20909261925o2b24d36fjeda61b89be58d3e0@mail.gmail.com>
	<4ABF1728.9@v.loewis.de>
	<ca471dc20909270937w45b1eed4n169fbe1ade7cb572@mail.gmail.com>
Message-ID: <4ABFC1F1.5050507@v.loewis.de>

>> For the broadcast address, it's different, since you might also use
>> it in programming (i.e. when sending broadcasts).
> 
> So there is no need to look up the broadcast address in the
> configuration? Don't you have to look up the rest of the network
> configuration too? (Or otherwise how do you know your network address
> and the value of /N?)

You would have to know either the broadcast address directly, or
the network (i.e. address and prefix length).

>>> What is actually configured on a particular host to be the broadcast
>>> address is a separate issue, even if *by convention* in most cases it
>>> is given by one of the above functions -- the network object doesn't
>>> care, the configuration object is something else (and outside the
>>> scope of this PEP).
>> It's a little bit stronger than convention; there are RFCs saying
>> that the all-ones (-1) address has to be the broadcast address.
> 
> Sure, but what is the status of those RFCs? (Plenty of RFCs are
> unimplemented or superseded by others etc.)

The one that says that the broadcast address is -1 (and 0 should
also be supported) is STD 3. The one that talks about 31-bit prefixes
(RFC 3021) is a proposed standard.

> RFC 3021 adds an odd wart to the "all ones" and "all zeros" functions
> (or perhaps only to "all ones"?).

To both: it allows them to be used as host addresses, even though STD 3
says that they are reserved and must not be assigned to hosts.

> If after further examination of the
> facts we find that it should be honored, then what do we do for
> .network and .broadcast on a /32 "network"? 

For these, I would first like to find out what their specification
is.

Regards,
Martin


From peter at hda3.com  Sun Sep 27 21:56:03 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 12:56:03 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
Message-ID: <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>

On Sun, Sep 27, 2009 at 12:40 PM, James Y Knight <foom at fuhm.net> wrote:
>
> On Sep 27, 2009, at 3:18 PM, Peter Moody wrote:
>
>> administrators) would use it, but it's doable. what you're claiming is
>> that my use case is invalid.
>>
>> that's what I claim is broken.
>
> He's claiming your solution to address your use case is confusing, not that
> the use case is invalid.

this isn't actually true.

Steven D'Aprano wrote:

  Could you explain what benefit there is for allowing the user to create
  network objects that don't represent networks? Is there a use-case
  where these networks-that-aren't-networks are something other than a
  typo? Under what circumstances would I want to specify a network as
  192.168.1.1/24 instead of 192.168.1.0/24?

that pretty flatly states that my use case is invalid.

>> I'm not going to make ipaddr
>> less useful (strictly removing functionality), more bulky and
>> confusing (adding more confusingly named classes and methods) or
>> otherwise break the library in a vain attempt to have it included in
>> the stdlib.
>
> If I understand correctly, the proposal for addressing the issue is to make
> two rather simple changes:

i wish it were this easy.

> 1) if strict=False, mask off the bits described by the netmask when creating
> an IPNetwork, such that the host bits are always 0.

I haven't heard anyone suggest auto-masking bits, but otherwise that
would be strict=True.

> 2) add a single new function:
>
> def parse_net_and_addr(s):
> ?return (IPNetwork(s), IPAddress(s.split('/')[0]))

I've only heard talk of new classes and new methods, not new
constructor functions. In fact, when I asked for explicit
clarification of what was required, a new constructor function "ala
IPAddress and IPNetwork (so the current classes remain largely the
same and the various constructors enforce certain restrictions)", I
was told, no, a new object AddressWithMask was actually required. I've
no problem with a constructor like this, but this is not what people
have been asking for.

Cheers,
/peter

> James
>

From solipsis at pitrou.net  Sun Sep 27 22:15:26 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 20:15:26 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
Message-ID: <loom.20090927T220341-422@post.gmane.org>

Peter Moody <peter <at> hda3.com> writes:
> 
> On Sun, Sep 27, 2009 at 12:40 PM, James Y Knight <foom <at> fuhm.net> wrote:
> >
> > On Sep 27, 2009, at 3:18 PM, Peter Moody wrote:
> >
> >> administrators) would use it, but it's doable. what you're claiming is
> >> that my use case is invalid.
> >>
> >> that's what I claim is broken.
> >
> > He's claiming your solution to address your use case is confusing, not that
> > the use case is invalid.
> 
> this isn't actually true.
> 
> Steven D'Aprano wrote:
[...]

That's Steven, your original sentence was about me.

> > 1) if strict=False, mask off the bits described by the netmask when creating
> > an IPNetwork, such that the host bits are always 0.
> 
> I haven't heard anyone suggest auto-masking bits, but otherwise that
> would be strict=True.

I would expect strict=True to raise an error if the lower bits are non-zero, not
to silently erase them. strict=False would be the option that silently erases
lower bits.
(that's why it's named `strict`, after all :-))

> > 2) add a single new function:
> >
> > def parse_net_and_addr(s):
> > ?return (IPNetwork(s), IPAddress(s.split('/')[0]))
> 
> I've only heard talk of new classes and new methods, not new
> constructor functions.

Well, "method" in that context meant "class method" since the results aren't
dependent on a particular instance. Of course, both a class method or a
module-level function would be fine.

Regards

Antoine.



From peter at hda3.com  Sun Sep 27 22:36:37 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 13:36:37 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090927T220341-422@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
Message-ID: <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>

On Sun, Sep 27, 2009 at 1:15 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Peter Moody <peter <at> hda3.com> writes:
>>
>> On Sun, Sep 27, 2009 at 12:40 PM, James Y Knight <foom <at> fuhm.net> wrote:
>> >
>> > On Sep 27, 2009, at 3:18 PM, Peter Moody wrote:
>> >
>> >> administrators) would use it, but it's doable. what you're claiming is
>> >> that my use case is invalid.
>> >>
>> >> that's what I claim is broken.
>> >
>> > He's claiming your solution to address your use case is confusing, not that
>> > the use case is invalid.
>>
>> this isn't actually true.
>>
>> Steven D'Aprano wrote:
> [...]
>
> That's Steven, your original sentence was about me.

my original sentence was about everyone who was arguing that the
current implementation was designed for an invalid use-case. it was in
reply to your email, but it was to everyone (you, Nick, rdm, Steven,
etc)

>> > 1) if strict=False, mask off the bits described by the netmask when creating
>> > an IPNetwork, such that the host bits are always 0.
>>
>> I haven't heard anyone suggest auto-masking bits, but otherwise that
>> would be strict=True.
>
> I would expect strict=True to raise an error if the lower bits are non-zero, not
> to silently erase them. strict=False would be the option that silently erases
> lower bits.
> (that's why it's named `strict`, after all :-))
>
>> > 2) add a single new function:
>> >
>> > def parse_net_and_addr(s):
>> > ?return (IPNetwork(s), IPAddress(s.split('/')[0]))
>>
>> I've only heard talk of new classes and new methods, not new
>> constructor functions.
>
> Well, "method" in that context meant "class method" since the results aren't
> dependent on a particular instance. Of course, both a class method or a
> module-level function would be fine.

so this is not the response I got when I asked what was required
before. Would adding this constructor function satisfy your concerns
(given sensible strict settings in the constructor, etc)?

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From solipsis at pitrou.net  Sun Sep 27 22:49:15 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 20:49:15 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
Message-ID: <loom.20090927T224041-529@post.gmane.org>

Peter Moody <peter <at> hda3.com> writes:
> 
> >> > def parse_net_and_addr(s):
> >> > ?return (IPNetwork(s), IPAddress(s.split('/')[0]))
> >>
> >> I've only heard talk of new classes and new methods, not new
> >> constructor functions.
> >
> > Well, "method" in that context meant "class method" since the results aren't
> > dependent on a particular instance. Of course, both a class method or a
> > module-level function would be fine.
> 
> so this is not the response I got when I asked what was required
> before. Would adding this constructor function satisfy your concerns
> (given sensible strict settings in the constructor, etc)?

Assuming the Network type loses the notion of a specific host (or host address,
or `ip`) attached to it, yes.

Or to put it more clearly:

>>> Network('192.168.0.1/24', strict=False)
Network('192.168.0.0/24')
>>> Network('192.168.0.1/24', strict=False) == Network('192.168.0.0/24')
True

Regards

Antoine.



From peter at hda3.com  Sun Sep 27 22:59:07 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 13:59:07 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090927T224041-529@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
Message-ID: <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>

On Sun, Sep 27, 2009 at 1:49 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Peter Moody <peter <at> hda3.com> writes:
>>
>> >> > def parse_net_and_addr(s):
>> >> > ?return (IPNetwork(s), IPAddress(s.split('/')[0]))
>> >>
>> >> I've only heard talk of new classes and new methods, not new
>> >> constructor functions.
>> >
>> > Well, "method" in that context meant "class method" since the results aren't
>> > dependent on a particular instance. Of course, both a class method or a
>> > module-level function would be fine.
>>
>> so this is not the response I got when I asked what was required
>> before. Would adding this constructor function satisfy your concerns
>> (given sensible strict settings in the constructor, etc)?
>
> Assuming the Network type loses the notion of a specific host (or host address,
> or `ip`) attached to it, yes.

this is "less useful (strictly removing functionality)" and is an
example of what I explicitly said I was not going to do with ipaddr.

Cheers,
/peter

> Or to put it more clearly:
>
>>>> Network('192.168.0.1/24', strict=False)
> Network('192.168.0.0/24')
>>>> Network('192.168.0.1/24', strict=False) == Network('192.168.0.0/24')
> True
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From steven.bethard at gmail.com  Sun Sep 27 22:59:32 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 27 Sep 2009 13:59:32 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
Message-ID: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>

Below is a new PEP based on discussions from the stdlib-sig, which
proposes to add the argparse module to the standard library in Python
2.7 and 3.2.

Looking forward to your feedback!

Steve

http://www.python.org/dev/peps/pep-0389/
----------------------------------------------------------------------
PEP: 389
Title: argparse - new command line parsing module
Version: $Revision: 75097 $
Last-Modified: $Date: 2009-09-27 12:42:40 -0700 (Sun, 27 Sep 2009) $
Author: Steven Bethard <steven.bethard at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 25-Sep-2009
Python-Version: 2.7 and 3.2
Post-History:


Abstract
========
This PEP proposes inclusion of the argparse [1]_ module in the Python
standard library in Python 2.7 and 3.2.


Motivation
==========
The argparse module is a command line parsing library which provides
more functionality than the existing command line parsing modules in
the standard library, getopt [2]_ and optparse [3]_. It includes
support for positional arguments (not just options), subcommands,
required options, options syntaxes like "/f" and "+rgb", zero-or-more
and one-or-more style arguments, and many other features the other
two lack.

The argparse module is also already a popular third-party replacement
for these modules. It is used in projects like IPython (the Scipy
Python shell) [4]_, is included in Debian testing and unstable [5]_,
and since 2007 has had various requests for its inclusion in the
standard library [6]_ [7]_ [8]_. This popularity suggests it may be
a valuable addition to the Python libraries.


Why aren't getopt and optparse enough?
======================================
One argument against adding argparse is that thare are "already two
different option parsing modules in the standard library" [9]_. The
following is a list of features provided by argparse but not present
in getopt or optparse:

* While it is true there are two *option* parsing libraries, there
  are no full command line parsing libraries -- both getopt and
  optparse support only options and have no support for positional
  arguments. The argparse module handles both, and as a result, is
  able to generate better help messages, avoiding redundancies like
  the ``usage=`` string usually required by optparse.

* The argparse module values practicality over purity. Thus, argparse
  allows required options and customization of which characters are
  used to identify options, while optparse explicitly states "the
  phrase 'required option' is self-contradictory" and that the option
  syntaxes ``-pf``, ``-file``, ``+f``, ``+rgb``, ``/f`` and ``/file``
  "are not supported by optparse, and they never will be".

* The argparse module allows options to accept a variable number of
  arguments using ``nargs='?'``, ``nargs='*'`` or ``nargs='+'``. The
  optparse module provides an untested recipe for some part of this
  functionality [10]_ but admits that "things get hairy when you want
  an option to take a variable number of arguments."

* The argparse module supports subcommands, where a main command
  line parser dispatches to other command line parsers depending on
  the command line arguments. This is a common pattern in command
  line interfaces, e.g. ``svn co`` and ``svn up``.


Why isn't the functionality just being added to optparse?
=========================================================
Clearly all the above features offer improvements over what is
available through optparse. A reasonable question then is why these
features are not simply provided as patches to optparse, instead of
introducing an entirely new module. In fact, the original development
of argparse intended to do just that, but because of various fairly
constraining design decisions of optparse, this wasn't really
possible. Some of the problems included:

* The optparse module exposes the internals of its parsing algorithm.
  In particular, ``parser.largs`` and ``parser.rargs`` are guaranteed
  to be available to callbacks [11]_. This makes it extremely
  difficult to improve the parsing algorithm as was necessary in
  argparse for proper handling of positional arguments and variable
  length arguments. For example, ``nargs='+'`` in argparse is matched
  using regular expressions and thus has no notion of things like
  ``parser.largs``.

* The optparse extension APIs are extremely complex. For example,
  just to use a simple custom string-to-object conversion function,
  you have to subclass ``Option``, hack class attributes, and then
  specify your custom option type to the parser, like this::

    class MyOption(Option):
        TYPES = Option.TYPES + ("mytype",)
        TYPE_CHECKER = copy(Option.TYPE_CHECKER)
        TYPE_CHECKER["mytype"] = check_mytype
    parser = optparse.OptionParser(option_class=MyOption)
    parser.add_option("-m", type="mytype")

  For comparison, argparse simply allows conversion functions to be
  used as ``type=`` arguments directly, e.g.::

    parser = argparse.ArgumentParser()
    parser.add_option("-m", type=check_mytype)

  But given the baroque customization APIs of optparse, it is unclear
  how such a feature should interact with those APIs, and it is
  quite possible that introducing the simple argparse API would break
  existing custom Option code.

* Both optparse and argparse parse command line arguments and assign
  them as attributes to an object returned by ``parse_args``.
  However, the optparse module guarantees that the ``take_action``
  method of custom actions will always be passed a ``values`` object
  which provides an ``ensure_value`` method [12]_, while the argparse
  module allows attributes to be assigned to any object, e.g.::

	foo_object = ...
	parser.parse_args(namespace=foo_object)
	foo_object.some_attribute_parsed_from_command_line

  Modifying optparse to allow any object to be passed in would be
  difficult because simply passing the ``foo_object`` around instead
  of a ``Values`` instance will break existing custom actions that
  depend on the ``ensure_value`` method.

Because of issues like these, which made it unreasonably difficult
for argparse to stay compatible with the optparse APIs, argparse was
developed as an independent module. Given these issues, merging all
the argparse features into optparse with no backwards
incompatibilities seems unlikely.


Deprecation of getopt and optparse
==================================
There is still some debate over the best way (if at all) to encourage
users to move from getopt and optparse to their replacement,
argparse. The current recommendation of this PEP is the following
conservative deprecation strategy:

* Python 3.2, Python 2.7 and any later Python 2.X releases --
  PendingDeprecation warnings, which by default are not displayed,
  and documentation notes directing users of getopt and optparse to
  argparse.

* Python 3.3 -- Same as above

* Python 3.4 -- Deprecation warnings for getopt and optparse, which
  by default *are* displayed.

Though this is slower than the usual deprecation process, it seems
prudent to avoid producing any casually visible Deprecation warnings
until Python 3.X has had some additional time to attract developers.


Open Issues
===========
The argparse module supports Python from 2.3 up through 3.2 and as a
result relies on traditional ``%(foo)s`` style string formatting. It
has been suggested that it might be better to use the new style
``{foo}`` string formatting [13]_. This seems like a good idea, but
would break backwards compatibility for existing argparse-based
scripts unless we can come up with a way to reasonably support both
syntaxes.


References
==========
.. [1] argparse
   (http://code.google.com/p/argparse/)

.. [2] getopt
   (http://docs.python.org/library/getopt.html)

.. [3] optparse
   (http://docs.python.org/library/optparse.html)

.. [4] argparse in IPython
   (http://mail.scipy.org/pipermail/ipython-dev/2009-April/005102.html)

.. [5] argparse in Debian
   (http://packages.debian.org/search?keywords=python-argparse)

.. [6] 2007-01-03 request for argparse in the standard library
   (http://mail.python.org/pipermail/python-list/2007-January/592646.html)

.. [7] 2009-06-09 request for argparse in the standard library
   (http://bugs.python.org/issue6247)

.. [8] 2009-09-10 request for argparse in the standard library
   (http://mail.python.org/pipermail/stdlib-sig/2009-September/000342.html)

.. [9] Fredrik Lundh response to [6]_
   (http://mail.python.org/pipermail/python-list/2007-January/592675.html)

.. [10] optparse variable args
   (http://docs.python.org/library/optparse.html#callback-example-6-variable-arguments)

.. [11] parser.largs and parser.rargs
   (http://docs.python.org/library/optparse.html#how-callbacks-are-called)

.. [12] take_action values argument
   (http://docs.python.org/library/optparse.html#adding-new-actions)

.. [13] use {}-formatting instead of %-formatting
   (http://bugs.python.org/msg89279)


Copyright
=========
This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From ncoghlan at gmail.com  Sun Sep 27 22:41:17 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Sep 2009 06:41:17 +1000
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <h9njfi$9f4$2@ger.gmane.org>
References: <4ABF2047.7050501@gmail.com> <h9njfi$9f4$2@ger.gmane.org>
Message-ID: <4ABFCDED.8080105@gmail.com>

Antoine Pitrou wrote:
> This deserves its own PEP IMO, although I'm not sure it would be accepted 
> (ISTR the idea of a detailed IO exception hierarchy was already refused 
> in the past).

Not as such - a big exception hierarchy rewrite was rejected, but
nothing specifically limited to the IO exceptions.

Michael's response cut to the heart of the issue though - a richer IO
exception hierarchy can make life interesting for compatibility purposes
(especially when creating "file-like" interfaces to non-file objects).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Sun Sep 27 23:24:52 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 27 Sep 2009 14:24:52 -0700
Subject: [Python-Dev] Fuzziness in io module specs - PEP update
	proposition V2
In-Reply-To: <h9nihf$9f4$1@ger.gmane.org>
References: <4AB3DCC9.5050209@gmail.com> <4AB58159.3040303@canterbury.ac.nz> 
	<4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> 
	<4AB5FA81.5060408@gmail.com>
	<eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com> 
	<4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com> 
	<4ABF1B3B.4080106@gmail.com> <h9nihf$9f4$1@ger.gmane.org>
Message-ID: <ca471dc20909271424l57315d4ft18d4bbe7da8bdd4@mail.gmail.com>

On Sun, Sep 27, 2009 at 4:33 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> So here is the proposed semantic, which matches established conventions:
>>
>> *IOBase.truncate(n: int = None) -> int*
> [...]
>
> I still don't think there is a sufficient benefit in breaking
> compatibility. If you want the file pointer to remain the same, you can
> save it first and restore it afterwards manually.

What compatibility, though? f.truncate() behaves different in 2.x than
in 3.x, and in 2.x it seems to match the POSIX semantics (i.e. the
seek position is unchanged even though the file size is). Perhaps the
changed semantics was an oversight or a mistake?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fuzzyman at voidspace.org.uk  Sun Sep 27 23:32:06 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 27 Sep 2009 22:32:06 +0100
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4ABFCDED.8080105@gmail.com>
References: <4ABF2047.7050501@gmail.com> <h9njfi$9f4$2@ger.gmane.org>
	<4ABFCDED.8080105@gmail.com>
Message-ID: <4ABFD9D6.9070602@voidspace.org.uk>

Nick Coghlan wrote:
> Antoine Pitrou wrote:
>   
>> This deserves its own PEP IMO, although I'm not sure it would be accepted 
>> (ISTR the idea of a detailed IO exception hierarchy was already refused 
>> in the past).
>>     
>
> Not as such - a big exception hierarchy rewrite was rejected, but
> nothing specifically limited to the IO exceptions.
>
> Michael's response cut to the heart of the issue though - a richer IO
> exception hierarchy can make life interesting for compatibility purposes
> (especially when creating "file-like" interfaces to non-file objects).
>   

Some of the error messages are truly awful though as things stand, 
especially for someone new to Python. Try to read from a file handle 
opened in read mode for example: IOError: [Errno 9] Bad file descriptor

Michael

> Cheers,
> Nick.
>
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From brett at python.org  Sun Sep 27 23:57:34 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 27 Sep 2009 14:57:34 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
Message-ID: <bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>

I am going to state upfront that I am +1 for this and I encouraged
Steven to submit this PEP on the stdlib-SIG. I still remember watching
Steven's lightning talk at PyCon 2009 on argparse and being impressed
by it (along with the rest of the audience which was obviously
impressed as well).

I think argprase is a good improvement over what we have now (getopt
and optparse), it's stable, considered best-of-breed by the community
for a while (as shown as how many times argparse has been suggestion
for inclusion), and Steven is already a core committer so is already
set to maintain the code. That covers the usual checklist we have for
even looking at a PEP to add a module to the standard library.


On Sun, Sep 27, 2009 at 13:59, Steven Bethard <steven.bethard at gmail.com> wrote:
[SNIP]
> Deprecation of getopt and optparse
> ==================================
> There is still some debate over the best way (if at all) to encourage
> users to move from getopt and optparse to their replacement,
> argparse. The current recommendation of this PEP is the following
> conservative deprecation strategy:
>
> * Python 3.2, Python 2.7 and any later Python 2.X releases --
> ?PendingDeprecation warnings, which by default are not displayed,
> ?and documentation notes directing users of getopt and optparse to
> ?argparse.
>
> * Python 3.3 -- Same as above
>
> * Python 3.4 -- Deprecation warnings for getopt and optparse, which
> ?by default *are* displayed.
>
> Though this is slower than the usual deprecation process, it seems
> prudent to avoid producing any casually visible Deprecation warnings
> until Python 3.X has had some additional time to attract developers.
>

Just to give people ideas of how long these deprecations would last,
if Python 3.2 is released in June 2010 and we continue on our 18 month
release schedule (and actually release that quickly which we typically
don't), then getopt/optparse would have a pending deprecation for 3
years (until June 2013) and a deprecation warning for 1.5 years (until
January 2015), so 4.5 years total once the deprecations began and six
years since Python 3.0 came out. And obviously the deprecation can be
extended if it turns out Python 3 pick up is now at a rate we expect
(but only if people who plan to port are having issues; this does not
concern those who plan to stay on Python 2 indefinitely and do not
care about Python 3).

And we can also document suggestions on how to transition off of
getopt/optparse like Steven has in the argparse code
(http://argparse.googlecode.com/svn/tags/r101/doc/argparse-vs-optparse.html#upgrading-optparse-code).

>
> Open Issues
> ===========
> The argparse module supports Python from 2.3 up through 3.2 and as a
> result relies on traditional ``%(foo)s`` style string formatting. It
> has been suggested that it might be better to use the new style
> ``{foo}`` string formatting [13]_. This seems like a good idea, but
> would break backwards compatibility for existing argparse-based
> scripts unless we can come up with a way to reasonably support both
> syntaxes.

I see two solutions to this. One is to auto-detect either format and
then do the right thing. Seems potentially messy, although getting the
regex right shouldn't be a problem.

The other option is to rename the module if it is added to the
standard library ala simplejson/json. That would alleviate any issues
with someone including their own copy of argparse with their
application using %()s string interpolation over {}. I don't have a
name to suggest at the moment (nor do I think we should start
discussing that now unless this is a chosen solution), but it would
deal with the problem.

Either way, with {} being the future and talk of someday dropping %, I
think we should make sure the newer syntax is at least supported, if
not the only way to do things.

-Brett

From fuzzyman at voidspace.org.uk  Mon Sep 28 00:00:21 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 27 Sep 2009 23:00:21 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>
Message-ID: <4ABFE075.2070309@voidspace.org.uk>

Brett Cannon wrote:
> I am going to state upfront that I am +1 for this and I encouraged
> Steven to submit this PEP on the stdlib-SIG. I still remember watching
> Steven's lightning talk at PyCon 2009 on argparse and being impressed
> by it (along with the rest of the audience which was obviously
> impressed as well).
>
> I think argprase is a good improvement over what we have now (getopt
> and optparse), it's stable, considered best-of-breed by the community
> for a while (as shown as how many times argparse has been suggestion
> for inclusion), and Steven is already a core committer so is already
> set to maintain the code. That covers the usual checklist we have for
> even looking at a PEP to add a module to the standard library.
>
>   

I've used argparse and really like it. I could also have done with the 
subcommand support in recent changes to unittest.

+1 for inclusion.

Michael


> On Sun, Sep 27, 2009 at 13:59, Steven Bethard <steven.bethard at gmail.com> wrote:
> [SNIP]
>   
>> Deprecation of getopt and optparse
>> ==================================
>> There is still some debate over the best way (if at all) to encourage
>> users to move from getopt and optparse to their replacement,
>> argparse. The current recommendation of this PEP is the following
>> conservative deprecation strategy:
>>
>> * Python 3.2, Python 2.7 and any later Python 2.X releases --
>>  PendingDeprecation warnings, which by default are not displayed,
>>  and documentation notes directing users of getopt and optparse to
>>  argparse.
>>
>> * Python 3.3 -- Same as above
>>
>> * Python 3.4 -- Deprecation warnings for getopt and optparse, which
>>  by default *are* displayed.
>>
>> Though this is slower than the usual deprecation process, it seems
>> prudent to avoid producing any casually visible Deprecation warnings
>> until Python 3.X has had some additional time to attract developers.
>>
>>     
>
> Just to give people ideas of how long these deprecations would last,
> if Python 3.2 is released in June 2010 and we continue on our 18 month
> release schedule (and actually release that quickly which we typically
> don't), then getopt/optparse would have a pending deprecation for 3
> years (until June 2013) and a deprecation warning for 1.5 years (until
> January 2015), so 4.5 years total once the deprecations began and six
> years since Python 3.0 came out. And obviously the deprecation can be
> extended if it turns out Python 3 pick up is now at a rate we expect
> (but only if people who plan to port are having issues; this does not
> concern those who plan to stay on Python 2 indefinitely and do not
> care about Python 3).
>
> And we can also document suggestions on how to transition off of
> getopt/optparse like Steven has in the argparse code
> (http://argparse.googlecode.com/svn/tags/r101/doc/argparse-vs-optparse.html#upgrading-optparse-code).
>
>   
>> Open Issues
>> ===========
>> The argparse module supports Python from 2.3 up through 3.2 and as a
>> result relies on traditional ``%(foo)s`` style string formatting. It
>> has been suggested that it might be better to use the new style
>> ``{foo}`` string formatting [13]_. This seems like a good idea, but
>> would break backwards compatibility for existing argparse-based
>> scripts unless we can come up with a way to reasonably support both
>> syntaxes.
>>     
>
> I see two solutions to this. One is to auto-detect either format and
> then do the right thing. Seems potentially messy, although getting the
> regex right shouldn't be a problem.
>
> The other option is to rename the module if it is added to the
> standard library ala simplejson/json. That would alleviate any issues
> with someone including their own copy of argparse with their
> application using %()s string interpolation over {}. I don't have a
> name to suggest at the moment (nor do I think we should start
> discussing that now unless this is a chosen solution), but it would
> deal with the problem.
>
> Either way, with {} being the future and talk of someday dropping %, I
> think we should make sure the newer syntax is at least supported, if
> not the only way to do things.
>
> -Brett
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ncoghlan at gmail.com  Sun Sep 27 22:34:51 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Sep 2009 06:34:51 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4ABF015E.5060002@gmail.com>	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<loom.20090927T185114-628@post.gmane.org>	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>	<loom.20090927T201052-343@post.gmane.org>	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
Message-ID: <4ABFCC6B.3020409@gmail.com>

Peter Moody wrote:
> Steven D'Aprano wrote:
> 
>   Could you explain what benefit there is for allowing the user to create
>   network objects that don't represent networks? Is there a use-case
>   where these networks-that-aren't-networks are something other than a
>   typo? Under what circumstances would I want to specify a network as
>   192.168.1.1/24 instead of 192.168.1.0/24?
> 
> that pretty flatly states that my use case is invalid.

Looks like Steven was asking you to explain your use case to him rather
than stating that it didn't exist. (although I will grant that the
network definition use case had been well covered, I will also grant
that these threads have been somewhat interminable and I can easily
understand someone missing those use case definitions - the example of
being able to define a network given only the network mask and the IP
address of the host running the current application should be mentioned
explicitly in the PEP)

>> 1) if strict=False, mask off the bits described by the netmask when creating
>> an IPNetwork, such that the host bits are always 0.
> 
> I haven't heard anyone suggest auto-masking bits, but otherwise that
> would be strict=True.

Actually, that's exactly what I suggested in this thread - that
IPNetwork should lose the "ip" property and that the definition of
"network equality" should be based on the network address not on the
host address that happened to be used when constructing the network object.

The part that is confusing with the current library implementation is
the fact that we can have two IPNetwork objects, supposedly describing
the same network, compare unequal because they happened to be defined
differently. It would be analagous to having "Decimal('3.0') !=
Decimal('3.00')".

>> 2) add a single new function:
>>
>> def parse_net_and_addr(s):
>>  return (IPNetwork(s), IPAddress(s.split('/')[0]))
> 
> I've only heard talk of new classes and new methods, not new
> constructor functions. In fact, when I asked for explicit
> clarification of what was required, a new constructor function "ala
> IPAddress and IPNetwork (so the current classes remain largely the
> same and the various constructors enforce certain restrictions)", I
> was told, no, a new object AddressWithMask was actually required. I've
> no problem with a constructor like this, but this is not what people
> have been asking for.

No, AddressWithMask was merely defined as a concept separate from an IP
Network to indicate how the current IPNetwork class was being overloaded
with two distinct concepts and hence exhibiting unnecessary behavioural
quirks.

The use case for "given a netmask and an arbitrary host on that network,
give me the appropriate IPNetwork object" has been well established by
this discussion (although still isn't particularly well described even
in the latest PEP update). This is what "strict=False" covers, and I'm
now happy that practicality means this is appropriate default behaviour
for the IPNetwork constructor.

The use case for "create an IPNetwork object and have it *remember*
which particular host address was used to define it" is still not
established at all.

It is this latter use case which is covered by the AddressWithMask
concept (which may be perfectly validly represented as a simple
IPNetwork/IPAddress tuple, which was also mentioned when the concern was
first raised).

The main concern here is to take the "AddressWithMask" behaviour *off*
the IPNetwork objects. It interferes with their interpretation as
descriptions of networks because they maintain a piece of irrelevant
state and pay attention to it in comparison operations.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Mon Sep 28 00:32:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 22:32:35 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review, and the inclusion process
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
Message-ID: <loom.20090928T000905-169@post.gmane.org>

Peter Moody <peter <at> hda3.com> writes:
> 
> this is "less useful (strictly removing functionality)" and is an
> example of what I explicitly said I was not going to do with ipaddr.

(please note the conditional wording here) Assuming that a significant number of
people agree that there is a design problem, if you don't want to make the
necessary changes, then I don't see a reason why ipaddr would enter the stdlib.
The functionality (IP address handling) hasn't really seen a huge demand.

On stdlib-sig recently, a number of people complained that our criteria for
including existing libraries in the stdlib should be higher (they complained
about the quality of some existing modules, including optparse, which by the way
prompted the current proposal to get argparse in the stdlib). I think this PEP
is a good moment to judge and decide how demanding or tolerant we (and
especially the complainers ;-)) want to be.

Regards

Antoine.



From solipsis at pitrou.net  Mon Sep 28 00:38:31 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 22:38:31 +0000 (UTC)
Subject: [Python-Dev] IO module precisions and exception hierarchy
References: <4ABF2047.7050501@gmail.com> <h9njfi$9f4$2@ger.gmane.org>
	<4ABFCDED.8080105@gmail.com>
Message-ID: <h9oph7$hkf$1@ger.gmane.org>

Le Mon, 28 Sep 2009 06:41:17 +1000, Nick Coghlan a ?crit?:
> 
> Not as such - a big exception hierarchy rewrite was rejected, but
> nothing specifically limited to the IO exceptions.
> 
> Michael's response cut to the heart of the issue though - a richer IO
> exception hierarchy can make life interesting for compatibility purposes
> (especially when creating "file-like" interfaces to non-file objects).

Well, not more interesting than currently where you need to replicate 
errno numbers if you want to make the errors precise enough, since an API 
consumer wanting to check specific error conditions will discriminate on 
errno.

If you don't want to go to that level of perfection, you just have to 
raise a plain IOError (or OSError :-)) without bothering about errno or 
subclasses; like you would do today.

Regards

Antoine.


From solipsis at pitrou.net  Mon Sep 28 00:44:22 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 27 Sep 2009 22:44:22 +0000 (UTC)
Subject: [Python-Dev] Fuzziness in io module specs - PEP
	update	proposition V2
References: <4AB3DCC9.5050209@gmail.com> <4AB58159.3040303@canterbury.ac.nz>
	<4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz>
	<4AB5FA81.5060408@gmail.com>
	<eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com>
	<4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com>
	<4ABF1B3B.4080106@gmail.com> <h9nihf$9f4$1@ger.gmane.org>
	<ca471dc20909271424l57315d4ft18d4bbe7da8bdd4@mail.gmail.com>
Message-ID: <h9ops5$hkf$2@ger.gmane.org>

Le Sun, 27 Sep 2009 14:24:52 -0700, Guido van Rossum a ?crit?:
[truncate()]
> 
> What compatibility, though?

Compatibility accross the 3.x line.

> f.truncate() behaves different in 2.x than
> in 3.x, and in 2.x it seems to match the POSIX semantics (i.e. the seek
> position is unchanged even though the file size is). Perhaps the changed
> semantics was an oversight or a mistake?

Perhaps it was, indeed. I don't know who made that decision in the first 
place.

Regards

Antoine.



From peter at hda3.com  Mon Sep 28 01:17:48 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 16:17:48 -0700
Subject: [Python-Dev] PEP 3144 review, and the inclusion process
In-Reply-To: <loom.20090928T000905-169@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
	<loom.20090928T000905-169@post.gmane.org>
Message-ID: <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com>

On Sun, Sep 27, 2009 at 3:32 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Peter Moody <peter <at> hda3.com> writes:
>>
>> this is "less useful (strictly removing functionality)" and is an
>> example of what I explicitly said I was not going to do with ipaddr.
>
> (please note the conditional wording here) Assuming that a significant number of
> people agree that there is a design problem, if you don't want to make the
> necessary changes, then I don't see a reason why ipaddr would enter the stdlib.

I've never said otherwise. In fact, from an email last night, "If what
the community requires is the library you've described, then ipaddr is
not that library." The changes *you* require make ipaddr significantly
less useful to me. I'm not prepared to make those changes in an
attempt seek acceptance to the stdlib, especially if the stdlib is in
such flux that I'll get to do this again in 18 months.

> The functionality (IP address handling) hasn't really seen a huge demand.

then no one would be the worse off if I don't make those suggested changes.

> On stdlib-sig recently, a number of people complained that our criteria for
> including existing libraries in the stdlib should be higher (they complained
> about the quality of some existing modules, including optparse, which by the way
> prompted the current proposal to get argparse in the stdlib). I think this PEP
> is a good moment to judge and decide how demanding or tolerant we (and
> especially the complainers ;-)) want to be.

Sounds like design by committee to satisfy the squeakiest wheel. I'm
trying, but I can't think of anything worse.

Cheers,
/peter

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From steve at pearwood.info  Mon Sep 28 02:13:38 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 28 Sep 2009 10:13:38 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909271053n174ee3dy9173ecf73e3dd39b@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<8517e9350909271053n174ee3dy9173ecf73e3dd39b@mail.gmail.com>
Message-ID: <200909281013.39069.steve@pearwood.info>

On Mon, 28 Sep 2009 03:53:27 am Peter Moody wrote:

> >> ? I *understand* what you're saying, I *understand* that
> >> 192.168.1.1/24 isn't a network,
> >
> > But you still want to treat it as one.
> >
> > Could you explain what benefit there is for allowing the user to
> > create network objects that don't represent networks? Is there a
> > use-case where these networks-that-aren't-networks are something
> > other than a typo? Under what circumstances would I want to specify
> > a network as 192.168.1.1/24 instead of 192.168.1.0/24?
>
> this is pretty ridiculous. if in the last two months you haven't seen
> a single use case, then you haven't been paying attention.

No, I haven't read every single email in excruciating detail in this 
extremely long thread. I'd bet most other people haven't.

I'm trying to understand why you want something which, in *my* 
ignorance, seems patently ridiculous to me: allowing Network objects 
which aren't Networks, particularly since in private email to me you 
denied that there was a use-case for the requested (Address object + 
netmask). But it seems to me that this is exactly what you have in the 
current implementation, except you call it a Network object, and have 
rejected the suggestion that the non-network bits of the address should 
be zeroed.

That is, you've rejected the idea of having:

>>> IPv4Network(192.168.1.1/24)
IPv4Network(192.168.1.0/24)

as reducing functionality, presumably because the address 192.168.1.1 is 
lost. Sounds just like an Address+netmask to me, with added 
network-like behaviour.

Some people have requested a way of explicitly calculating a network 
from an Address and netmask, and you've been hostile to the idea. But 
it seems to me that your API does exactly that.

I'm not the only person who thinks your API conflates two different 
concepts into a single class, and I'm trying to see your side of the 
argument. But your hostile attitude isn't making it easy.



-- 
Steven D'Aprano

From fperez.net at gmail.com  Mon Sep 28 02:31:00 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 28 Sep 2009 00:31:00 +0000 (UTC)
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>
Message-ID: <h9p044$uki$1@ger.gmane.org>

On Sun, 27 Sep 2009 14:57:34 -0700, Brett Cannon wrote:

> I am going to state upfront that I am +1 for this and I encouraged
> Steven to submit this PEP on the stdlib-SIG. I still remember watching
> Steven's lightning talk at PyCon 2009 on argparse and being impressed by
> it (along with the rest of the audience which was obviously impressed as
> well).
> 
> I think argprase is a good improvement over what we have now (getopt and
> optparse), it's stable, considered best-of-breed by the community for a
> while (as shown as how many times argparse has been suggestion for
> inclusion), and Steven is already a core committer so is already set to
> maintain the code. That covers the usual checklist we have for even
> looking at a PEP to add a module to the standard library.

FWIW, +1 from IPython: we ship a private copy of argparse as well (we use 
it in several places but we want the basic ipython to be installable just 
with the stdlib). Steven was gracious enough to relicense it as BSD at our 
request:

http://mail.scipy.org/pipermail/ipython-dev/2009-September/005537.html

so we could ship this copy without any GPL incompatibilities, but we'd 
much rather rely on stdlib components.

Best,

f


From peter at hda3.com  Mon Sep 28 02:31:56 2009
From: peter at hda3.com (Peter Moody)
Date: Sun, 27 Sep 2009 17:31:56 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909281013.39069.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<8517e9350909271053n174ee3dy9173ecf73e3dd39b@mail.gmail.com>
	<200909281013.39069.steve@pearwood.info>
Message-ID: <8517e9350909271731ra859ba5yda1d5f30e13a8b26@mail.gmail.com>

On Sun, Sep 27, 2009 at 5:13 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Mon, 28 Sep 2009 03:53:27 am Peter Moody wrote:
>
>> >> ? I *understand* what you're saying, I *understand* that
>> >> 192.168.1.1/24 isn't a network,
>> >
>> > But you still want to treat it as one.
>> >
>> > Could you explain what benefit there is for allowing the user to
>> > create network objects that don't represent networks? Is there a
>> > use-case where these networks-that-aren't-networks are something
>> > other than a typo? Under what circumstances would I want to specify
>> > a network as 192.168.1.1/24 instead of 192.168.1.0/24?
>>
>> this is pretty ridiculous. if in the last two months you haven't seen
>> a single use case, then you haven't been paying attention.
>
> No, I haven't read every single email in excruciating detail in this
> extremely long thread. I'd bet most other people haven't.
>
> I'm trying to understand why you want something which, in *my*
> ignorance, seems patently ridiculous to me: allowing Network objects
> which aren't Networks, particularly since in private email to me you
> denied that there was a use-case for the requested (Address object +
> netmask).

no, it seems as though you're either misrepresenting my position or
you misunderstood what I said. I said that there wasn't a use-case for
explicitly pulling that functionality out into a a new class, and
having 6 classes.

> But it seems to me that this is exactly what you have in the
> current implementation, except you call it a Network object, and have
> rejected the suggestion that the non-network bits of the address should
> be zeroed.

I have not rejected this. I have rejected the suggestion that the
current IPv?Network classes do this by default.

> That is, you've rejected the idea of having:
>
>>>> IPv4Network(192.168.1.1/24)
> IPv4Network(192.168.1.0/24)

yes, I and everyone have rejected that idea. this should either raise
an  error, or do what it does now, that is, return
IPv4Network('192.168.1.1/24')

> as reducing functionality, presumably because the address 192.168.1.1 is
> lost.

good guess.

> Sounds just like an Address+netmask to me, with added
> network-like behaviour.
>
> Some people have requested a way of explicitly calculating a network
> from an Address and netmask, and you've been hostile to the idea. But
> it seems to me that your API does exactly that.

You mean the suggestion to include an IPv?Network object attribute
with the IPv?Address objects? I thought that was dropped long ago.

> I'm not the only person who thinks your API conflates two different
> concepts into a single class, and I'm trying to see your side of the
> argument. But your hostile attitude isn't making it easy.

apologies if you find me hostile.  I'm not sure how you'd expect me to
be if, after 2 months 200+ messages, both on list and off, you were
unable to recall a single use-case.

> --
> Steven D'Aprano
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From solipsis at pitrou.net  Mon Sep 28 02:32:49 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Sep 2009 00:32:49 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review, and the inclusion process
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
	<loom.20090928T000905-169@post.gmane.org>
	<8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com>
Message-ID: <loom.20090928T022351-635@post.gmane.org>

Peter Moody <peter <at> hda3.com> writes:
> 
> I've never said otherwise. In fact, from an email last night, "If what
> the community requires is the library you've described, then ipaddr is
> not that library." The changes *you* require make ipaddr significantly
> less useful to me. I'm not prepared to make those changes in an
> attempt seek acceptance to the stdlib, especially if the stdlib is in
> such flux that I'll get to do this again in 18 months.

Well, then I'm not sure why we have a PEP at all.
If you don't want any significant changes and if you consider it to be *your*
library, ipaddr can remain a third-party package that interested people can
easily install (no pun ;-)) since AFAIK it's pure Python. It will also make
maintenance easier for you, while freeing us (core developers) from having to
bother about it in our daily development tasks.

At least that's what I would advocate right now - not sure about what others
think.

Regards

Antoine.



From bioinformed at gmail.com  Mon Sep 28 02:43:16 2009
From: bioinformed at gmail.com (Kevin Jacobs <jacobs@bioinformed.com>)
Date: Sun, 27 Sep 2009 20:43:16 -0400
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <h9p044$uki$1@ger.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>
	<h9p044$uki$1@ger.gmane.org>
Message-ID: <2e1434c10909271743k68b51a36m8b1717a89d48bcf@mail.gmail.com>

On Sun, Sep 27, 2009 at 8:31 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> On Sun, 27 Sep 2009 14:57:34 -0700, Brett Cannon wrote:
>
> > I am going to state upfront that I am +1 for this and I encouraged
> > Steven to submit this PEP on the stdlib-SIG. I still remember watching
> > Steven's lightning talk at PyCon 2009 on argparse and being impressed by
> > it (along with the rest of the audience which was obviously impressed as
> > well).
> >
> > I think argprase is a good improvement over what we have now (getopt and
> > optparse), it's stable, considered best-of-breed by the community for a
> > while (as shown as how many times argparse has been suggestion for
> > inclusion), and Steven is already a core committer so is already set to
> > maintain the code. That covers the usual checklist we have for even
> > looking at a PEP to add a module to the standard library.
>
> FWIW, +1 from IPython: we ship a private copy of argparse as well (we use
> it in several places but we want the basic ipython to be installable just
> with the stdlib). Steven was gracious enough to relicense it as BSD at our
> request:
>

+1

Coincidentally I was converting several packages to argparse as I saw the
python-dev mail come in.  A great deal of hairy code needed to jury-rig
sub-commands and other "advanced processing" with optparse is simply
disappearing.  I like the idea that stdlib can evolve to encompass best of
breed solutions and that effort is being expended to improve existing
functionality even if it means increasing the diversity of APIs.

~Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090927/62b3a2f2/attachment.htm>

From solipsis at pitrou.net  Mon Sep 28 02:51:33 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Sep 2009 00:51:33 +0000 (UTC)
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
Message-ID: <loom.20090928T024853-85@post.gmane.org>


Hello,

I am neutral on the idea of adding argparse. However, I'm -1 on deprecating
optparse. It is very widely used (tons of scripts use it), and ok for many uses;
deprecating it is totally unhelpful and gratuitous.

Regards

Antoine.



From greg.ewing at canterbury.ac.nz  Mon Sep 28 03:01:21 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 28 Sep 2009 14:01:21 +1300
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4ABFCC6B.3020409@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4ABF015E.5060002@gmail.com>
	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<4ABFCC6B.3020409@gmail.com>
Message-ID: <4AC00AE1.7080009@canterbury.ac.nz>

Nick Coghlan wrote:

> The use case for "given a netmask and an arbitrary host on that network,
> give me the appropriate IPNetwork object" has been well established by
> this discussion (although still isn't particularly well described even
> in the latest PEP update). This is what "strict=False" covers,

I think I'd rather have a separate constructor function
or method for this, rather than a somewhat cryptically
named boolean parameter.

-- 
Greg

From rdmurray at bitdance.com  Mon Sep 28 03:01:35 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 27 Sep 2009 21:01:35 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909271916480.18193@kimball.webabinitio.net>

On Sun, 27 Sep 2009 at 13:59, Peter Moody wrote:
> On Sun, Sep 27, 2009 at 1:49 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Peter Moody <peter <at> hda3.com> writes:
>>>
>>>>>> def parse_net_and_addr(s):
>>>>>> ?return (IPNetwork(s), IPAddress(s.split('/')[0]))
>>>>>
>>>>> I've only heard talk of new classes and new methods, not new
>>>>> constructor functions.
>>>>
>>>> Well, "method" in that context meant "class method" since the results aren't
>>>> dependent on a particular instance. Of course, both a class method or a
>>>> module-level function would be fine.
>>>
>>> so this is not the response I got when I asked what was required
>>> before. Would adding this constructor function satisfy your concerns
>>> (given sensible strict settings in the constructor, etc)?
>>
>> Assuming the Network type loses the notion of a specific host (or host address,
>> or `ip`) attached to it, yes.
>
> this is "less useful (strictly removing functionality)" and is an
> example of what I explicitly said I was not going to do with ipaddr.

In that case, I vote -1 on the inclusion of ipaddr in the stdlib.

--David

From greg.ewing at canterbury.ac.nz  Mon Sep 28 03:11:06 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 28 Sep 2009 14:11:06 +1300
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4ABFD9D6.9070602@voidspace.org.uk>
References: <4ABF2047.7050501@gmail.com> <h9njfi$9f4$2@ger.gmane.org>
	<4ABFCDED.8080105@gmail.com> <4ABFD9D6.9070602@voidspace.org.uk>
Message-ID: <4AC00D2A.6020504@canterbury.ac.nz>

Michael Foord wrote:

> Some of the error messages are truly awful though as things stand, 
> especially for someone new to Python. Try to read from a file handle 
> opened in read mode for example: IOError: [Errno 9] Bad file descriptor

Subdividing the IOError exception won't help with
that, because all you have to go on when deciding
which exception to raise is the error code returned
by the OS. If the same error code results from a
bunch of different things, there's not much Python
can do to sort them out.

The error messages produced for the various error
codes could perhaps be improved, but that's an
orthogonal issue.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Mon Sep 28 03:26:17 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 28 Sep 2009 14:26:17 +1300
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
Message-ID: <4AC010B9.4050003@canterbury.ac.nz>

Peter Moody wrote:
> On Sun, Sep 27, 2009 at 1:49 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
 >
>> Assuming the Network type loses the notion of a specific host (or host address,
>> or `ip`) attached to it, yes.
> 
> this is "less useful (strictly removing functionality)" and is an
> example of what I explicitly said I was not going to do with ipaddr.

Would you be kind enough to explain exactly what use
case you have for retaining this information?

Apologies if you've done so before -- I've been
trying to follow this discussion, but that point
doesn't seem to have come through clearly.

-- 
Greg

From doug.hellmann at gmail.com  Mon Sep 28 03:22:20 2009
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Sun, 27 Sep 2009 21:22:20 -0400
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4ABFE075.2070309@voidspace.org.uk>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>
	<4ABFE075.2070309@voidspace.org.uk>
Message-ID: <F5B27C82-50C3-4838-B507-53DF82C35D4C@gmail.com>


On Sep 27, 2009, at 6:00 PM, Michael Foord wrote:

> Brett Cannon wrote:
>> I am going to state upfront that I am +1 for this and I encouraged
>> Steven to submit this PEP on the stdlib-SIG. I still remember  
>> watching
>> Steven's lightning talk at PyCon 2009 on argparse and being impressed
>> by it (along with the rest of the audience which was obviously
>> impressed as well).
>>
>> I think argprase is a good improvement over what we have now (getopt
>> and optparse), it's stable, considered best-of-breed by the community
>> for a while (as shown as how many times argparse has been suggestion
>> for inclusion), and Steven is already a core committer so is already
>> set to maintain the code. That covers the usual checklist we have for
>> even looking at a PEP to add a module to the standard library.
>>
>>
>
> I've used argparse and really like it. I could also have done with  
> the subcommand support in recent changes to unittest.
>
> +1 for inclusion.

+1 as well.  I wish I had been able to use this library for a recent  
project using sub-commands.

Doug


From drobinow at gmail.com  Mon Sep 28 04:01:09 2009
From: drobinow at gmail.com (David Robinow)
Date: Sun, 27 Sep 2009 22:01:09 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC010B9.4050003@canterbury.ac.nz>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
	<4AC010B9.4050003@canterbury.ac.nz>
Message-ID: <4eb0089f0909271901u3d61319bi1372ab1b6f3e7d37@mail.gmail.com>

On Sun, Sep 27, 2009 at 9:26 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Would you be kind enough to explain exactly what use
> case you have for retaining this information?
>
> Apologies if you've done so before -- I've been
> trying to follow this discussion, but that point
> doesn't seem to have come through clearly.
Please don't apologize. It shouldn't matter how many times it's been
mentioned on the mailing list.
 The PEP process is intended to summarize the discussion. A PEP should
pretty much stand alone, with specific external links when necessary.
 The use case should be added to the PEP.

From digitalxero at gmail.com  Mon Sep 28 02:12:52 2009
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Sun, 27 Sep 2009 18:12:52 -0600
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4ABFCC6B.3020409@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<loom.20090927T185114-628@post.gmane.org>
	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<4ABFCC6B.3020409@gmail.com>
Message-ID: <e9764b730909271712l2bb660b7l895c2a24a4f10e6b@mail.gmail.com>

Looking though the tests you have setup for ipaddr it is clear that
you want the following to be True

ip1 = ipaddr.IPv4Network('1.1.1.0/24')
ip2 = ipaddr.IPv4Network('1.1.1.1/24')

ip1 == ip2

based on this test
self.assertEquals(ip1.compare_networks(ip2), 0)

but your ><= operators all compare against _ip instead of network. I
submitted a patch for review @ http://codereview.appspot.com/124057

From steven.bethard at gmail.com  Mon Sep 28 04:55:46 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 27 Sep 2009 19:55:46 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <loom.20090928T024853-85@post.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
Message-ID: <d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>

On Sun, Sep 27, 2009 at 5:51 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> I am neutral on the idea of adding argparse. However, I'm -1 on deprecating
> optparse. It is very widely used (tons of scripts use it), and ok for many uses;
> deprecating it is totally unhelpful and gratuitous.

Could you elaborate?  If you are worried about Python 2.X scripts,
note from the PEP that the only things that will ever show up in
Python 2.X are:

  PendingDeprecation warnings, which by default are not displayed,
  and documentation notes directing users of getopt and optparse to
  argparse.

I think these messages are useful for folks using 2.X who some day
would like to migrate to 3.X, and at the same time should have zero
effect on existing scripts.

If you think getopt and optparse should stick around in 3.X, why is
that? If you think there are things that getopt and optparse do better
than argparse, could you please give some examples?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From steven.bethard at gmail.com  Mon Sep 28 05:18:17 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 27 Sep 2009 20:18:17 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
Message-ID: <d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>

On Sun, Sep 27, 2009 at 8:08 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/9/27 Steven Bethard <steven.bethard at gmail.com>:
>> If you think getopt and optparse should stick around in 3.X, why is
>> that? If you think there are things that getopt and optparse do better
>> than argparse, could you please give some examples?
>
> Transitioning to Python 3 is already a pain. bytes/str/unicode things
> are going to be enough by themselves to drive people nuts. We don't
> want to be forcing them to change APIs if optparse is already working
> just fine for them. The job is the stdlib is not to force people to
> use the "best" or "right" tools. Several years in the future I would
> be more supportive of depreacting optparse, but more ways in which 2.x
> and 3.x grow farther apart are not going to help jumping the great
> version divide.

The first release where any real deprecation message would show up is
Python 3.4, more than 3 years away. If you think 3 years isn't long
enough for people to be over the Python 3 transition, let's stick in
another version in there and make it 4.5 years.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From steven.bethard at gmail.com  Mon Sep 28 05:46:03 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 27 Sep 2009 20:46:03 -0700
Subject: [Python-Dev]  PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
	<1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>
	<d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
Message-ID: <d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>

On Sun, Sep 27, 2009 at 8:41 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/9/27 Steven Bethard <steven.bethard at gmail.com>:
>> The first release where any real deprecation message would show up is
>> Python 3.4, more than 3 years away. If you think 3 years isn't long
>> enough for people to be over the Python 3 transition, let's stick in
>> another version in there and make it 4.5 years.
>
> So, why even bother deprecating it if nobody is going to see the warnings?

I feel like I'm repeating the PEP, but here it is again anyway. There
will be messages in the docs and pending deprecation warnings (which
don't show up by default but can be requested) starting in Python 2.7
and 3.2. Regular deprecation warnings wouldn't show up until Python
3.4, 3 years away. This compromise was intended exactly to address the
issue you brought up about people getting over the Python 3
transition.

Steve
--
Where did you get that preposterous hypothesis?
Did Steve tell you that?
? ? ? ?--- The Hiphopopotamus

From martin at v.loewis.de  Mon Sep 28 05:57:04 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 28 Sep 2009 05:57:04 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <200909280330.30643.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
Message-ID: <4AC03410.4010104@v.loewis.de>

>> Finally, to Stephen's point about seeing the other side of the
>> argument, I wrote this offlist a week ago:
>>
>>   I *understand* what you're saying, I *understand* that
>> 192.168.1.1/24 isn't a network, 
> 
> But you still want to treat it as one.
> 
> Could you explain what benefit there is for allowing the user to create 
> network objects that don't represent networks? Is there a use-case 
> where these networks-that-aren't-networks are something other than a 
> typo? Under what circumstances would I want to specify a network as 
> 192.168.1.1/24 instead of 192.168.1.0/24?
> 

It's fairly obvious to me why the library should support 192.168.1.1/24
as an input, and return a network.

End-users are likely going to enter such things (e.g. 82.94.164.162/29),
as they will know one IP address in the network (namely, of one machine
that they are looking at), and they will know the prefix length (more
often, how large the network is - 8 or 16 or 32). So very clearly,
end users should not be required to make the computation in their heads.

So Python code has to make the computation, and it seems most natural
that the IP library is the piece of code that is able to compute a
network out of that input.

Does that answer your question?

Regards,
Martin

From ubershmekel at gmail.com  Mon Sep 28 05:59:45 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Mon, 28 Sep 2009 06:59:45 +0300
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
	<1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>
	<d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
	<d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>
Message-ID: <9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com>

-1 for deprecating getopt. getopt is super-simple and especially useful for
c programmers learning python.

+1 for argparse.+1 for eventual deprecation of optparse - optparse and
argparse have a very similar syntax and having both is just
confusing. tsboapooowtdi


On Mon, Sep 28, 2009 at 6:46 AM, Steven Bethard <steven.bethard at gmail.com>wrote:

> On Sun, Sep 27, 2009 at 8:41 PM, Benjamin Peterson <benjamin at python.org>
> wrote:
> > 2009/9/27 Steven Bethard <steven.bethard at gmail.com>:
> >> The first release where any real deprecation message would show up is
> >> Python 3.4, more than 3 years away. If you think 3 years isn't long
> >> enough for people to be over the Python 3 transition, let's stick in
> >> another version in there and make it 4.5 years.
> >
> > So, why even bother deprecating it if nobody is going to see the
> warnings?
>
> I feel like I'm repeating the PEP, but here it is again anyway. There
> will be messages in the docs and pending deprecation warnings (which
> don't show up by default but can be requested) starting in Python 2.7
> and 3.2. Regular deprecation warnings wouldn't show up until Python
> 3.4, 3 years away. This compromise was intended exactly to address the
> issue you brought up about people getting over the Python 3
> transition.
>
> Steve
> --
> Where did you get that preposterous hypothesis?
> Did Steve tell you that?
>        --- The Hiphopopotamus
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090928/07ca1805/attachment.htm>

From martin at v.loewis.de  Mon Sep 28 06:09:53 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 28 Sep 2009 06:09:53 +0200
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<loom.20090928T024853-85@post.gmane.org>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
Message-ID: <4AC03711.6000203@v.loewis.de>

> If you think getopt and optparse should stick around in 3.X, why is
> that? If you think there are things that getopt and optparse do better
> than argparse, could you please give some examples?

I personally consider getopt superior to both optparse and argparse.
Those are terribly verbose in specifying arguments, whereas getopt's
sequence-of-letters is really nice and compact.

Regards,
Martin

From benjamin at python.org  Mon Sep 28 05:08:37 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 27 Sep 2009 22:08:37 -0500
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
Message-ID: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>

2009/9/27 Steven Bethard <steven.bethard at gmail.com>:
> If you think getopt and optparse should stick around in 3.X, why is
> that? If you think there are things that getopt and optparse do better
> than argparse, could you please give some examples?

Transitioning to Python 3 is already a pain. bytes/str/unicode things
are going to be enough by themselves to drive people nuts. We don't
want to be forcing them to change APIs if optparse is already working
just fine for them. The job is the stdlib is not to force people to
use the "best" or "right" tools. Several years in the future I would
be more supportive of depreacting optparse, but more ways in which 2.x
and 3.x grow farther apart are not going to help jumping the great
version divide.


-- 
Regards,
Benjamin

From benjamin at python.org  Mon Sep 28 05:41:16 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 27 Sep 2009 22:41:16 -0500
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
Message-ID: <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>

2009/9/27 Steven Bethard <steven.bethard at gmail.com>:
> On Sun, Sep 27, 2009 at 8:08 PM, Benjamin Peterson <benjamin at python.org> wrote:
>> 2009/9/27 Steven Bethard <steven.bethard at gmail.com>:
>>> If you think getopt and optparse should stick around in 3.X, why is
>>> that? If you think there are things that getopt and optparse do better
>>> than argparse, could you please give some examples?
>>
>> Transitioning to Python 3 is already a pain. bytes/str/unicode things
>> are going to be enough by themselves to drive people nuts. We don't
>> want to be forcing them to change APIs if optparse is already working
>> just fine for them. The job is the stdlib is not to force people to
>> use the "best" or "right" tools. Several years in the future I would
>> be more supportive of depreacting optparse, but more ways in which 2.x
>> and 3.x grow farther apart are not going to help jumping the great
>> version divide.
>
> The first release where any real deprecation message would show up is
> Python 3.4, more than 3 years away. If you think 3 years isn't long
> enough for people to be over the Python 3 transition, let's stick in
> another version in there and make it 4.5 years.

So, why even bother deprecating it if nobody is going to see the warnings?



-- 
Regards,
Benjamin

From steven.bethard at gmail.com  Mon Sep 28 06:24:54 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 27 Sep 2009 21:24:54 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC03711.6000203@v.loewis.de>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
	<4AC03711.6000203@v.loewis.de>
Message-ID: <d11dcfba0909272124x27c7e232n6186908e523c318e@mail.gmail.com>

On Sun, Sep 27, 2009 at 9:09 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> If you think getopt and optparse should stick around in 3.X, why is
>> that? If you think there are things that getopt and optparse do better
>> than argparse, could you please give some examples?
>
> I personally consider getopt superior to both optparse and argparse.
> Those are terribly verbose in specifying arguments, whereas getopt's
> sequence-of-letters is really nice and compact.

Thanks for the concrete example. Although I'm unconvinced that the
characters you save in the sequence of letters in the getopt.getopt
call aren't afterwards wasted on type conversions, if/else statements
and variable assignments in the subsequent loop, it would be pretty
simple to add to argparse something like::

    ArgumentParser.add_getopt_arguments(options[, long_options])

Then you could replace your getopt code::

    try:
        opts, args = getopt.getopt(sys.argv[1:], "ho:v", ["help", "output="])
    except getopt.GetoptError, err:
        # print help information and exit:
        print str(err) # will print something like "option -a not recognized"
        usage()
        sys.exit(2)
    output = None
    verbose = False
    for o, a in opts:
        if o == "-v":
            verbose = True
        elif o in ("-h", "--help"):
            usage()
            sys.exit()
        elif o in ("-o", "--output"):
            output = a
        else:
            assert False, "unhandled option"

with argparse code like::

    parser = argparse.ArgumentParser()
    parser.add_getopt_arguments("ho:v", ["help", "output="])
    args = parser.parse_args()
    verbose = args.v
    output = args.o or args.output

If something like this were available, would you be alright with
deprecating getopt?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From steven.bethard at gmail.com  Mon Sep 28 05:56:03 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 27 Sep 2009 20:56:03 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <1afaf6160909272049m30328a39tf1bf63faa31d33c2@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
	<1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>
	<d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
	<1afaf6160909272049m30328a39tf1bf63faa31d33c2@mail.gmail.com>
Message-ID: <d11dcfba0909272056y7af39878i374890ac657c3c60@mail.gmail.com>

On Sun, Sep 27, 2009 at 8:49 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/9/27 Steven Bethard <steven.bethard at gmail.com>:
>> On Sun, Sep 27, 2009 at 8:41 PM, Benjamin Peterson <benjamin at python.org> wrote:
>>> 2009/9/27 Steven Bethard <steven.bethard at gmail.com>:
>>>> The first release where any real deprecation message would show up is
>>>> Python 3.4, more than 3 years away. If you think 3 years isn't long
>>>> enough for people to be over the Python 3 transition, let's stick in
>>>> another version in there and make it 4.5 years.
>>>
>>> So, why even bother deprecating it if nobody is going to see the warnings?
>>
>> I feel like I'm repeating the PEP, but here it is again anyway. There
>> will be messages in the docs and pending deprecation warnings (which
>> don't show up by default but can be requested) starting in Python 2.7
>> and 3.2. Regular deprecation warnings wouldn't show up until Python
>> 3.4, 3 years away. This compromise was intended exactly to address the
>> issue you brought up about people getting over the Python 3
>> transition.
>
> But that doesn't tell me why we should deprecate optparse, when it may
> work perfectly well for some people.

Because it's basically unmaintained, and anything you can do in
optparse you can do in argparse with almost identical syntax. So
encouraging people to move from getopt and optparse to argparse is a
net gain for us as Python maintainers -- that's two fewer modules in
the standard library that someone has to take care of bug reports for.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From solipsis at pitrou.net  Mon Sep 28 08:51:03 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Sep 2009 06:51:03 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
Message-ID: <loom.20090928T084811-560@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> > Could you explain what benefit there is for allowing the user to create 
> > network objects that don't represent networks? Is there a use-case 
> > where these networks-that-aren't-networks are something other than a 
> > typo? Under what circumstances would I want to specify a network as 
> > 192.168.1.1/24 instead of 192.168.1.0/24?
> > 
> 
[...]
> 
> So Python code has to make the computation, and it seems most natural
> that the IP library is the piece of code that is able to compute a
> network out of that input.

The thing is, it doesn't create a network, it creates a hybrid "network plus
host" which retains knowledge about the original host (that is, 192.168.1.1
rather than simply 192.168.1.0, if you enter "192.168.1.1/24").

That's what the OP meant with "networks-that-aren't-networks", and that's what
people are objecting to.

Regards

Antoine.



From chambon.pascal at gmail.com  Mon Sep 28 10:08:16 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Mon, 28 Sep 2009 10:08:16 +0200
Subject: [Python-Dev] Fuzziness in io module specs -	PEP
 update	proposition V2
In-Reply-To: <h9nihf$9f4$1@ger.gmane.org>
References: <4AB3DCC9.5050209@gmail.com>
	<4AB3E594.5090907@mrabarnett.plus.com>	<97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net>	<200909201007.11923.steve@pearwood.info>	<4AB58159.3040303@canterbury.ac.nz>
	<4AB5D6CB.6050604@gmail.com>	<4AB5E8DD.6050204@canterbury.ac.nz>
	<4AB5FA81.5060408@gmail.com>	<eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com>	<4AB6590E.2070800@gmail.com>
	<4AB67894.5020903@mrabarnett.plus.com>	<4ABF1B3B.4080106@gmail.com>
	<h9nihf$9f4$1@ger.gmane.org>
Message-ID: <4AC06EF0.50509@gmail.com>

Antoine Pitrou a ?crit :
> Hello,
>
>   
>> So here is the proposed semantic, which matches established conventions:
>>
>> *IOBase.truncate(n: int = None) -> int*
>>     
> [...]
>
> I still don't think there is a sufficient benefit in breaking 
> compatibility. If you want the file pointer to remain the same, you can 
> save it first and restore it afterwards manually.
>
>   
Sure, but won't this truncate become some kind of a burden for py3k, if 
it's twice misleading (it's not a real truncation since it can extend 
the file, and it's not even a truncation or resizing in posix/win32 
style, since the filepointer is moved) ?
Since it was an undocumented behaviour, and py3k doesn't seem to be 
present yet in production environments (or is it ?), I'd promote this 
late-but-maybe-not-too-late change.
But if the consensus prefers the current behaviour, well, it'll be OK to 
me too, as long as it's sufficiently documented and advertised.

>> *Propositions of doc update*
>>     
>
> Please open tracker issues for these kinds of suggestions.
>   
Is the tracker Ok for simple suggestions too ? I thought it was rather 
for obvious bugfixes, and to-be-discused propositions had better be in 
mailing-lists... OK then, I'll open bugtracker issues for these. B-)



> Instead of "than size", perhaps "than n".
>
Whoups, indeed >_<
Actually the signature would rather be:
*IOBase.truncate(size: int = None) -> int*
And I forgot to mention that truncate returns the new file size 
(according to the current PEP)...


> Should an exception be raised if start and/or end are out of range?
I'd advocate it yep, for the sake of "explicit errors". However, should 
it be a ValueError (the ones io functions normally use) or an IndexError 
(which is technically more accurate, but might confuse the user) ?


Regards,
Pascal




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090928/026d23a8/attachment-0001.htm>

From mal at egenix.com  Mon Sep 28 10:25:45 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 28 Sep 2009 10:25:45 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to	ucs4
In-Reply-To: <cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>	<4AB67430.2060306@egenix.com>
	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
Message-ID: <4AC07309.1010608@egenix.com>

Zooko O'Whielacronx wrote:
> Folks:
> 
> I'm sorry, I think I didn't make my concern clear.  My users, and lots
> of other users, are having a problem with incompatibility between
> Python binary extension modules.  One way to improve the situation
> would be if the Python devs would use their "bully pulpit" -- their
> unique position as a source respected by all Linux distributions --
> and say "We recommend that Linux distributions use UCS4 for
> compatibility with one another".  This would not abrogate anyone's
> ability to choose their preferred setting nor, as far as I can tell,
> would it interfere with the ongoing development of Python.

-1

Please note that we did not choose to ship Python as UCS4 binary
on Linux - the Linux distributions did.

The Python default is UCS2 for a good reason: it's a good trade-off
between memory consumption, functionality and performance.

As already mentioned, I also don't understand how the changing
the Python default on Linux would help your users in any way -
if you let distutils compile your extensions, it's automatically
going to use the right Unicode setting for you (as well as your
users).

Unfortunately, this automatic support doesn't help you when
shipping e.g. setuptools eggs, but this is a tool problem,
not one of Python: setuptools completely ignores the fact
that there are two ways to build Python.

I'd suggest you ask the tool maintainers to adjust their tools
to support the Python Unicode option.

> Here are the details:
> 
> I'm the maintainer of several Python packages.  I work hard to make it
> easy for users, even users who don't know anything about Python, to
> use my software.  There have been many pain points in this process and
> I've spent a lot of time on it for about three years now working on
> packaging, including the tools such as setuptools and distutils and
> the new "distribute" tool.  Python packaging has been improving during
> these years -- things are looking up.
> 
> One of the remaining pain points is that I can distribute binaries of
> my Python extension modules for Windows or Mac, but if I distribute a
> binary Python extension module on Linux, then if the user has a
> different UCS2/UCS4 setting then they won't be able to use the
> extension module.  The current de facto standard for Linux is UCS4 --
> it is used by Debian, Ubuntu, Fedora, RHEL, OpenSuSE, etc. etc..  The
> vast majority of Linux users in practice have UCS4, and most binary
> Python modules are compiled for UCS4.
> 
> That means that a few folks will get left out.  Those folks, from my
> experience, are people who built their python executable themselves
> without specifying an override for the default, and the smaller Linux
> distributions who insist on doing whatever upstream Python devs
> recommend instead of doing whatever the other Linux distros are doing.
>  One of the data points that I reported was a Python interpreter that
> was built locally on an Ubuntu server.  Since the person building it
> didn't know to override the default setting of --enable-unicode, he
> ended up with a Python interpreter built for UCS2, even though all the
> Python extension modules shipped by Ubuntu were built with UCS4.

People building their own Python version will usually also build
their own extensions, so I don't really believe that the above
scenario is very common.

Also note that Python will complain loudly when you try to load
a UCS2 extension in a UCS4 build and vice-versa. We've made sure
that any extension using the Python Unicode C API has to be built
for the same UCS version of Python. This is done by using different
names for the C APIs at the C level.

> These are not isolated incidents.  The following google searches
> suggest that a number of people spend time trying to figure out why
> Python extension modules fail on their linux systems:
> 
> http://www.google.com/search?q=PyUnicodeUCS4_FromUnicode+undefined+symbol
> http://www.google.com/search?q=+PyUnicodeUCS2_FromUnicode+undefined+symbol
> http://www.google.com/search?q=_PyUnicodeUCS2_AsDefaultEncodedString+undefined+symbol

Perhaps we should add a FAQ entry for these linker errors
(which are caused by the mentioned C API changes to prevent
mixing UCS version) ?!

Here's a quick way to determine you Python Unicode build type:

python -c "import sys;print((sys.maxunicode<66000)and'UCS2'or'UCS4')"

Perhaps we should include this info as well as an 32/64-bit indicator
and the processor type in the Python startup line:

# python
Python 2.6 (r26:66714, Feb  3 2009, 20:49:49, UCS4, 64-bit, x86_64)
[GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2
Type "help", "copyright", "credits" or "license" for more information.

This would help users find the right binaries to install as
extension.

> Another data point is the Mandriva Linux distribution.  It is probably
> much smaller than Debian, Ubuntu, or RedHat, but it is still one of
> the major, well-known distributions.  I requested of the Python
> maintainer for Mandriva, Michael Scherer, that they switch from UCS2
> to UCS4 in order to reduce compatibility problems like these.  His
> answer as I understood it was that it is best to follow the
> recommendations of the upstream Python devs by using the default
> setting instead of choosing a setting for himself.

Which is IMHO what all Linux distributions should have done.

Distributions should really not be put in charge of upstream
coding design decisions.

Regards,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 28 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Mon Sep 28 10:36:07 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 28 Sep 2009 10:36:07 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to	ucs4
In-Reply-To: <4AC07309.1010608@egenix.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>	<4AB67430.2060306@egenix.com>	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
	<4AC07309.1010608@egenix.com>
Message-ID: <4AC07577.2040903@egenix.com>

M.-A. Lemburg wrote:
> Also note that Python will complain loudly when you try to load
> a UCS2 extension in a UCS4 build and vice-versa. We've made sure
> that any extension using the Python Unicode C API has to be built
> for the same UCS version of Python. This is done by using different
> names for the C APIs at the C level.
> 
>> These are not isolated incidents.  The following google searches
>> suggest that a number of people spend time trying to figure out why
>> Python extension modules fail on their linux systems:
>>
>> http://www.google.com/search?q=PyUnicodeUCS4_FromUnicode+undefined+symbol
>> http://www.google.com/search?q=+PyUnicodeUCS2_FromUnicode+undefined+symbol
>> http://www.google.com/search?q=_PyUnicodeUCS2_AsDefaultEncodedString+undefined+symbol
> 
> Perhaps we should add a FAQ entry for these linker errors
> (which are caused by the mentioned C API changes to prevent
> mixing UCS version) ?!

There already is one:

http://www.python.org/doc/faq/extending/#when-importing-module-x-why-do-i-get-undefined-symbol-pyunicodeucs2

I wonder why it doesn't show up in the Google searches.

> Here's a quick way to determine you Python Unicode build type:
> 
> python -c "import sys;print((sys.maxunicode<66000)and'UCS2'or'UCS4')"
> 
> Perhaps we should include this info as well as an 32/64-bit indicator
> and the processor type in the Python startup line:
> 
> # python
> Python 2.6 (r26:66714, Feb  3 2009, 20:49:49, UCS4, 64-bit, x86_64)
> [GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> 
> This would help users find the right binaries to install as
> extension.

Regards,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 28 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From floris.bruynooghe at gmail.com  Mon Sep 28 10:38:20 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Mon, 28 Sep 2009 09:38:20 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
	<1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>
	<d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
	<d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>
	<9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com>
Message-ID: <20090928083820.GA488@laurie.devork>

On Mon, Sep 28, 2009 at 06:59:45AM +0300, Yuvgoog Greenle wrote:
> -1 for deprecating getopt. getopt is super-simple and especially useful for
> c programmers learning python.
> 
> +1 for argparse.+1 for eventual deprecation of optparse - optparse and
> argparse have a very similar syntax and having both is just
> confusing. tsboapooowtdi

+1 on all of this :-)

It would be a shame to see getopt go but optparse -> argparse seems
logical.


Floris

From chambon.pascal at gmail.com  Mon Sep 28 10:50:51 2009
From: chambon.pascal at gmail.com (Pascal Chambon)
Date: Mon, 28 Sep 2009 10:50:51 +0200
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4ABF4996.6000900@canterbury.ac.nz>
References: <4ABF2047.7050501@gmail.com> <4ABF4996.6000900@canterbury.ac.nz>
Message-ID: <4AC078EB.1090500@gmail.com>


>
>>  +-InvalidFileNameError (filepath max lengths, or "? / : " characters 
>> in a windows file name...)
>
> This might be a bit too precise. Unix just has EINVAL, which
> covers any kind of invalid parameter, not just file names.
>
Allright thanks, an InvalidParameter (or similar) exception should do it 
better then.


> Personally I'd love to see a richer set of exceptions for IO errors, 
> so long as they can be implemented for all supported platforms and no 
> information (err number from the os) is lost.
>
> I've been implementing a fake 'file' type [1] for Silverlight which 
> does IO operations using local browser storage. The use case is for an 
> online Python tutorial running in the browser [2]. Whilst implementing 
> the exception behaviour (writing to a file open in read mode, etc) I 
> considered improving the exception messages as they are very poor - 
> but decided that being similar to CPython was more important.
>
> Michael
>
> [1] 
> http://code.google.com/p/trypython/source/browse/trunk/trypython/app/storage.py 
> and 
> http://code.google.com/p/trypython/source/browse/trunk/trypython/app/tests/test_storage.py 
>
> [2] http://www.trypython.org/
>
Cool stuff  :-)
It's indeed quite unsure at the moment which exceptions it will really 
be possible (and relevant) to implement in a cross-platform way... I 
guess I should use my own fileio implementation as a playground and a 
proof of concept, before we specify anything for CPython.


> What happens isn't specified, but in practice (with the current 
> implementation) the overwriting will happen at the byte level, without 
> any check for correctness at the character level.
>
> Actually, read+write text streams are implemented quite crudely, and 
> little testing is done of them. The reason, as you discovered, is that 
> the semantics are too weak, and it is not obvious how stronger semantics 
> could look like. People wanting to do sophisticated random reads+writes 
> over a text file should probably handle the encoding themselves and 
> access the file at the binary level.
>   
It sounds ok to me, as long as we notify users about this danger (I've 
myself just realized about it). Most newcomers may happily open an UTF8 
text file, and read/write in it carelessly, without realizing that the 
characters they write actually screw up the file...


> How about just making IOError = OSError, and introducing your proposed 
> subclasses? Does the usage of IOError vs OSError have *any* useful 
> semantics?
>
I though that OSError dealt with a larger set of errors than IOError, 
but after checking the errno codes, it seems that they're all more or 
less related to IO problems (if we include interprocess communication in 
I/O). So theoretically, IOErrors and OSErrors might be merged. Note that 
in this case, windowsErrors would have to become children of 
EnvironmentError, because windows error code really seem to go farther 
than io errors (they deal with recursion limits, thousands of PC 
parameters...).
The legacy is so heavy that OSError would have to remain as is, I think, 
but we might simply forget it in new io modules, and concentrate on an 
IOError hierarchy to provide all the info needed by the developer.



>
>> Some of the error messages are truly awful though as things stand, 
>> especially for someone new to Python. Try to read from a file handle 
>> opened in read mode for example: IOError: [Errno 9] Bad file descriptor
> Subdividing the IOError exception won't help with
> that, because all you have to go on when deciding
> which exception to raise is the error code returned
> by the OS. If the same error code results from a
> bunch of different things, there's not much Python
> can do to sort them out.
>
Well, you don't only have the error number, you also have the context of 
this exception. IOErrors subclasses would particularly be useful in a 
"high level" IO contect, when each single method can issue lots of 
system calls (to check the file, lock it, edit it...). If the error is 
raised during your locking operation, you can decide to sort it as 
"LockingError" even if the error code provided might appear in several 
different situations.

Regards,
Pascal



From fuzzyman at voidspace.org.uk  Mon Sep 28 12:03:35 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 28 Sep 2009 11:03:35 +0100
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4AC00D2A.6020504@canterbury.ac.nz>
References: <4ABF2047.7050501@gmail.com>
	<h9njfi$9f4$2@ger.gmane.org>	<4ABFCDED.8080105@gmail.com>
	<4ABFD9D6.9070602@voidspace.org.uk>
	<4AC00D2A.6020504@canterbury.ac.nz>
Message-ID: <4AC089F7.8030004@voidspace.org.uk>

Greg Ewing wrote:
> Michael Foord wrote:
>
>> Some of the error messages are truly awful though as things stand, 
>> especially for someone new to Python. Try to read from a file handle 
>> opened in read mode for example: IOError: [Errno 9] Bad file descriptor
>
> Subdividing the IOError exception won't help with
> that, because all you have to go on when deciding
> which exception to raise is the error code returned
> by the OS. If the same error code results from a
> bunch of different things, there's not much Python
> can do to sort them out.
>

Well, the file type knows what mode it is opened in so catching these 
errors is really not difficult at all.

Michael


> The error messages produced for the various error
> codes could perhaps be improved, but that's an
> orthogonal issue.
>


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ndbecker2 at gmail.com  Mon Sep 28 13:09:59 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 28 Sep 2009 07:09:59 -0400
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
Message-ID: <h9q5i7$hv2$1@ger.gmane.org>

If the plan is to migrate from optparse to argparse, this could be made a 
bit easier.  If it weren't for the fact that some names are different in 
argparse than optparse, I believe many optparse usages could port with no 
change.


From fuzzyman at voidspace.org.uk  Mon Sep 28 13:13:32 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 28 Sep 2009 12:13:32 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <h9q5i7$hv2$1@ger.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<h9q5i7$hv2$1@ger.gmane.org>
Message-ID: <4AC09A5C.9060007@voidspace.org.uk>

Neal Becker wrote:
> If the plan is to migrate from optparse to argparse, this could be made a 
> bit easier.  If it weren't for the fact that some names are different in 
> argparse than optparse, I believe many optparse usages could port with no 
> change.
>   

The new names in argparse fit with the fact that argparse has a 
different 'rationale' than optparse. What about an optparse 
compatibility layer to make porting easier?

Although it can't cover the whole optparse API (as explained in the PEP) 
it could perhaps cover most 'normal' usage?

Michael

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From rdmurray at bitdance.com  Mon Sep 28 13:34:46 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 28 Sep 2009 07:34:46 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC03410.4010104@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>

On Mon, 28 Sep 2009 at 05:57, "Martin v. L?wis" wrote:
>>> Finally, to Stephen's point about seeing the other side of the
>>> argument, I wrote this offlist a week ago:
>>>
>>>   I *understand* what you're saying, I *understand* that
>>> 192.168.1.1/24 isn't a network,
>>
>> But you still want to treat it as one.
>>
>> Could you explain what benefit there is for allowing the user to create
>> network objects that don't represent networks? Is there a use-case
>> where these networks-that-aren't-networks are something other than a
>> typo? Under what circumstances would I want to specify a network as
>> 192.168.1.1/24 instead of 192.168.1.0/24?
>>
>
> It's fairly obvious to me why the library should support 192.168.1.1/24
> as an input, and return a network.
>
> End-users are likely going to enter such things (e.g. 82.94.164.162/29),
> as they will know one IP address in the network (namely, of one machine
> that they are looking at), and they will know the prefix length (more
> often, how large the network is - 8 or 16 or 32). So very clearly,
> end users should not be required to make the computation in their heads.
>
> So Python code has to make the computation, and it seems most natural
> that the IP library is the piece of code that is able to compute a
> network out of that input.
>
> Does that answer your question?

The fundamental divide here is between two behaviors.

ipaddr:

     >>> x = IPv4Network('192.168.1.1/24')
     >>> y = IPv4Network('192.168.1.0/24')
     >>> x == y
     False
     >>> x.ip
     IPv4Address('192.168.1.1')

desired:

     >>> x = IPv4Network('192.168.1.1/24')
     >>> y = IPv4Network('192.168.1.0/24')
     >>> x == y
     True
     >>> x.ip
     Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
     AttributeError: 'IPv4Network' object has no attribute 'ip'

Everything else is pretty much bikeshedding and can be dealt with.  This
is fundamental and Peter has indicated he will not change it.

--David

From eric at trueblade.com  Mon Sep 28 13:48:47 2009
From: eric at trueblade.com (Eric Smith)
Date: Mon, 28 Sep 2009 07:48:47 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
Message-ID: <4AC0A29F.1090907@trueblade.com>

R. David Murray wrote:
> The fundamental divide here is between two behaviors.
> 
> ipaddr:
> 
>     >>> x = IPv4Network('192.168.1.1/24')
>     >>> y = IPv4Network('192.168.1.0/24')
>     >>> x == y
>     False
>     >>> x.ip
>     IPv4Address('192.168.1.1')
> 
> desired:
> 
>     >>> x = IPv4Network('192.168.1.1/24')
>     >>> y = IPv4Network('192.168.1.0/24')
>     >>> x == y
>     True
>     >>> x.ip
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     AttributeError: 'IPv4Network' object has no attribute 'ip'
> 
> Everything else is pretty much bikeshedding and can be dealt with.  This
> is fundamental and Peter has indicated he will not change it.

I think that's an excellent summary, David. And I'm -1 unless it changes 
to the "desired" behavior above.

Eric.

From rdmurray at bitdance.com  Mon Sep 28 13:50:25 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 28 Sep 2009 07:50:25 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
Message-ID: <Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net>

On Mon, 28 Sep 2009 at 07:34, R. David Murray wrote:
> The fundamental divide here is between two behaviors.
>
> ipaddr:
>
>    > > >  x = IPv4Network('192.168.1.1/24')
>    > > >  y = IPv4Network('192.168.1.0/24')
>    > > >  x == y
>     False
>    > > >  x.ip
>     IPv4Address('192.168.1.1')
>
> desired:
>
>    > > >  x = IPv4Network('192.168.1.1/24')
>    > > >  y = IPv4Network('192.168.1.0/24')
>    > > >  x == y
>     True
>    > > >  x.ip
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     AttributeError: 'IPv4Network' object has no attribute 'ip'
>
> Everything else is pretty much bikeshedding and can be dealt with.  This
> is fundamental and Peter has indicated he will not change it.

By the way, I think this is a reasonable position on Peter's part.
He's got an installed base that expects the current behavior (ie: the
"field tested module" has this behavior; making this change would produce
a fundamentally new module that has _not_ been field tested).

But I also think that the current behavior is not a good behavior for
a module in the stdlib.

--David

From ncoghlan at gmail.com  Mon Sep 28 14:11:55 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Sep 2009 22:11:55 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909271731ra859ba5yda1d5f30e13a8b26@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>	<8517e9350909271053n174ee3dy9173ecf73e3dd39b@mail.gmail.com>	<200909281013.39069.steve@pearwood.info>
	<8517e9350909271731ra859ba5yda1d5f30e13a8b26@mail.gmail.com>
Message-ID: <4AC0A80B.1000407@gmail.com>

Peter Moody wrote:
>> That is, you've rejected the idea of having:
>>
>>>>> IPv4Network(192.168.1.1/24)
>> IPv4Network(192.168.1.0/24)
> 
> yes, I and everyone have rejected that idea. this should either raise
> an  error, or do what it does now, that is, return
> IPv4Network('192.168.1.1/24')

Um, no, several people (me included) have said that defining an
IPNetwork based on its *network address* and its netmask, instead of the
arbitrary hostname.


>> as reducing functionality, presumably because the address 192.168.1.1 is
>> lost.
> 
> good guess.
> 
>> Sounds just like an Address+netmask to me, with added
>> network-like behaviour.
>>
>> Some people have requested a way of explicitly calculating a network
>> from an Address and netmask, and you've been hostile to the idea. But
>> it seems to me that your API does exactly that.
> 
> You mean the suggestion to include an IPv?Network object attribute
> with the IPv?Address objects? I thought that was dropped long ago.

No, we're talking about any way of calculating the canonical network
object given a host within that network and the netmask.

With the current incarnation of ipaddr, the way you do that is to create
an IPNetwork object with net.ip != net.network. But then you get a
bizarre artifact, in that the same network will not compare equal if it
is derived using a different IP address.

There are *3* major concepts in ipaddr:
- single address
- network (defined by network address + net mask)
- arbitrary host associated with a specific network

Since those 3 concepts are being mapped to only two classes, 3 different
ways have been proposed of handling the 3rd concept:
- a "network" attribute on IPAddress objects (rejected, largely by
consensus as far as I can tell)
- a 2-tuple containing an IPAddress object and an IPNetwork object
(rejected implicitly by your refusal to remove the .ip attribute from
IPNetwork objects)
- the current implementation, which uses an IPNetwork object with net.ip
!= net.network.

That 3rd approach would be largely OK, *except* that the current
implementation of it breaks the definition of network equality.

Currently, IPNetwork("192.168.1.0/24") != IPNetwork("192.168.1.1/24"),
despite the fact that they refer to the exact same network. I believe
that is the fundamental point of contention here.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Sep 28 14:34:36 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Sep 2009 22:34:36 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <F5B27C82-50C3-4838-B507-53DF82C35D4C@gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>	<4ABFE075.2070309@voidspace.org.uk>
	<F5B27C82-50C3-4838-B507-53DF82C35D4C@gmail.com>
Message-ID: <4AC0AD5C.1000008@gmail.com>

Doug Hellmann wrote:
> 
> On Sep 27, 2009, at 6:00 PM, Michael Foord wrote:
> 
>> Brett Cannon wrote:
>>> I am going to state upfront that I am +1 for this and I encouraged
>>> Steven to submit this PEP on the stdlib-SIG. I still remember watching
>>> Steven's lightning talk at PyCon 2009 on argparse and being impressed
>>> by it (along with the rest of the audience which was obviously
>>> impressed as well).
>>>
>>> I think argprase is a good improvement over what we have now (getopt
>>> and optparse), it's stable, considered best-of-breed by the community
>>> for a while (as shown as how many times argparse has been suggestion
>>> for inclusion), and Steven is already a core committer so is already
>>> set to maintain the code. That covers the usual checklist we have for
>>> even looking at a PEP to add a module to the standard library.
>>>
>>>
>>
>> I've used argparse and really like it. I could also have done with the
>> subcommand support in recent changes to unittest.
>>
>> +1 for inclusion.
> 
> +1 as well.  I wish I had been able to use this library for a recent
> project using sub-commands.

+1 here as well (although we should definitely find a way to use
str.format strings instead of %-format ones... come to think of it, does
even the logging module support str.format style formatting in Py3k?)

Reading through the argparse vs optparse comparison in the argparse docs
when the topic of possibly adding it came up a few weeks back I kept
going "yep, I've rolled my own version of that, oh, and that, yeah, that
too...".

argparse probably wouldn't have helped me even if I'd known about it
(due to license review overhead), but it certainly addressed a whole
heap of problems I had encountered in practice, and in ways that were a
lot cleaner than the comparatively hacky approaches I had used for my
own purposes (internal testing and diagnostic scripts are like that...).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Sep 28 14:37:16 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Sep 2009 22:37:16 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<loom.20090928T024853-85@post.gmane.org>	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>	<1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>	<d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
	<d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>
Message-ID: <4AC0ADFC.1030604@gmail.com>

Steven Bethard wrote:
> I feel like I'm repeating the PEP, but here it is again anyway. There
> will be messages in the docs and pending deprecation warnings (which
> don't show up by default but can be requested) starting in Python 2.7
> and 3.2. Regular deprecation warnings wouldn't show up until Python
> 3.4, 3 years away. This compromise was intended exactly to address the
> issue you brought up about people getting over the Python 3
> transition.

For 2.x, I'd go even further and only have the PendingDeprecation
warnings show up under the "-3" command line flag.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Sep 28 14:24:55 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Sep 2009 22:24:55 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net>
Message-ID: <4AC0AB17.5070801@gmail.com>

R. David Murray wrote:
> On Mon, 28 Sep 2009 at 07:34, R. David Murray wrote:
>> The fundamental divide here is between two behaviors.
>>
>> ipaddr:
>>
>>    > > >  x = IPv4Network('192.168.1.1/24')
>>    > > >  y = IPv4Network('192.168.1.0/24')
>>    > > >  x == y
>>     False
>>    > > >  x.ip
>>     IPv4Address('192.168.1.1')
>>
>> desired:
>>
>>    > > >  x = IPv4Network('192.168.1.1/24')
>>    > > >  y = IPv4Network('192.168.1.0/24')
>>    > > >  x == y
>>     True
>>    > > >  x.ip
>>     Traceback (most recent call last):
>>       File "<stdin>", line 1, in <module>
>>     AttributeError: 'IPv4Network' object has no attribute 'ip'
>>
>> Everything else is pretty much bikeshedding and can be dealt with.  This
>> is fundamental and Peter has indicated he will not change it.
> 
> By the way, I think this is a reasonable position on Peter's part.
> He's got an installed base that expects the current behavior (ie: the
> "field tested module" has this behavior; making this change would produce
> a fundamentally new module that has _not_ been field tested).
> 
> But I also think that the current behavior is not a good behavior for
> a module in the stdlib.

I should note that I've softened my position slightly from what I posted
yesterday. I could live with the following compromise:

    >>> x = IPv4Network('192.168.1.1/24')
    >>> y = IPv4Network('192.168.1.0/24')
    >>> x == y # Equality is the part I really want to see changed
    True
    >>> x.ip
    IPv4Address('192.168.1.1')
    >>> y.ip
    IPv4Address('192.168.1.0')

So IPNetwork equality classes would be rendered conceptually coherent
(i.e. list(net1) == list(net2) would imply net1 == net2), but the "Host
with associated network" functionality would also be retained.

To check that the hosts were the same in addition to the network
definition one could easily check the .ip attribute in addition to
checking the networks themselves.

Something I also noticed is that "compare_networks" in the ipaddr
reference implementation doesn't follow the PEP - it defines IPv4
networks as being less than IPv6 networks instead of raising a TypeError.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Sep 28 14:40:44 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Sep 2009 22:40:44 +1000
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <h9oph7$hkf$1@ger.gmane.org>
References: <4ABF2047.7050501@gmail.com>
	<h9njfi$9f4$2@ger.gmane.org>	<4ABFCDED.8080105@gmail.com>
	<h9oph7$hkf$1@ger.gmane.org>
Message-ID: <4AC0AECC.4050104@gmail.com>

Antoine Pitrou wrote:
> Le Mon, 28 Sep 2009 06:41:17 +1000, Nick Coghlan a ?crit :
>> Michael's response cut to the heart of the issue though - a richer IO
>> exception hierarchy can make life interesting for compatibility purposes
>> (especially when creating "file-like" interfaces to non-file objects).
> 
> Well, not more interesting than currently where you need to replicate 
> errno numbers if you want to make the errors precise enough, since an API 
> consumer wanting to check specific error conditions will discriminate on 
> errno.
> 
> If you don't want to go to that level of perfection, you just have to 
> raise a plain IOError (or OSError :-)) without bothering about errno or 
> subclasses; like you would do today.

Fair point.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From exarkun at twistedmatrix.com  Mon Sep 28 14:58:18 2009
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Mon, 28 Sep 2009 12:58:18 -0000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC03410.4010104@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
Message-ID: <20090928125818.1533.2070636475.divmod.xquotient.382@localhost.localdomain>

On 03:57 am, martin at v.loewis.de wrote:
>>>Finally, to Stephen's point about seeing the other side of the
>>>argument, I wrote this offlist a week ago:
>>>
>>>   I *understand* what you're saying, I *understand* that
>>>192.168.1.1/24 isn't a network,
>>
>>But you still want to treat it as one.
>>
>>Could you explain what benefit there is for allowing the user to 
>>create
>>network objects that don't represent networks? Is there a use-case
>>where these networks-that-aren't-networks are something other than a
>>typo? Under what circumstances would I want to specify a network as
>>192.168.1.1/24 instead of 192.168.1.0/24?
>
>It's fairly obvious to me why the library should support 192.168.1.1/24
>as an input, and return a network.
>
>End-users are likely going to enter such things (e.g. 
>82.94.164.162/29),
>as they will know one IP address in the network (namely, of one machine
>that they are looking at), and they will know the prefix length (more
>often, how large the network is - 8 or 16 or 32). So very clearly,
>end users should not be required to make the computation in their 
>heads.
>
>So Python code has to make the computation, and it seems most natural
>that the IP library is the piece of code that is able to compute a
>network out of that input.

And this is a rather classic example of a misfeature.  "(Non-developer) 
End users will type it in" is not argument for supporting a particular 
string format as the primary constructor for an object.  Constructors 
are for *developers*.  They should be made most useful for *developers*. 
The issue of dealing with user input (which may have many other quirks) 
is separate and should be dealt with separately.

Jean-Paul

From jon+python-dev at unequivocal.co.uk  Mon Sep 28 15:03:17 2009
From: jon+python-dev at unequivocal.co.uk (Jon Ribbens)
Date: Mon, 28 Sep 2009 14:03:17 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing	module
In-Reply-To: <20090928083820.GA488@laurie.devork>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
	<1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>
	<d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
	<d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>
	<9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com>
	<20090928083820.GA488@laurie.devork>
Message-ID: <20090928130317.GR23367@snowy.squish.net>

On Mon, Sep 28, 2009 at 09:38:20AM +0100, Floris Bruynooghe wrote:
> On Mon, Sep 28, 2009 at 06:59:45AM +0300, Yuvgoog Greenle wrote:
> > -1 for deprecating getopt. getopt is super-simple and especially useful for
> > c programmers learning python.
> > 
> > +1 for argparse.+1 for eventual deprecation of optparse - optparse and
> > argparse have a very similar syntax and having both is just
> > confusing. tsboapooowtdi
> 
> +1 on all of this :-)
> 
> It would be a shame to see getopt go but optparse -> argparse seems
> logical.

+1 from me too - keep getopt, deprecate optparse.

From digitalxero at gmail.com  Mon Sep 28 15:12:57 2009
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Mon, 28 Sep 2009 07:12:57 -0600
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
Message-ID: <e9764b730909280612g4ad3a3a8i7154762b6031dcd0@mail.gmail.com>

On Mon, Sep 28, 2009 at 5:34 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> The fundamental divide here is between two behaviors.
>
> ipaddr:
>
> ? ?>>> x = IPv4Network('192.168.1.1/24')
> ? ?>>> y = IPv4Network('192.168.1.0/24')
> ? ?>>> x == y
> ? ?False
> ? ?>>> x.ip
> ? ?IPv4Address('192.168.1.1')
>
> desired:
>
> ? ?>>> x = IPv4Network('192.168.1.1/24')
> ? ?>>> y = IPv4Network('192.168.1.0/24')
> ? ?>>> x == y
> ? ?True
> ? ?>>> x.ip
> ? ?Traceback (most recent call last):
> ? ? ?File "<stdin>", line 1, in <module>
> ? ?AttributeError: 'IPv4Network' object has no attribute 'ip'


I submitted a patch to ipadrdr http://codereview.appspot.com/124057
which would do
    >>> x = IPv4Network('192.168.1.1/24')
    >>> y = IPv4Network('192.168.1.0/24')
    >>> x == y
    True
    >>> x.ip
     IPv4Address('192.168.1.1')

From ncoghlan at gmail.com  Mon Sep 28 13:49:34 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Sep 2009 21:49:34 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC00AE1.7080009@canterbury.ac.nz>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<4ABF015E.5060002@gmail.com>	<8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<loom.20090927T185114-628@post.gmane.org>	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>	<loom.20090927T201052-343@post.gmane.org>	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>	<4ABFCC6B.3020409@gmail.com>
	<4AC00AE1.7080009@canterbury.ac.nz>
Message-ID: <4AC0A2CE.7010302@gmail.com>

Greg Ewing wrote:
> Nick Coghlan wrote:
> 
>> The use case for "given a netmask and an arbitrary host on that network,
>> give me the appropriate IPNetwork object" has been well established by
>> this discussion (although still isn't particularly well described even
>> in the latest PEP update). This is what "strict=False" covers,
> 
> I think I'd rather have a separate constructor function
> or method for this, rather than a somewhat cryptically
> named boolean parameter.

Yes, I would prefer a separate method as well. It doesn't bother as much
as the broken definition of equality though.

In fact, if the definition of IPNetwork equality was fixed then leaving
strict mode out entirely would probably be OK.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From daniel at stutzbachenterprises.com  Mon Sep 28 16:04:58 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 28 Sep 2009 09:04:58 -0500
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC0AB17.5070801@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net>
	<4AC0AB17.5070801@gmail.com>
Message-ID: <eae285400909280704w39520754h53e31c6ed594332@mail.gmail.com>

On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> I should note that I've softened my position slightly from what I posted
> yesterday. I could live with the following compromise:
>
>    >>> x = IPv4Network('192.168.1.1/24')
>    >>> y = IPv4Network('192.168.1.0/24')
>     >>> x == y # Equality is the part I really want to see changed
>    True
>     >>> x.ip
>    IPv4Address('192.168.1.1')
>    >>> y.ip
>     IPv4Address('192.168.1.0')
>

With those semantics, IPv4Network objects with distinct IP addresses (but
the same network) could no longer be stored in a dictionary or set.  IMO, it
is a little counter-intuitive for objects to compare equal yet have
different properties.  I don't think this is a good compromise.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090928/a40aec1b/attachment.htm>

From dickinsm at gmail.com  Mon Sep 28 16:06:58 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 28 Sep 2009 15:06:58 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <eae285400909280704w39520754h53e31c6ed594332@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net>
	<4AC0AB17.5070801@gmail.com>
	<eae285400909280704w39520754h53e31c6ed594332@mail.gmail.com>
Message-ID: <5c6f2a5d0909280706n2a83689bl9e04fe705b91c202@mail.gmail.com>

On Mon, Sep 28, 2009 at 3:04 PM, Daniel Stutzbach
<daniel at stutzbachenterprises.com> wrote:
> On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> I should note that I've softened my position slightly from what I posted
>> yesterday. I could live with the following compromise:
>>
>> ? ?>>> x = IPv4Network('192.168.1.1/24')
>> ? ?>>> y = IPv4Network('192.168.1.0/24')
>> ? ?>>> x == y # Equality is the part I really want to see changed
>> ? ?True
>> ? ?>>> x.ip
>> ? ?IPv4Address('192.168.1.1')
>> ? ?>>> y.ip
>> ? ?IPv4Address('192.168.1.0')
>
> With those semantics, IPv4Network objects with distinct IP addresses (but
> the same network) could no longer be stored in a dictionary or set.? IMO, it
> is a little counter-intuitive for objects to compare equal yet have
> different properties.? I don't think this is a good compromise.

This worries me too.  It seems like a potentially dangerous half-measure.

Mark

From eric at trueblade.com  Mon Sep 28 16:16:30 2009
From: eric at trueblade.com (Eric Smith)
Date: Mon, 28 Sep 2009 10:16:30 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <5c6f2a5d0909280706n2a83689bl9e04fe705b91c202@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>	<4AC03410.4010104@v.loewis.de>	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>	<Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net>	<4AC0AB17.5070801@gmail.com>	<eae285400909280704w39520754h53e31c6ed594332@mail.gmail.com>
	<5c6f2a5d0909280706n2a83689bl9e04fe705b91c202@mail.gmail.com>
Message-ID: <4AC0C53E.3020805@trueblade.com>

Mark Dickinson wrote:
> On Mon, Sep 28, 2009 at 3:04 PM, Daniel Stutzbach
> <daniel at stutzbachenterprises.com> wrote:
>> On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> I should note that I've softened my position slightly from what I posted
>>> yesterday. I could live with the following compromise:
>>>
>>>    >>> x = IPv4Network('192.168.1.1/24')
>>>    >>> y = IPv4Network('192.168.1.0/24')
>>>    >>> x == y # Equality is the part I really want to see changed
>>>    True
>>>    >>> x.ip
>>>    IPv4Address('192.168.1.1')
>>>    >>> y.ip
>>>    IPv4Address('192.168.1.0')
>> With those semantics, IPv4Network objects with distinct IP addresses (but
>> the same network) could no longer be stored in a dictionary or set.  IMO, it
>> is a little counter-intuitive for objects to compare equal yet have
>> different properties.  I don't think this is a good compromise.
> 
> This worries me too.  It seems like a potentially dangerous half-measure.

I agree. I think keeping the concepts distinct and adding 2 new classes 
is a better solution.

Eric.


From steven.bethard at gmail.com  Mon Sep 28 16:28:39 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 28 Sep 2009 07:28:39 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <20090928130317.GR23367@snowy.squish.net>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
	<1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>
	<d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
	<d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>
	<9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com>
	<20090928083820.GA488@laurie.devork>
	<20090928130317.GR23367@snowy.squish.net>
Message-ID: <d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>

On Mon, Sep 28, 2009 at 6:03 AM, Jon Ribbens
<jon+python-dev at unequivocal.co.uk> wrote:
> On Mon, Sep 28, 2009 at 09:38:20AM +0100, Floris Bruynooghe wrote:
>> On Mon, Sep 28, 2009 at 06:59:45AM +0300, Yuvgoog Greenle wrote:
>> > -1 for deprecating getopt. getopt is super-simple and especially useful for
>> > c programmers learning python.
>> >
>> > +1 for argparse.+1 for eventual deprecation of optparse - optparse and
>> > argparse have a very similar syntax and having both is just
>> > confusing. tsboapooowtdi
>>
>> +1 on all of this :-)
>>
>> It would be a shame to see getopt go but optparse -> argparse seems
>> logical.
>
> +1 from me too - keep getopt, deprecate optparse.

Ok, sounds like there are a number of supporters for keeping getopt
around and just deprecating optparse. For those who'd like to keep
getopt around, I have a few questions:

* Would you be opposed to a note in the getopt documentation
suggesting argparse as an alternative?

* Would you like argparse to grow an add_getopt_arguments method (as
in my other post)?

* If argparse grew an add_getopt_arguments, would you still want to
keep getopt around? And if so, why?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From rdmurray at bitdance.com  Mon Sep 28 16:35:57 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 28 Sep 2009 10:35:57 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <9118C602-C5D3-49A9-B07F-CE61723D4D6A@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<h8sd2n$n1g$1@ger.gmane.org> <4ABE530D.8090609@v.loewis.de>
	<200909271108.32456.steve@pearwood.info> <4ABF0CA8.2030408@v.loewis.de>
	<9118C602-C5D3-49A9-B07F-CE61723D4D6A@gmail.com>
Message-ID: <Pine.LNX.4.64.0909281000200.18193@kimball.webabinitio.net>

On Sun, 27 Sep 2009 at 10:06, David Moss wrote:
> On 27 Sep 2009, at 07:56, "Martin v. L??wis" <martin at v.loewis.de> wrote:
>>  I wouldn't ask for that: it should certainly be possible to supply
>>  masks. However, I would want to reject masks that don't correspond to
>>  a prefix, and have only the prefix length in the internal representation.
>> 
> +1 on rejection of netmasks without direct CIDR prefix equivalents. AFAIK 
> Cisco routers accept them but I don't see how they would be useful in 
> practice (unless someone can demonstrate their need for this).

Several years ago I ran into a network with a non-CIDR netmask.  This
was in the early days of CIDR, and I hope that that network is no longer
configured that way....

A little googling produced this hit:

http://tools.ietf.org/wg/mip4/draft-ietf-mip4-nemo-v4-base/draft-ietf-mip4-nemo-v4-base-11-from-10.diff.txt

which while not a standard itself clearly believes that non-contiguous
netmasks are valid, and is dated 2008.  Earlier RFCs (950 and 1219)
give them as valid but discouraged.

The linux stack (or at least the ifconfig command on my system) does not
appear to support non-contiguous netmasks.  From a brief search on Cisco's
site it looks like Cisco only partially supports non-contiguous masks
(you can use them in wildmasks in access lists, but not in subnet masks
on interfaces).  (I just tried setting a non-contiguous netmask on an
interface on one of my Ciscos, and it gave me a 'bad mask' error message.)

I think support for non-contiguous netmasks could be safely omitted,
but it would mean the module did not fully support the standards, which
should be noted in the docs (and why).

--David

From steve at pearwood.info  Mon Sep 28 16:36:56 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 29 Sep 2009 00:36:56 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
Message-ID: <200909290036.56742.steve@pearwood.info>

On Mon, 28 Sep 2009 06:59:32 am Steven Bethard wrote:

> Why aren't getopt and optparse enough?
[snip]

As a newbie, I found optparse intimidating because it provided so many 
features. I expect argparse will be the same, only more so. 
(Disclaimer: I haven't used argparse, I'm merely making a prediction 
based on the available functionality.)

As it turned out, optparse wasn't anywhere near as difficult to use as I 
expected it to be, but I suggest that newbies will have the same 
response to argparse as I had to optparse -- "this looks complicated, 
I'll just roll my own". At least I had the option to use getopt.

Not every module needs to be newbie-friendly, but something as 
fundamental as argument parsing should at least provide a gentle 
learning curve.

Currently, getopt more-or-less plays the role of "bare-bones" command 
line parsing, which I think is important. Assuming argparse is added to 
the std lib, I'm -0.5 on deprecating getopt, 0 on deprecating optparse.



-- 
Steven D'Aprano

From digitalxero at gmail.com  Mon Sep 28 16:42:15 2009
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Mon, 28 Sep 2009 08:42:15 -0600
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <eae285400909280704w39520754h53e31c6ed594332@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net>
	<4AC0AB17.5070801@gmail.com>
	<eae285400909280704w39520754h53e31c6ed594332@mail.gmail.com>
Message-ID: <e9764b730909280742l64280d1dwa252191ea9c031a4@mail.gmail.com>

On Mon, Sep 28, 2009 at 8:04 AM, Daniel Stutzbach
<daniel at stutzbachenterprises.com> wrote:
> On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> I should note that I've softened my position slightly from what I posted
>> yesterday. I could live with the following compromise:
>>
>> ? ?>>> x = IPv4Network('192.168.1.1/24')
>> ? ?>>> y = IPv4Network('192.168.1.0/24')
>> ? ?>>> x == y # Equality is the part I really want to see changed
>> ? ?True
>> ? ?>>> x.ip
>> ? ?IPv4Address('192.168.1.1')
>> ? ?>>> y.ip
>> ? ?IPv4Address('192.168.1.0')
>
> With those semantics, IPv4Network objects with distinct IP addresses (but
> the same network) could no longer be stored in a dictionary or set.? IMO, it
> is a little counter-intuitive for objects to compare equal yet have
> different properties.? I don't think this is a good compromise.

Thats not true, the patch I submitted
http://codereview.appspot.com/124057 still allows the networks to be
included in a set or as a dict key

>>> net1 = IPNetwork("10.1.2.3/24")
>>> net2 = IPNetwork("10.1.2.0/24")
>>> print hash(net1) == hash(net2)
False
>>> print net1 == net2
True
>>> test = {net1: "something", net2: "something else"}
>>> print test
{IPv4Network('10.1.2.0/24'): 'something else',
IPv4Network('10.1.2.3/24'): 'something'}
>>> test2 = set([net1, net2])
>>> print test2
set([IPv4Network('10.1.2.0/24'), IPv4Network('10.1.2.3/24')])

From steven.bethard at gmail.com  Mon Sep 28 16:43:04 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 28 Sep 2009 07:43:04 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <h9q5i7$hv2$1@ger.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<h9q5i7$hv2$1@ger.gmane.org>
Message-ID: <d11dcfba0909280743n7ce99d03o64396b33bcddc739@mail.gmail.com>

On Mon, Sep 28, 2009 at 4:09 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
> If the plan is to migrate from optparse to argparse, this could be made a
> bit easier. ?If it weren't for the fact that some names are different in
> argparse than optparse, I believe many optparse usages could port with no
> change.

I could conceivably add an OptionParser class which::

* Has an add_option method which delegates to add_argument

* Has a parse_args method which delegates to parse_known_args

* Registers the string names for types that optparse uses

That might allow some users to just switch their import as long as
they're not using any callbacks, catching any exceptions, extending
any APIs, etc. I'm not 100% convinced that it's good to add a subtly
different API that people would have to unlearn, but if there's enough
support for this, I'm open to the idea.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From skip at pobox.com  Mon Sep 28 16:47:38 2009
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 28 Sep 2009 09:47:38 -0500
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC0AD5C.1000008@gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>
	<4ABFE075.2070309@voidspace.org.uk>
	<F5B27C82-50C3-4838-B507-53DF82C35D4C@gmail.com>
	<4AC0AD5C.1000008@gmail.com>
Message-ID: <19136.52362.562678.628471@montanaro.dyndns.org>


    Nick> +1 here as well (although we should definitely find a way to use
    Nick> str.format strings instead of %-format ones... come to think of
    Nick> it, does even the logging module support str.format style
    Nick> formatting in Py3k?)

Assuming argparse currently works with versions of Python < 2.6 I see no
reason to make such a change.  This would just introduce needless
differences between the version delivered with Python and the PyPI version
and make it more difficult for the author to keep the two code bases in
sync.

Skip

From guido at python.org  Mon Sep 28 16:56:29 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Sep 2009 07:56:29 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <e9764b730909280742l64280d1dwa252191ea9c031a4@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> 
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net> 
	<Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net> 
	<4AC0AB17.5070801@gmail.com>
	<eae285400909280704w39520754h53e31c6ed594332@mail.gmail.com> 
	<e9764b730909280742l64280d1dwa252191ea9c031a4@mail.gmail.com>
Message-ID: <ca471dc20909280756k31eb3e1x82f5cda8780d8e4c@mail.gmail.com>

On Mon, Sep 28, 2009 at 7:42 AM, Dj Gilcrease <digitalxero at gmail.com> wrote:
> On Mon, Sep 28, 2009 at 8:04 AM, Daniel Stutzbach
> <daniel at stutzbachenterprises.com> wrote:
>> On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>
>>> I should note that I've softened my position slightly from what I posted
>>> yesterday. I could live with the following compromise:
>>>
>>> ? ?>>> x = IPv4Network('192.168.1.1/24')
>>> ? ?>>> y = IPv4Network('192.168.1.0/24')
>>> ? ?>>> x == y # Equality is the part I really want to see changed
>>> ? ?True
>>> ? ?>>> x.ip
>>> ? ?IPv4Address('192.168.1.1')
>>> ? ?>>> y.ip
>>> ? ?IPv4Address('192.168.1.0')
>>
>> With those semantics, IPv4Network objects with distinct IP addresses (but
>> the same network) could no longer be stored in a dictionary or set.? IMO, it
>> is a little counter-intuitive for objects to compare equal yet have
>> different properties.? I don't think this is a good compromise.
>
> Thats not true, the patch I submitted
> http://codereview.appspot.com/124057 still allows the networks to be
> included in a set or as a dict key
>
>>>> net1 = IPNetwork("10.1.2.3/24")
>>>> net2 = IPNetwork("10.1.2.0/24")
>>>> print hash(net1) == hash(net2)
> False
>>>> print net1 == net2
> True

Hold it right there! That's wrong. You can't have two objects that
compare equal but whose hashes differ. It will break dict lookup. The
other way around is fine: two objects may differ and still have the
same hash.

>>>> test = {net1: "something", net2: "something else"}
>>>> print test
> {IPv4Network('10.1.2.0/24'): 'something else',
> IPv4Network('10.1.2.3/24'): 'something'}
>>>> test2 = set([net1, net2])
>>>> print test2
> set([IPv4Network('10.1.2.0/24'), IPv4Network('10.1.2.3/24')])
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Mon Sep 28 16:57:28 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 28 Sep 2009 15:57:28 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <e9764b730909280742l64280d1dwa252191ea9c031a4@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net>
	<4AC0AB17.5070801@gmail.com>
	<eae285400909280704w39520754h53e31c6ed594332@mail.gmail.com>
	<e9764b730909280742l64280d1dwa252191ea9c031a4@mail.gmail.com>
Message-ID: <5c6f2a5d0909280757n4e79e1a9laf49f4991e43345e@mail.gmail.com>

On Mon, Sep 28, 2009 at 3:42 PM, Dj Gilcrease <digitalxero at gmail.com> wrote:
> On Mon, Sep 28, 2009 at 8:04 AM, Daniel Stutzbach
> <daniel at stutzbachenterprises.com> wrote:
>> On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>
>>> I should note that I've softened my position slightly from what I posted
>>> yesterday. I could live with the following compromise:
>>>
>>> ? ?>>> x = IPv4Network('192.168.1.1/24')
>>> ? ?>>> y = IPv4Network('192.168.1.0/24')
>>> ? ?>>> x == y # Equality is the part I really want to see changed
>>> ? ?True
>>> ? ?>>> x.ip
>>> ? ?IPv4Address('192.168.1.1')
>>> ? ?>>> y.ip
>>> ? ?IPv4Address('192.168.1.0')
>>
>> With those semantics, IPv4Network objects with distinct IP addresses (but
>> the same network) could no longer be stored in a dictionary or set.? IMO, it
>> is a little counter-intuitive for objects to compare equal yet have
>> different properties.? I don't think this is a good compromise.
>
> Thats not true, the patch I submitted
> http://codereview.appspot.com/124057 still allows the networks to be
> included in a set or as a dict key
>
>>>> net1 = IPNetwork("10.1.2.3/24")
>>>> net2 = IPNetwork("10.1.2.0/24")
>>>> print hash(net1) == hash(net2)
> False
>>>> print net1 == net2
> True

In that case, your patch breaks the rather fundamental rule that
Python objects that compare equal should have equal hash.  :-)

Relying on hashes to be distinct isn't safe.

Mark

From solipsis at pitrou.net  Mon Sep 28 16:57:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Sep 2009 14:57:14 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net>
	<4AC0AB17.5070801@gmail.com>
	<eae285400909280704w39520754h53e31c6ed594332@mail.gmail.com>
	<e9764b730909280742l64280d1dwa252191ea9c031a4@mail.gmail.com>
Message-ID: <loom.20090928T164939-234@post.gmane.org>

Dj Gilcrease <digitalxero <at> gmail.com> writes:
> 
> >>> net1 = IPNetwork("10.1.2.3/24")
> >>> net2 = IPNetwork("10.1.2.0/24")
> >>> print hash(net1) == hash(net2)
> False
> >>> print net1 == net2
> True

Ouch :-)
It is very bad practice for equal objects to hash differently. Differing hashes
are supposed to imply non-equal objects, so as to allow the implementation of
associative containers with good performance characteristics.

Regards

Antoine.



From daniel at stutzbachenterprises.com  Mon Sep 28 17:01:38 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 28 Sep 2009 10:01:38 -0500
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909281000200.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<h8sd2n$n1g$1@ger.gmane.org> <4ABE530D.8090609@v.loewis.de>
	<200909271108.32456.steve@pearwood.info>
	<4ABF0CA8.2030408@v.loewis.de>
	<9118C602-C5D3-49A9-B07F-CE61723D4D6A@gmail.com>
	<Pine.LNX.4.64.0909281000200.18193@kimball.webabinitio.net>
Message-ID: <eae285400909280801g9a5137bsd0f0aa6ade1466d@mail.gmail.com>

On Mon, Sep 28, 2009 at 9:35 AM, R. David Murray <rdmurray at bitdance.com>wrote:

> A little googling produced this hit:
>
>
> http://tools.ietf.org/wg/mip4/draft-ietf-mip4-nemo-v4-base/draft-ietf-mip4-nemo-v4-base-11-from-10.diff.txt
>
> which while not a standard itself clearly believes that non-contiguous
> netmasks are valid, and is dated 2008.  Earlier RFCs (950 and 1219)
> give them as valid but discouraged.
>

That's a draft for RFC 5177.  In the RFC, all references to non-contiguous
subnets have been removed.

http://tools.ietf.org/html/rfc5177

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090928/1887a254/attachment.htm>

From mal at egenix.com  Mon Sep 28 17:03:55 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 28 Sep 2009 17:03:55 +0200
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <loom.20090928T024853-85@post.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<loom.20090928T024853-85@post.gmane.org>
Message-ID: <4AC0D05B.2060506@egenix.com>

Antoine Pitrou wrote:
> 
> Hello,
> 
> I am neutral on the idea of adding argparse. However, I'm -1 on deprecating
> optparse. It is very widely used (tons of scripts use it), and ok for many uses;
> deprecating it is totally unhelpful and gratuitous.

You can add me to that camp as well:

+0 on adding argparse
-1 on deprecating wide-used modules such as getopt and optparse

Note that the last -1 only applies to the module APIs, not the
code in those modules.

If argparse provides an API compatibility layer for getopt and
optparse, the original code of getopt and optparse could go
without problem, IMHO, leaving only the mostly empty modules
together with compatibility layers.

OTOH, as discussed on the stdlib SIG list, those two modules
hardly ever require any maintenance, so keeping them as they
are is certainly an option as well.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 28 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From fuzzyman at voidspace.org.uk  Mon Sep 28 17:08:48 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 28 Sep 2009 16:08:48 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC0D05B.2060506@egenix.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<loom.20090928T024853-85@post.gmane.org>
	<4AC0D05B.2060506@egenix.com>
Message-ID: <4AC0D180.9030607@voidspace.org.uk>

M.-A. Lemburg wrote:
> Antoine Pitrou wrote:
>   
>> Hello,
>>
>> I am neutral on the idea of adding argparse. However, I'm -1 on deprecating
>> optparse. It is very widely used (tons of scripts use it), and ok for many uses;
>> deprecating it is totally unhelpful and gratuitous.
>>     
>
> You can add me to that camp as well:
>
> +0 on adding argparse
> -1 on deprecating wide-used modules such as getopt and optparse
>
> Note that the last -1 only applies to the module APIs, not the
> code in those modules.
>
> If argparse provides an API compatibility layer for getopt and
> optparse, the original code of getopt and optparse could go
> without problem, IMHO, leaving only the mostly empty modules
> together with compatibility layers.
>
> OTOH, as discussed on the stdlib SIG list, those two modules
> hardly ever require any maintenance, so keeping them as they
> are is certainly an option as well.
>
>   
Although as also discussed on the stdlib-sig there is also a cost to 
leaving unmaintained and unneeded modules in the standard library. As we 
are 'end-of-lifing' Python 2.X there seems to be little point in going 
down the deprecation route (there won't be enough releases to ever get 
to module removal) but discussion about the standard library in 
situations like this is badly needed for 3.X.

Certainly a slow deprecation procedure, as suggested by Steven, is 
warranted.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From steve at pearwood.info  Mon Sep 28 17:13:41 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 29 Sep 2009 01:13:41 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC03410.4010104@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
Message-ID: <200909290113.41871.steve@pearwood.info>

On Mon, 28 Sep 2009 01:57:04 pm Martin v. L?wis wrote:
> >> Finally, to Stephen's point about seeing the other side of the
> >> argument, I wrote this offlist a week ago:
> >>
> >>   I *understand* what you're saying, I *understand* that
> >> 192.168.1.1/24 isn't a network,
> >
> > But you still want to treat it as one.
> >
> > Could you explain what benefit there is for allowing the user to
> > create network objects that don't represent networks? Is there a
> > use-case where these networks-that-aren't-networks are something
> > other than a typo? Under what circumstances would I want to specify
> > a network as 192.168.1.1/24 instead of 192.168.1.0/24?
>
> It's fairly obvious to me why the library should support
> 192.168.1.1/24 as an input, and return a network.
>
> End-users are likely going to enter such things (e.g.
> 82.94.164.162/29), as they will know one IP address in the network
> (namely, of one machine that they are looking at), and they will know
> the prefix length (more often, how large the network is - 8 or 16 or
> 32). So very clearly, end users should not be required to make the
> computation in their heads.

Thank you Martin.

I had somehow -- justifiably or not, it doesn't matter -- got the 
impression from Peter off-list that this was *not* a use-case he gave 
any credence to, but of course the current API supports it, which 
confused me no end.

I'm not sure that just because the user enters '82.94.164.162/29' into 
the UI, the API must accept it. As Jean-Paul suggests, the issue of 
dealing with user-input is different from API aimed at developers. 
There are at least three options:

(1) The status quo: IPv4Network accepts the host address plus mask, and 
returns an object which behaves like a network but remembers the host 
address.

(2) Like (1), but the address is normalised to the canonical network 
address (which if I've calculated it correctly should be 
'82.94.164.160').

In this case, whether the user enters '82.94.164.162/29' 
or '82.94.164.169/29', the network returned will be the same. If the 
developer wants to save the host address as well as the network for 
some reason, he should store the host address as a separate object.

(3) Like (2), except the constructor is strict and only accepts the 
canonical network address, and will raise an exception if 
given '82.94.164.162/29'. This may require at least one helper function 
to calculate the network/mask from the address/mask.


The PEP now states:

"It should be understood that, above all, this module is designed with 
the network administrator in mind.  In practice, this means that a 
number of assumptions are made with regards to common usage and the 
library prefers the usefulness of accepted practice over strict 
adherence to RFCs. For example, ipaddr accepts '192.168.1.1/24' as a 
network definition because this is a very common way of describing an 
address + netmask despite the fact that 192.168.1.1 is actually an IP 
address on the network 192.168.1.0/24. ..."

which nicely explains why (3) is not used, but doesn't explain why (1) 
should be preferred over (2).



-- 
Steven D'Aprano

From foom at fuhm.net  Mon Sep 28 17:13:47 2009
From: foom at fuhm.net (James Y Knight)
Date: Mon, 28 Sep 2009 11:13:47 -0400
Subject: [Python-Dev] please consider changing --enable-unicode default
	to	ucs4
In-Reply-To: <4AC07309.1010608@egenix.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>	<4AB67430.2060306@egenix.com>
	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
	<4AC07309.1010608@egenix.com>
Message-ID: <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net>

On Sep 28, 2009, at 4:25 AM, M.-A. Lemburg wrote:
> Distributions should really not be put in charge of upstream
> coding design decisions.

I don't think you can blame distros for this one....

 From PEP 0261:
     It is also proposed that one day --enable-unicode will just
     default to the width of your platforms wchar_t.

On linux, wchar_t is 4 bytes.

If there's a consensus amongst python upstream that all the distros  
should be shipping Python with UCS2 unicode strings, you should reach  
out to them and say this, in a rather more clear fashion. Currently,  
most signs point towards UCS4 builds as being the better option.

Or, one might reasonably wonder why UCS-4 is an option at all, if  
nobody should enable it.

> People building their own Python version will usually also build
> their own extensions, so I don't really believe that the above
> scenario is very common.

I'd just like to note that I've run into this trap multiple times. I  
built a custom python, and expected it to work with all the existing,  
installed, extensions (same major version as the system install, just  
patched). And then had to build it again with UCS4, for it to actually  
work. Of course building twice isn't the end of the world, and I'm  
certainly used to having to twiddle build options on software to get  
it working, but, this *does* happen, and *is* a tiny bit irritating.

James

From ubershmekel at gmail.com  Mon Sep 28 17:18:45 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Mon, 28 Sep 2009 18:18:45 +0300
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
	<1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>
	<d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
	<d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>
	<9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com>
	<20090928083820.GA488@laurie.devork>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
Message-ID: <9d153b7c0909280818v30760a44p11ab24ba9251d4ef@mail.gmail.com>

>
> * Would you be opposed to a note in the getopt documentation suggesting
> argparse as an alternative?


from the top of http://docs.python.org/library/getopt.html - "A more
convenient, flexible, and powerful alternative is the optparse module."I
think this statement should be emphasized better but it's already there so
no worries.

* Would you like argparse to grow an add_getopt_arguments method (as in my
> other post)?

* If argparse grew an add_getopt_arguments, would you still want to keep
> getopt around? And if so, why?


Argparse/optparse use a parser instance and getopt doesn't. I think that's a
saved line you have to keep saving so this would be the better syntax imho:
    optlist, args = argparse.getopt(args, 'abc:d:')

I was thinking on the general issue of what's better for the std-lib:
1. Mini modules - 3+ small modules in each subject that can be useful in
different contexts. For example the xml libs.
or
2. Big modules - 1 uber-module for each subject that does everything. Maybe
logging is an example of this.

I think in general python std-lib went the path of mini modules. Its
advantages
1. Each mini module can be coherent and while python isn't really
OnlyOneWayToDoIt, each module in itself has OnlyOneWayToDoIt.
2. Documentation is less cluttered by a large amount of functions that all
do the same thing but differently. This is very important for python's
learning curve.

disadvantages:
1. You have to craft your documentation carefully as to be very explicit on
why you have all these modules for the same thing. Probably each module
should explain it's purpose in relation to the other modules and this
explanation should appear at the top of the documentation of each of the
mini modules in the subject. Today it would be very hard for me to figure
out how the xml modules interrelate, urllib has improved in python3 but
still isn't perfect.
2. I could be using some silly way of doing things (getopt) while I really
wanted to do it the good way (argparse).

I think that while adding getopt functionality to argparse might be nice but
would eventually cause clutter.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090928/6c30da8f/attachment.htm>

From sesquile at gmail.com  Mon Sep 28 17:23:22 2009
From: sesquile at gmail.com (m h)
Date: Mon, 28 Sep 2009 09:23:22 -0600
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
Message-ID: <e36b84ee0909280823r7747ec4fy7a7def5a32591607@mail.gmail.com>

Perhaps this is OT, but since command line parsing is part of
configuration, I figure I'd throw it out there.  My scripts often have
configuration that the command line can override and I loosely follow
the example hierarchy[0] listed in The Art of Unix Programming.

Some configuration I want in a config file (but I want to override
from the command line) and sometimes it's very nice to use environment
variables for configuration.  So I do something like this:


Chaining Configuration
----------------------

Ugly code to cascade configuration

>>> class Unset(object): pass
>>> def cascade_value(opt=None, opt_name=None, env_name=None, cfg=None, cfg_section=None, cfg_name=None, default=None):
...    """
...    opt - result of OptionParser.parse_args
...    opt_name - string of opt name you want to access
...    """
...    # get from cmd line
...    value = Unset()
...    if opt and opt_name:
...        try:
...            value = opt.__getattr__(opt_name)
...        except AttributeError, e:
...            pass
...    if not isinstance(value, Unset):
...        return value
...    # get from ENV
...    if env_name:
...        try:
...            value = os.environ[env_name]
...        except KeyError, e:
...            pass
...    if not isinstance(value, Unset):
...        return value
...    # get from config file
...    if cfg and cfg_section and cfg_name:
...        try:
...            value = cfg.get(cfg_section, cfg_name)
...        except ConfigParser.NoOptionError, e:
...            pass
...    if not isinstance(value, Unset):
...        return value
...    return default


>>> cascade_value(opt=opt, opt_name='author', cfg=cfg, cfg_section='Properties', cfg_name='author')
'Matt'

Does anyone else have interest in such functionality?  Is it outside
the realm of this PEP?  Ideally I'd like to have something like
``env_name``, ``config_key`` named parameters for options, then pass
an optional list of configs files to the parser instance.  Then rather
than looking solely at command line options, argparse could also pull
values from config files, then env and finally the command line.

cheers,

matt harrison


0 - http://www.faqs.org/docs/artu/ch10s02.html

From p.f.moore at gmail.com  Mon Sep 28 17:25:54 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 28 Sep 2009 16:25:54 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
Message-ID: <79990c6b0909280825q57fce79ej6b6c390699c3f32d@mail.gmail.com>

2009/9/28 Peter Moody <peter at hda3.com>:
>> That is, you've rejected the idea of having:
>>
>>>>> IPv4Network(192.168.1.1/24)
>> IPv4Network(192.168.1.0/24)
>
> yes, I and everyone have rejected that idea. this should either raise
> an  error, or do what it does now, that is, return
> IPv4Network('192.168.1.1/24')

Speaking as a member of the group "everyone", I have not rejected that
idea. It seems perfectly sensible to me. Yes, it "loses" the
information about the full IP address 192.168.1.1, but in my view,
that's what I asked it to do by requesting a network object back.

2009/9/28 R. David Murray <rdmurray at bitdance.com>:
> The fundamental divide here is between two behaviors.
>
> ipaddr:
>
> ? ?>>> x = IPv4Network('192.168.1.1/24')
> ? ?>>> y = IPv4Network('192.168.1.0/24')
> ? ?>>> x == y
> ? ?False
> ? ?>>> x.ip
> ? ?IPv4Address('192.168.1.1')
>
> desired:
>
> ? ?>>> x = IPv4Network('192.168.1.1/24')
> ? ?>>> y = IPv4Network('192.168.1.0/24')
> ? ?>>> x == y
> ? ?True
> ? ?>>> x.ip
> ? ?Traceback (most recent call last):
> ? ? ?File "<stdin>", line 1, in <module>
> ? ?AttributeError: 'IPv4Network' object has no attribute 'ip'
>
> Everything else is pretty much bikeshedding and can be dealt with. ?This
> is fundamental and Peter has indicated he will not change it.

Agreed. I would prefer the option marked "desired". As a naive "casual
user" of the module, I claim that the current behaviour is
non-intuitive and I anticipate it causing me issues (not many, and I
won't use the module often, so I concede - before someone asks - that
it'll be a minor inconvenience to me).

Given that I am a minor user, as noted above, I remain -1 on
inclusion, largely because it feels to me like the module will suffer
the same fate as optparse - rejecting certain changes which have
relatively widespread support on essentially philosophical grounds.

Paul.

From fuzzyman at voidspace.org.uk  Mon Sep 28 17:26:52 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 28 Sep 2009 16:26:52 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <e36b84ee0909280823r7747ec4fy7a7def5a32591607@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<e36b84ee0909280823r7747ec4fy7a7def5a32591607@mail.gmail.com>
Message-ID: <4AC0D5BC.40108@voidspace.org.uk>

m h wrote:
> Perhaps this is OT, but since command line parsing is part of
> configuration, I figure I'd throw it out there.  My scripts often have
> configuration that the command line can override and I loosely follow
> the example hierarchy[0] listed in The Art of Unix Programming.
>
> Some configuration I want in a config file (but I want to override
> from the command line) and sometimes it's very nice to use environment
> variables for configuration.  So I do something like this:
>
>   

Integration with command line options is an occasionally-requested 
feature for ConfigObj. I've always said I would be open to patches...

In other words, yes I think there is demand for it. Whether it belongs 
*immediately* in the standard library is another matter, but if you 
wrote a layer that unified ConfigParser and argparse I think you will 
find that people use it.

It is well outside the realm of this PEP however.

All the best,

Michael Foord

> Chaining Configuration
> ----------------------
>
> Ugly code to cascade configuration
>
>   
>>>> class Unset(object): pass
>>>> def cascade_value(opt=None, opt_name=None, env_name=None, cfg=None, cfg_section=None, cfg_name=None, default=None):
>>>>         
> ...    """
> ...    opt - result of OptionParser.parse_args
> ...    opt_name - string of opt name you want to access
> ...    """
> ...    # get from cmd line
> ...    value = Unset()
> ...    if opt and opt_name:
> ...        try:
> ...            value = opt.__getattr__(opt_name)
> ...        except AttributeError, e:
> ...            pass
> ...    if not isinstance(value, Unset):
> ...        return value
> ...    # get from ENV
> ...    if env_name:
> ...        try:
> ...            value = os.environ[env_name]
> ...        except KeyError, e:
> ...            pass
> ...    if not isinstance(value, Unset):
> ...        return value
> ...    # get from config file
> ...    if cfg and cfg_section and cfg_name:
> ...        try:
> ...            value = cfg.get(cfg_section, cfg_name)
> ...        except ConfigParser.NoOptionError, e:
> ...            pass
> ...    if not isinstance(value, Unset):
> ...        return value
> ...    return default
>
>
>   
>>>> cascade_value(opt=opt, opt_name='author', cfg=cfg, cfg_section='Properties', cfg_name='author')
>>>>         
> 'Matt'
>
> Does anyone else have interest in such functionality?  Is it outside
> the realm of this PEP?  Ideally I'd like to have something like
> ``env_name``, ``config_key`` named parameters for options, then pass
> an optional list of configs files to the parser instance.  Then rather
> than looking solely at command line options, argparse could also pull
> values from config files, then env and finally the command line.
>
> cheers,
>
> matt harrison
>
>
> 0 - http://www.faqs.org/docs/artu/ch10s02.html
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From steve at pearwood.info  Mon Sep 28 17:27:02 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 29 Sep 2009 01:27:02 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
Message-ID: <200909290127.03215.steve@pearwood.info>

On Tue, 29 Sep 2009 12:28:39 am Steven Bethard wrote:

> Ok, sounds like there are a number of supporters for keeping getopt
> around and just deprecating optparse. For those who'd like to keep
> getopt around, I have a few questions:
>
> * Would you be opposed to a note in the getopt documentation
> suggesting argparse as an alternative?

+1

> * Would you like argparse to grow an add_getopt_arguments method (as
> in my other post)?

0

> * If argparse grew an add_getopt_arguments, would you still want to
> keep getopt around? And if so, why?

Simplicity of the learning curve. Using it is as simple as:

getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])

Does argparse allow anything as simple as that?




-- 
Steven D'Aprano

From phd at phd.pp.ru  Mon Sep 28 17:33:35 2009
From: phd at phd.pp.ru (Oleg Broytman)
Date: Mon, 28 Sep 2009 19:33:35 +0400
Subject: [Python-Dev] Application configuration (was: PEP 389: argparse)
In-Reply-To: <e36b84ee0909280823r7747ec4fy7a7def5a32591607@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<e36b84ee0909280823r7747ec4fy7a7def5a32591607@mail.gmail.com>
Message-ID: <20090928153335.GA4747@phd.pp.ru>

On Mon, Sep 28, 2009 at 09:23:22AM -0600, m h wrote:
> Perhaps this is OT, but since command line parsing is part of
> configuration, I figure I'd throw it out there.  My scripts often have
> configuration that the command line can override and I loosely follow
> the example hierarchy[0] listed in The Art of Unix Programming.
> 
> Some configuration I want in a config file (but I want to override
> from the command line) and sometimes it's very nice to use environment
> variables for configuration.

   The idea was discussed in the list many years ago but it seems to be
forgotten now.

> Does anyone else have interest in such functionality?  Is it outside
> the realm of this PEP?

   It is outside the scope of this particular PEP, but it is certainly an
interesting idea.
   Actually it is the Right Thing, but I doubt it is possible to implement
such a generic configuration *framework* in the Python standard *library*.
Applications use different configfile layouts, some programs use w32
registry instead of configfiles...

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From jnoller at gmail.com  Mon Sep 28 17:35:26 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 28 Sep 2009 10:35:26 -0500
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>
Message-ID: <4222a8490909280835va690e38u2f5d97a35bf9fce0@mail.gmail.com>

On Sun, Sep 27, 2009 at 4:57 PM, Brett Cannon <brett at python.org> wrote:
> I am going to state upfront that I am +1 for this and I encouraged
> Steven to submit this PEP on the stdlib-SIG. I still remember watching
> Steven's lightning talk at PyCon 2009 on argparse and being impressed
> by it (along with the rest of the audience which was obviously
> impressed as well).
>
> I think argprase is a good improvement over what we have now (getopt
> and optparse), it's stable, considered best-of-breed by the community
> for a while (as shown as how many times argparse has been suggestion
> for inclusion), and Steven is already a core committer so is already
> set to maintain the code. That covers the usual checklist we have for
> even looking at a PEP to add a module to the standard library.

+1 to what brett said.

jesse

From steven.bethard at gmail.com  Mon Sep 28 17:37:16 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 28 Sep 2009 08:37:16 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC0D5BC.40108@voidspace.org.uk>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<e36b84ee0909280823r7747ec4fy7a7def5a32591607@mail.gmail.com>
	<4AC0D5BC.40108@voidspace.org.uk>
Message-ID: <d11dcfba0909280837h633cc615geb09dde15b08697c@mail.gmail.com>

On Mon, Sep 28, 2009 at 8:26 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> m h wrote:
>>
>> Perhaps this is OT, but since command line parsing is part of
>> configuration, I figure I'd throw it out there. ?My scripts often have
>> configuration that the command line can override and I loosely follow
>> the example hierarchy[0] listed in The Art of Unix Programming.
>>
>> Some configuration I want in a config file (but I want to override
>> from the command line) and sometimes it's very nice to use environment
>> variables for configuration. ?So I do something like this:
>
> Integration with command line options is an occasionally-requested feature
> for ConfigObj. I've always said I would be open to patches...
>
> In other words, yes I think there is demand for it. Whether it belongs
> *immediately* in the standard library is another matter, but if you wrote a
> layer that unified ConfigParser and argparse I think you will find that
> people use it.
>
> It is well outside the realm of this PEP however.

What Michael said. ;-) If you'd like to provide a patch for argparse
that provides such functionality, I'd be happy to review it.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From digitalxero at gmail.com  Mon Sep 28 17:30:05 2009
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Mon, 28 Sep 2009 09:30:05 -0600
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909280756k31eb3e1x82f5cda8780d8e4c@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
	<Pine.LNX.4.64.0909280747030.18193@kimball.webabinitio.net>
	<4AC0AB17.5070801@gmail.com>
	<eae285400909280704w39520754h53e31c6ed594332@mail.gmail.com>
	<e9764b730909280742l64280d1dwa252191ea9c031a4@mail.gmail.com>
	<ca471dc20909280756k31eb3e1x82f5cda8780d8e4c@mail.gmail.com>
Message-ID: <e9764b730909280830m72166babmc22e9ffc36d30c02@mail.gmail.com>

On Mon, Sep 28, 2009 at 8:56 AM, Guido van Rossum <guido at python.org> wrote:
> Hold it right there! That's wrong. You can't have two objects that
> compare equal but whose hashes differ. It will break dict lookup. The
> other way around is fine: two objects may differ and still have the
> same hash.


mm I never knew that, guess I'll go fix my patch to change the
__hash__ method as well, though when I tested it dict lookup seemed to
work fine


>>> net1 = IPNetwork("10.1.2.0/29")
>>> i = 0
>>> test = {}
>>> for ip in net1:
	test[IPNetwork("%s/29" %(ip))] = i
	i += 1

	
>>> print test[IPv4Network('10.1.2.4/29')]
4
>>> print test
{IPv4Network('10.1.2.0/29'): 0, IPv4Network('10.1.2.1/29'): 1,
IPv4Network('10.1.2.2/29'): 2, IPv4Network('10.1.2.3/29'): 3,
IPv4Network('10.1.2.4/29'): 4, IPv4Network('10.1.2.5/29'): 5,
IPv4Network('10.1.2.6/29'): 6, IPv4Network('10.1.2.7/29'): 7}
>>>

From steven.bethard at gmail.com  Mon Sep 28 17:49:15 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 28 Sep 2009 08:49:15 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <200909290127.03215.steve@pearwood.info>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
Message-ID: <d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>

On Mon, Sep 28, 2009 at 8:27 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Tue, 29 Sep 2009 12:28:39 am Steven Bethard wrote:
>> * Would you like argparse to grow an add_getopt_arguments method (as
>> in my other post)?
>
> 0
>
>> * If argparse grew an add_getopt_arguments, would you still want to
>> keep getopt around? And if so, why?
>
> Simplicity of the learning curve. Using it is as simple as:
>
> getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])

You forgot the for-loop, nested if/else statements and type conversions. ;-)

> Does argparse allow anything as simple as that?

If you mean, does argparse allow configuration to be specified using
the getopt style ("a:b", ["alpha=", "beta"]), no, it currently
doesn't. But if this is useful functionality, and would reasonably
replace the getopt use cases, then I'd be happy to add it. In the
simplest version, we might add something like::

    options, args = argparse.getopt("a:b", ["alpha=", "beta"])

where you could then use "options" without any looping::

    alpha = options.a or options.alpha
    beta = options.b or options.beta

But if people still like the traditional getopt loop with nested
if/elses better, then we might as well just keep getopt around and not
add anything to argparse. I'm fine with that too, I just want to make
sure that there isn't an obvious deficiency in argparse that we could
easily fix.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From floris.bruynooghe at gmail.com  Mon Sep 28 17:54:52 2009
From: floris.bruynooghe at gmail.com (Floris Bruynooghe)
Date: Mon, 28 Sep 2009 16:54:52 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
References: <d11dcfba0909271955u1d514691n58fe12f4afaf7b32@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
	<1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>
	<d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
	<d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>
	<9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com>
	<20090928083820.GA488@laurie.devork>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
Message-ID: <20090928155452.GB13277@laurie.devork>

On Mon, Sep 28, 2009 at 07:28:39AM -0700, Steven Bethard wrote:
> Ok, sounds like there are a number of supporters for keeping getopt
> around and just deprecating optparse. For those who'd like to keep
> getopt around, I have a few questions:
> 
> * Would you be opposed to a note in the getopt documentation
> suggesting argparse as an alternative?

No, I'd encourage such a note.

> * Would you like argparse to grow an add_getopt_arguments method (as
> in my other post)?

That looks nice indeed.

> * If argparse grew an add_getopt_arguments, would you still want to
> keep getopt around? And if so, why?

If clearly documented as "Hey, getopt users, check this out!" then I'd
be indifferent to the proposed deprecation of getopt.

Regards
Floris

From olemis at gmail.com  Mon Sep 28 18:03:50 2009
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 28 Sep 2009 11:03:50 -0500
Subject: [Python-Dev] Application configuration (was: PEP 389: argparse)
In-Reply-To: <20090928153335.GA4747@phd.pp.ru>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<e36b84ee0909280823r7747ec4fy7a7def5a32591607@mail.gmail.com>
	<20090928153335.GA4747@phd.pp.ru>
Message-ID: <24ea26600909280903v106403a7y23e1839aba40d83f@mail.gmail.com>

On Mon, Sep 28, 2009 at 10:33 AM, Oleg Broytman <phd at phd.pp.ru> wrote:
> On Mon, Sep 28, 2009 at 09:23:22AM -0600, m h wrote:
>
>> Does anyone else have interest in such functionality? ?Is it outside
>> the realm of this PEP?
>
> ? It is outside the scope of this particular PEP, but it is certainly an
> interesting idea.

:)

> ? Actually it is the Right Thing, but I doubt it is possible to implement
> such a generic configuration *framework* in the Python standard *library*.
> Applications use different configfile layouts, some programs use w32
> registry instead of configfiles...
>

If this is the only reason :

Q:
  - What if only INI files are considered ?

In the end, that's the format included in stdlib and used by stdlib
commands (e.g. distutils ;o), and also by others (e.g. Trac, Hg, ...).
Reducing the scope in this direction could yield good results.

OTOH, if other cfg need to be supported, then possibly an ?object
layer? (e.g. compatible with ConfigParser) could be used in order to
support further config layouts built outside stdlib. For instance, in
the case of WinReg :

{{{
#!python

RegParser = RegConfigParser(HKEY_CURRENT_USER, r'Software\MyApp\xxx')
RegParser.sections()                 # Children of this reg key
RegParser.options('s1')              # Values under
                                               #
HKEY_CURRENT_USER\Software\MyApp\xxx\s1
...
}}}

... and similar solutions could be implemented outside stdlib for e.g. YAML, ...

PS: BTW, is there something like this already for WinReg ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Sobrepasa las 300 descargas el m?dulo dutest  -
http://feedproxy.google.com/~r/simelo-es/~3/U5rff5iTQPI/sobrepasa-las-300-descargas-el-modulo.html

From mal at egenix.com  Mon Sep 28 18:12:56 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 28 Sep 2009 18:12:56 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to	ucs4
In-Reply-To: <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>	<4AB67430.2060306@egenix.com>	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>	<4AC07309.1010608@egenix.com>
	<4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net>
Message-ID: <4AC0E088.8080000@egenix.com>

James Y Knight wrote:
> On Sep 28, 2009, at 4:25 AM, M.-A. Lemburg wrote:
>> Distributions should really not be put in charge of upstream
>> coding design decisions.
> 
> I don't think you can blame distros for this one....
>
> From PEP 0261:
>     It is also proposed that one day --enable-unicode will just
>     default to the width of your platforms wchar_t.
> 
> On linux, wchar_t is 4 bytes.

The PEP also has this to say:

    This has the effect of doubling the size of most Unicode
    strings. In order to avoid imposing this cost on every
    user, Python 2.2 will allow the 4-byte implementation as a
    build-time option. Users can choose whether they care about
    wide characters or prefer to preserve memory.

And that's still true today. It was the main reason for not
making it the default on those days. Today, Python 3.x
uses Unicode for all strings, so while the RAM situation has
changed somewhat since Python 2.2, the change has a much
wider effect on the Python memory foot-print than in late 2001.

> If there's a consensus amongst python upstream that all the distros
> should be shipping Python with UCS2 unicode strings, you should reach
> out to them and say this, in a rather more clear fashion. Currently,
> most signs point towards UCS4 builds as being the better option.

UCS4 is the better option if you use lots of non-BMP code points
and if you have to regularly interface with C APIs using wchar_t
on Unix.

> Or, one might reasonably wonder why UCS-4 is an option at all, if nobody
> should enable it.

See above: there are use cases where this does make a lot of sense.

E.g. non-BMP code points can only be represented using surrogates on
UCS2 builds and these can be tricky to deal with (or at least
many people feel like it's tricky to deal with them ;-).

>> People building their own Python version will usually also build
>> their own extensions, so I don't really believe that the above
>> scenario is very common.
> 
> I'd just like to note that I've run into this trap multiple times. I
> built a custom python, and expected it to work with all the existing,
> installed, extensions (same major version as the system install, just
> patched). And then had to build it again with UCS4, for it to actually
> work. Of course building twice isn't the end of the world, and I'm
> certainly used to having to twiddle build options on software to get it
> working, but, this *does* happen, and *is* a tiny bit irritating.

Which is why I think that Python should include some more information
on the type of built being used, e.g. by placing the information
prominently on the startup line.

I still don't believe the above use case is a common one, though.

That said, Zooko's original motivation for the proposed change
is making installation of extensions easier for users. That's
a tools question much more than a Python Unicode one.

Aside: --enable-unicode is gone in Python 3.x. You now only
have the choice to use the default (which is UCS2) or switch on
the optional support for UCS4 by using --with-wide-unicode.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 28 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From digitalxero at gmail.com  Mon Sep 28 18:24:30 2009
From: digitalxero at gmail.com (Dj Gilcrease)
Date: Mon, 28 Sep 2009 10:24:30 -0600
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <79990c6b0909280825q57fce79ej6b6c390699c3f32d@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
	<79990c6b0909280825q57fce79ej6b6c390699c3f32d@mail.gmail.com>
Message-ID: <e9764b730909280924ib3ff2e2u4eaed4e3ff10cf89@mail.gmail.com>

Ok updated patch so it meets the following conditions

>>> IPv4Network(192.168.1.1/24)
IPv4Network(192.168.1.0/24)

>>> x = IPv4Network('192.168.1.1/24')
>>> y = IPv4Network('192.168.1.0/24')
>>> x == y
True
>>> hash(x) == hash(y)
True

The ip you instantiated the Network with is still accessible though, so
>>> x.ip
IPv4Address('192.168.1.1')
>>> y.ip
IPv4Address('192.168.1.0')

I left it this way since there seems to be conflicting opinions on if
this should exist in the Network object or not

http://codereview.appspot.com/124057

From guido at python.org  Mon Sep 28 18:39:06 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Sep 2009 09:39:06 -0700
Subject: [Python-Dev] PEP 3144 review, and the inclusion process
In-Reply-To: <loom.20090928T022351-635@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> 
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> 
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> 
	<loom.20090928T000905-169@post.gmane.org>
	<8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> 
	<loom.20090928T022351-635@post.gmane.org>
Message-ID: <ca471dc20909280939o75ed5ddbp91d5854175b35fd7@mail.gmail.com>

On Sun, Sep 27, 2009 at 5:32 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Peter Moody <peter <at> hda3.com> writes:
>>
>> I've never said otherwise. In fact, from an email last night, "If what
>> the community requires is the library you've described, then ipaddr is
>> not that library." The changes *you* require make ipaddr significantly
>> less useful to me. I'm not prepared to make those changes in an
>> attempt seek acceptance to the stdlib, especially if the stdlib is in
>> such flux that I'll get to do this again in 18 months.
>
> Well, then I'm not sure why we have a PEP at all.
> If you don't want any significant changes and if you consider it to be *your*
> library, ipaddr can remain a third-party package that interested people can
> easily install (no pun ;-)) since AFAIK it's pure Python. It will also make
> maintenance easier for you, while freeing us (core developers) from having to
> bother about it in our daily development tasks.
>
> At least that's what I would advocate right now - not sure about what others
> think.

I think Peter is pretty frustrated by the many attacks on "his"
library. There are probably a number of different things going on
simultaneous: Peter has been driven into the defense by attacks both
reasonable and unreasonable, there have been misunderstandings all
around, teasing out use cases (by both parties) has been a problem.

Things might have gone differently if the PEP had started out with
multiple authors. Maybe it's not too late to add one or more other
interested parties to the PEP with the purpose of making the PEP more
clearly the result of a consensus-gathering process.  Any volunteers?

At the same time I don't think a complete reset of the proposed API is
necessary.  I am rather more thinking of judicious API tweaks in order
to cover some new use cases, without requiring a complete rewrite or
destroying the usability of the proposal for Peter's original use
cases. (In general I am pretty happy with the ipaddr code and API; it
looks like what I would have done, but then I am blissfully unaware of
some of the issues brought up in this thread.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From robert.kern at gmail.com  Mon Sep 28 18:39:43 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 28 Sep 2009 11:39:43 -0500
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909280837h633cc615geb09dde15b08697c@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<e36b84ee0909280823r7747ec4fy7a7def5a32591607@mail.gmail.com>	<4AC0D5BC.40108@voidspace.org.uk>
	<d11dcfba0909280837h633cc615geb09dde15b08697c@mail.gmail.com>
Message-ID: <h9qosf$ktf$1@ger.gmane.org>

On 2009-09-28 10:37 AM, Steven Bethard wrote:
> On Mon, Sep 28, 2009 at 8:26 AM, Michael Foord
> <fuzzyman at voidspace.org.uk>  wrote:
>> m h wrote:
>>>
>>> Perhaps this is OT, but since command line parsing is part of
>>> configuration, I figure I'd throw it out there.  My scripts often have
>>> configuration that the command line can override and I loosely follow
>>> the example hierarchy[0] listed in The Art of Unix Programming.
>>>
>>> Some configuration I want in a config file (but I want to override
>>> from the command line) and sometimes it's very nice to use environment
>>> variables for configuration.  So I do something like this:
>>
>> Integration with command line options is an occasionally-requested feature
>> for ConfigObj. I've always said I would be open to patches...
>>
>> In other words, yes I think there is demand for it. Whether it belongs
>> *immediately* in the standard library is another matter, but if you wrote a
>> layer that unified ConfigParser and argparse I think you will find that
>> people use it.
>>
>> It is well outside the realm of this PEP however.
>
> What Michael said. ;-) If you'd like to provide a patch for argparse
> that provides such functionality, I'd be happy to review it.

cfgparse appears to fill this role for optparse. It might be straightforward to 
port it to argparse.

http://cfgparse.sourceforge.net/

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From peter at hda3.com  Mon Sep 28 19:22:05 2009
From: peter at hda3.com (Peter Moody)
Date: Mon, 28 Sep 2009 10:22:05 -0700
Subject: [Python-Dev] PEP 3144 review, and the inclusion process
In-Reply-To: <ca471dc20909280939o75ed5ddbp91d5854175b35fd7@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
	<loom.20090928T000905-169@post.gmane.org>
	<8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com>
	<loom.20090928T022351-635@post.gmane.org>
	<ca471dc20909280939o75ed5ddbp91d5854175b35fd7@mail.gmail.com>
Message-ID: <8517e9350909281022v39aeb83clb352709822e0b26b@mail.gmail.com>

[cc += david moss]

On Mon, Sep 28, 2009 at 9:39 AM, Guido van Rossum <guido at python.org> wrote:
> On Sun, Sep 27, 2009 at 5:32 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Peter Moody <peter <at> hda3.com> writes:
>>>
>>> I've never said otherwise. In fact, from an email last night, "If what
>>> the community requires is the library you've described, then ipaddr is
>>> not that library." The changes *you* require make ipaddr significantly
>>> less useful to me. I'm not prepared to make those changes in an
>>> attempt seek acceptance to the stdlib, especially if the stdlib is in
>>> such flux that I'll get to do this again in 18 months.
>>
>> Well, then I'm not sure why we have a PEP at all.
>> If you don't want any significant changes and if you consider it to be *your*
>> library, ipaddr can remain a third-party package that interested people can
>> easily install (no pun ;-)) since AFAIK it's pure Python. It will also make
>> maintenance easier for you, while freeing us (core developers) from having to
>> bother about it in our daily development tasks.
>>
>> At least that's what I would advocate right now - not sure about what others
>> think.
>
> I think Peter is pretty frustrated by the many attacks on "his"
> library. There are probably a number of different things going on
> simultaneous: Peter has been driven into the defense by attacks both
> reasonable and unreasonable, there have been misunderstandings all
> around, teasing out use cases (by both parties) has been a problem.
>
> Things might have gone differently if the PEP had started out with
> multiple authors. Maybe it's not too late to add one or more other
> interested parties to the PEP with the purpose of making the PEP more
> clearly the result of a consensus-gathering process. ?Any volunteers?

David called me a little over a week ago and expressed an interest in
doing exactly this cross continent/ocean coordination has been a
little difficult thus far and I'm not certain what his feelings on
this are now.

> At the same time I don't think a complete reset of the proposed API is
> necessary. ?I am rather more thinking of judicious API tweaks in order
> to cover some new use cases, without requiring a complete rewrite or
> destroying the usability of the proposal for Peter's original use
> cases. (In general I am pretty happy with the ipaddr code and API; it
> looks like what I would have done, but then I am blissfully unaware of
> some of the issues brought up in this thread.)
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From guido at python.org  Mon Sep 28 19:33:21 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Sep 2009 10:33:21 -0700
Subject: [Python-Dev] Fuzziness in io module specs - PEP update
	proposition V2
In-Reply-To: <h9ops5$hkf$2@ger.gmane.org>
References: <4AB3DCC9.5050209@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> 
	<4AB5FA81.5060408@gmail.com>
	<eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com> 
	<4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com> 
	<4ABF1B3B.4080106@gmail.com> <h9nihf$9f4$1@ger.gmane.org> 
	<ca471dc20909271424l57315d4ft18d4bbe7da8bdd4@mail.gmail.com> 
	<h9ops5$hkf$2@ger.gmane.org>
Message-ID: <ca471dc20909281033w4b9c78d9mf56212c42253bfb2@mail.gmail.com>

On Sun, Sep 27, 2009 at 3:44 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Sun, 27 Sep 2009 14:24:52 -0700, Guido van Rossum a ?crit?:
> [truncate()]
>>
>> What compatibility, though?
>
> Compatibility accross the 3.x line.

Well, in this case, maybe compatibility with 2.x is more important --
this isn't something we can easily warn about in 2.6+. In addition
there's the POSIX rules.

>> f.truncate() behaves different in 2.x than
>> in 3.x, and in 2.x it seems to match the POSIX semantics (i.e. the seek
>> position is unchanged even though the file size is). Perhaps the changed
>> semantics was an oversight or a mistake?
>
> Perhaps it was, indeed. I don't know who made that decision in the first
> place.

It might well have been me (when implementing the earliest version of
io.py), and I might well have *though* that I implemented the same
rules as 2.x, and never bothered to check. :-(

All in all I think we should change this before it's too late; it will
affect a very small number of apps (perhaps none?), but I would rather
have the right semantics in the future. Also, it's trivial to write
code that doesn't care (in fact code running under 2.x and 3.x
probably will have to be written so that it doesn't care) so it's not
like changing this is going to make life harder for people wanting
multiple-version support.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From solipsis at pitrou.net  Mon Sep 28 20:17:01 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Sep 2009 18:17:01 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review, and the inclusion process
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
	<loom.20090928T000905-169@post.gmane.org>
	<8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com>
	<loom.20090928T022351-635@post.gmane.org>
	<ca471dc20909280939o75ed5ddbp91d5854175b35fd7@mail.gmail.com>
Message-ID: <loom.20090928T201525-766@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> At the same time I don't think a complete reset of the proposed API is
> necessary.  I am rather more thinking of judicious API tweaks in order
> to cover some new use cases, without requiring a complete rewrite or
> destroying the usability of the proposal for Peter's original use
> cases.

I think the current API is mostly fine as well, except for one roadblock for
which several people has raised concerns, and that Peter stated he doesn't want
to change.

Regards

Antoine.



From steve at pearwood.info  Mon Sep 28 20:21:20 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 29 Sep 2009 04:21:20 +1000
Subject: [Python-Dev] PEP 3144 review, and the inclusion process
In-Reply-To: <ca471dc20909280939o75ed5ddbp91d5854175b35fd7@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090928T022351-635@post.gmane.org>
	<ca471dc20909280939o75ed5ddbp91d5854175b35fd7@mail.gmail.com>
Message-ID: <200909290421.21622.steve@pearwood.info>

On Tue, 29 Sep 2009 02:39:06 am Guido van Rossum wrote:

> I think Peter is pretty frustrated by the many attacks on "his"
> library. There are probably a number of different things going on
> simultaneous: Peter has been driven into the defense by attacks both
> reasonable and unreasonable, there have been misunderstandings all
> around, teasing out use cases (by both parties) has been a problem.

As one of those people who has been critical of parts of Peter's module, 
I'd like to publicly say that I *mostly* like it. Sometimes we focus 
too much on the negatives and don't remember to give credit where 
credit is due.

Hopefully we can reach consensus, or at least understanding, on the 
remaining issue(s).



-- 
Steven D'Aprano

From sesquile at gmail.com  Mon Sep 28 20:26:46 2009
From: sesquile at gmail.com (m h)
Date: Mon, 28 Sep 2009 12:26:46 -0600
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909280837h633cc615geb09dde15b08697c@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<e36b84ee0909280823r7747ec4fy7a7def5a32591607@mail.gmail.com>
	<4AC0D5BC.40108@voidspace.org.uk>
	<d11dcfba0909280837h633cc615geb09dde15b08697c@mail.gmail.com>
Message-ID: <e36b84ee0909281126l68bd15e8t20b5feab82a2a6b3@mail.gmail.com>

On Mon, Sep 28, 2009 at 9:37 AM, Steven Bethard
<steven.bethard at gmail.com> wrote:
> On Mon, Sep 28, 2009 at 8:26 AM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>> m h wrote:
>>>
>>> Perhaps this is OT, but since command line parsing is part of
>>> configuration, I figure I'd throw it out there. ?My scripts often have
>>> configuration that the command line can override and I loosely follow
>>> the example hierarchy[0] listed in The Art of Unix Programming.
>>>
>>> Some configuration I want in a config file (but I want to override
>>> from the command line) and sometimes it's very nice to use environment
>>> variables for configuration. ?So I do something like this:
>>
>> Integration with command line options is an occasionally-requested feature
>> for ConfigObj. I've always said I would be open to patches...
>>
>> In other words, yes I think there is demand for it. Whether it belongs
>> *immediately* in the standard library is another matter, but if you wrote a
>> layer that unified ConfigParser and argparse I think you will find that
>> people use it.

Having it integrated with the option parser is ideal.  More people
will use it if it is in the stdlib.  Just like more people will use
argparse.

>>
>> It is well outside the realm of this PEP however.
>
> What Michael said. ;-) If you'd like to provide a patch for argparse
> that provides such functionality, I'd be happy to review it.
>

Patch submitted [0].  Accidentally marked as "Defect" instead of enhancement.
Comments/critiques welcome.

0 - http://code.google.com/p/argparse/issues/detail?id=35&q=label%3AType-Defect

Cheers,

-matt

> Steve
> --
> Where did you get that preposterous hypothesis?
> Did Steve tell you that?
> ? ? ? ?--- The Hiphopopotamus
>

From drkjam at gmail.com  Mon Sep 28 20:24:37 2009
From: drkjam at gmail.com (DrKJam)
Date: Mon, 28 Sep 2009 19:24:37 +0100
Subject: [Python-Dev] PEP 3144 review, and the inclusion process
In-Reply-To: <8517e9350909281022v39aeb83clb352709822e0b26b@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
	<loom.20090928T000905-169@post.gmane.org>
	<8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com>
	<loom.20090928T022351-635@post.gmane.org>
	<ca471dc20909280939o75ed5ddbp91d5854175b35fd7@mail.gmail.com>
	<8517e9350909281022v39aeb83clb352709822e0b26b@mail.gmail.com>
Message-ID: <538a660a0909281124i63358f8asd7e3c4d8245d9ebe@mail.gmail.com>

2009/9/28 Peter Moody <peter at hda3.com>

> [cc += david moss]
>
> On Mon, Sep 28, 2009 at 9:39 AM, Guido van Rossum <guido at python.org>
> wrote:
> > On Sun, Sep 27, 2009 at 5:32 PM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> >> Peter Moody <peter <at> hda3.com> writes:
> >>>
> >>> I've never said otherwise. In fact, from an email last night, "If what
> >>> the community requires is the library you've described, then ipaddr is
> >>> not that library." The changes *you* require make ipaddr significantly
> >>> less useful to me. I'm not prepared to make those changes in an
> >>> attempt seek acceptance to the stdlib, especially if the stdlib is in
> >>> such flux that I'll get to do this again in 18 months.
> >>
> >> Well, then I'm not sure why we have a PEP at all.
> >> If you don't want any significant changes and if you consider it to be
> *your*
> >> library, ipaddr can remain a third-party package that interested people
> can
> >> easily install (no pun ;-)) since AFAIK it's pure Python. It will also
> make
> >> maintenance easier for you, while freeing us (core developers) from
> having to
> >> bother about it in our daily development tasks.
> >>
> >> At least that's what I would advocate right now - not sure about what
> others
> >> think.
> >
> > I think Peter is pretty frustrated by the many attacks on "his"
> > library. There are probably a number of different things going on
> > simultaneous: Peter has been driven into the defense by attacks both
> > reasonable and unreasonable, there have been misunderstandings all
> > around, teasing out use cases (by both parties) has been a problem.
> >
> > Things might have gone differently if the PEP had started out with
> > multiple authors. Maybe it's not too late to add one or more other
> > interested parties to the PEP with the purpose of making the PEP more
> > clearly the result of a consensus-gathering process.  Any volunteers?
>
> David called me a little over a week ago and expressed an interest in
> doing exactly this cross continent/ocean coordination has been a
> little difficult thus far and I'm not certain what his feelings on
> this are now.
>
>
Sure, I'm happy to volunteer and help out. Let's have a good hard look at
all this and see what we can come up with.


> > At the same time I don't think a complete reset of the proposed API is
> > necessary.  I am rather more thinking of judicious API tweaks in order
> > to cover some new use cases, without requiring a complete rewrite or
> > destroying the usability of the proposal for Peter's original use
> > cases. (In general I am pretty happy with the ipaddr code and API; it
> > looks like what I would have done, but then I am blissfully unaware of
> > some of the issues brought up in this thread.)
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/>
> )
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090928/bc7e6ef2/attachment.htm>

From guido at python.org  Mon Sep 28 20:43:06 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Sep 2009 11:43:06 -0700
Subject: [Python-Dev] PEP 3144 review, and the inclusion process
In-Reply-To: <loom.20090928T201525-766@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> 
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> 
	<loom.20090928T000905-169@post.gmane.org>
	<8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> 
	<loom.20090928T022351-635@post.gmane.org>
	<ca471dc20909280939o75ed5ddbp91d5854175b35fd7@mail.gmail.com> 
	<loom.20090928T201525-766@post.gmane.org>
Message-ID: <ca471dc20909281143p74914e7bp9fe759a3150f0fc8@mail.gmail.com>

On Mon, Sep 28, 2009 at 11:17 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Guido van Rossum <guido <at> python.org> writes:
>>
>> At the same time I don't think a complete reset of the proposed API is
>> necessary. ?I am rather more thinking of judicious API tweaks in order
>> to cover some new use cases, without requiring a complete rewrite or
>> destroying the usability of the proposal for Peter's original use
>> cases.
>
> I think the current API is mostly fine as well, except for one roadblock for
> which several people has raised concerns, and that Peter stated he doesn't want
> to change.

Well, I'm sure somebody can come up with a compromise that works for everyone.

Assuming this is the issue about e.g. IPNetwork('7.7.7.7/30') !=
IPNetwork('7.7.7.4/30') even though these describe the same network
(and in fact their .network and .broadcast attributes have the same
values), there are several compromises; I don't know the use cases
well enough to pick one:

- There could be an optional keyword parameter that decides whether
the last bits are cleared or not (one would have to argue which
behavior to use by default).

- The __eq__ (and hence __hash__) could be made to return equality
even though the .ip attribute returns a different value and the repr()
is different. This is not unheard-of; Python's __eq__ is best thought
of as some sort of equivalence. For example two dicts that only differ
in the order of the items still compare equal (even though .keys() and
repr() return different values).

I note that it's easy to convert the unnormalized form to the normalized form:

a = IPNetwork('7.7.7.7/30')
b = IPNetwork(a.network, a.prefixlen)

It would also be easy to have a property or method that returns the
normalized version; as an optimization it could return self if self is
already normalized. And there could be an explicit API to test whether
a network is normalized (something more explicit than a.network ==
a.ip).

I'll leave it to others to discuss the pros and cons of each
alternative. I don't know why Peter wants to preserve the unnormalized
data but I'm sure he has a good use case, otherwise he wouldn't be so
adamant to keep it. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Mon Sep 28 21:52:53 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 28 Sep 2009 21:52:53 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
Message-ID: <4AC11415.9060305@v.loewis.de>

> The fundamental divide here is between two behaviors.
> 
> ipaddr:
> 
>     >>> x = IPv4Network('192.168.1.1/24')
>     >>> y = IPv4Network('192.168.1.0/24')
>     >>> x == y
>     False
>     >>> x.ip
>     IPv4Address('192.168.1.1')
> 
> desired:
> 
>     >>> x = IPv4Network('192.168.1.1/24')
>     >>> y = IPv4Network('192.168.1.0/24')
>     >>> x == y
>     True
>     >>> x.ip
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     AttributeError: 'IPv4Network' object has no attribute 'ip'

I can understand the debate around the comparison.

I cannot understand the complaint that networks where the base
address is not null-terminated should be banned, or useless
(which was the original concern that I responded to).

> Everything else is pretty much bikeshedding and can be dealt with.  This
> is fundamental and Peter has indicated he will not change it.

I can see why either behavior is desirable (i.e. comparison should
either use the specified address, or the network address); following
the Python Zen, we should refuse the temptation to guess.

Unfortunately, that would mean that we should provide two comparison
operations, and have neither available as == (unless one is more clearly
right than the other, which I'm skeptical about).

Regards,
Martin

From martin at v.loewis.de  Mon Sep 28 21:55:03 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 28 Sep 2009 21:55:03 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <20090928125818.1533.2070636475.divmod.xquotient.382@localhost.localdomain>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>	<4AC03410.4010104@v.loewis.de>
	<20090928125818.1533.2070636475.divmod.xquotient.382@localhost.localdomain>
Message-ID: <4AC11497.4060905@v.loewis.de>

>> It's fairly obvious to me why the library should support 192.168.1.1/24
>> as an input, and return a network.
>>
>> End-users are likely going to enter such things (e.g. 82.94.164.162/29),
>> as they will know one IP address in the network (namely, of one machine
>> that they are looking at), and they will know the prefix length (more
>> often, how large the network is - 8 or 16 or 32). So very clearly,
>> end users should not be required to make the computation in their heads.
>>
>> So Python code has to make the computation, and it seems most natural
>> that the IP library is the piece of code that is able to compute a
>> network out of that input.
> 
> And this is a rather classic example of a misfeature.  "(Non-developer)
> End users will type it in" is not argument for supporting a particular
> string format as the primary constructor for an object.  Constructors
> are for *developers*.  They should be made most useful for *developers*.
> The issue of dealing with user input (which may have many other quirks)
> is separate and should be dealt with separately.

Still, people proposing that 192.168.1.1/24 should be rejected (I still
don't know whether anybody is actually proposing that) *have* to provide
a proposal how this should be input instead. Otherwise, such proposals
must be rejected, since there is a clear and evident need to support
such input somehow.

Regards,
Martin

From brett at python.org  Mon Sep 28 21:22:55 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 28 Sep 2009 12:22:55 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com> 
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com> 
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
Message-ID: <bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>

On Mon, Sep 28, 2009 at 08:49, Steven Bethard <steven.bethard at gmail.com> wrote:
> On Mon, Sep 28, 2009 at 8:27 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> On Tue, 29 Sep 2009 12:28:39 am Steven Bethard wrote:
>>> * Would you like argparse to grow an add_getopt_arguments method (as
>>> in my other post)?
>>
>> 0
>>
>>> * If argparse grew an add_getopt_arguments, would you still want to
>>> keep getopt around? And if so, why?
>>
>> Simplicity of the learning curve. Using it is as simple as:
>>
>> getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])
>
> You forgot the for-loop, nested if/else statements and type conversions. ;-)
>

=) I do wonder if people who are advocating for getopt sticking around
realize how much extra code must be written to make sure what it gives
back to you is in some sane manner.

Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])``
as an example and simply assume that 'alpha' takes a string as an
argument and that it's required and that 'beta' is a boolean flag. To
pull everything out you could do::

  options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])
  options_dict = dict(options)
  alpha = options_dict.get('-a', options_dict.get('--alpha', ''))
  beta = '-b' in options_dict or '--beta' in options_dict

  main(alpha, beta, args)

Obviously if one of the getopt supporters has a better way of doing
this then please speak up.

Now, Steven, can you show how best to do this in argparse? I am
willing to bet that the total number of lines to do this is not that
much more and does not require you to know to use 'or' or the dict
constructor along with dict.get() in order to keep it compact. I can
only imagine what some newbie might try to do in order to be correct
(if they even try).

-Brett

From steven.bethard at gmail.com  Mon Sep 28 21:57:08 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 28 Sep 2009 12:57:08 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
Message-ID: <d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>

On Mon, Sep 28, 2009 at 12:22 PM, Brett Cannon <brett at python.org> wrote:
> On Mon, Sep 28, 2009 at 08:49, Steven Bethard <steven.bethard at gmail.com> wrote:
>> On Mon, Sep 28, 2009 at 8:27 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>>> On Tue, 29 Sep 2009 12:28:39 am Steven Bethard wrote:
>>>> * Would you like argparse to grow an add_getopt_arguments method (as
>>>> in my other post)?
>>>
>>> 0
>>>
>>>> * If argparse grew an add_getopt_arguments, would you still want to
>>>> keep getopt around? And if so, why?
>>>
>>> Simplicity of the learning curve. Using it is as simple as:
>>>
>>> getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])
>>
>> You forgot the for-loop, nested if/else statements and type conversions. ;-)
>>
>
> =) I do wonder if people who are advocating for getopt sticking around
> realize how much extra code must be written to make sure what it gives
> back to you is in some sane manner.
>
> Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])``
> as an example and simply assume that 'alpha' takes a string as an
> argument and that it's required and that 'beta' is a boolean flag. To
> pull everything out you could do::
>
> ?options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])
> ?options_dict = dict(options)
> ?alpha = options_dict.get('-a', options_dict.get('--alpha', ''))
> ?beta = '-b' in options_dict or '--beta' in options_dict
>
> ?main(alpha, beta, args)
>
> Obviously if one of the getopt supporters has a better way of doing
> this then please speak up.
>
> Now, Steven, can you show how best to do this in argparse?

Here's the same option parsing in argparse:

    parser = argparse.ArgumentParser()
    parser.add_argument('-a', '--alpha')
    parser.add_argument('-b', '--beta', action='store_true')
    args = parser.parse_args()

    main(args.alpha, args.beta)

Or if those final positional arguments were actually meaningful, then
you would add one more argument like this::

    parser = argparse.ArgumentParser()
    parser.add_argument('-a', '--alpha')
    parser.add_argument('-b', '--beta', action='store_true')
    parser.add_argument('gammas', nargs='*')
    args = parser.parse_args()

    main(args.alpha, args.beta, args.gammas)

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From martin at v.loewis.de  Mon Sep 28 22:11:01 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 28 Sep 2009 22:11:01 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <loom.20090928T084811-560@post.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
Message-ID: <4AC11855.7080406@v.loewis.de>

> Martin v. L?wis <martin <at> v.loewis.de> writes:
>>> Could you explain what benefit there is for allowing the user to create 
>>> network objects that don't represent networks? Is there a use-case 
>>> where these networks-that-aren't-networks are something other than a 
>>> typo? Under what circumstances would I want to specify a network as 
>>> 192.168.1.1/24 instead of 192.168.1.0/24?
>>>
> [...]
>> So Python code has to make the computation, and it seems most natural
>> that the IP library is the piece of code that is able to compute a
>> network out of that input.
> 
> The thing is, it doesn't create a network, it creates a hybrid "network plus
> host" which retains knowledge about the original host (that is, 192.168.1.1
> rather than simply 192.168.1.0, if you enter "192.168.1.1/24").
> 
> That's what the OP meant with "networks-that-aren't-networks", and that's what
> people are objecting to.

That's not the question that was asked, though - the question asked
was "Under what circumstances would I want to specify...". I hope
most people agree that it is desirable to be able to specify a network
not just by its network address.

Regards,
Martin

From solipsis at pitrou.net  Mon Sep 28 22:11:56 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Sep 2009 20:11:56 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<Pine.LNX.4.64.0909280725360.18193@kimball.webabinitio.net>
	<4AC11415.9060305@v.loewis.de>
Message-ID: <loom.20090928T215826-406@post.gmane.org>


> Unfortunately, that would mean that we should provide two comparison
> operations, and have neither available as == (unless one is more clearly
> right than the other, which I'm skeptical about).

That's probably why some people have argued to have two separate classes for the
case of "networks" with significant lower bits.

(by "lower bits" I mean bits which would be zeroed when ANDing with the netmask)



From solipsis at pitrou.net  Mon Sep 28 22:14:32 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Sep 2009 20:14:32 +0000 (UTC)
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>	<4AC03410.4010104@v.loewis.de>
	<20090928125818.1533.2070636475.divmod.xquotient.382@localhost.localdomain>
	<4AC11497.4060905@v.loewis.de>
Message-ID: <loom.20090928T221238-820@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> Still, people proposing that 192.168.1.1/24 should be rejected (I still
> don't know whether anybody is actually proposing that) *have* to provide
> a proposal how this should be input instead.

It was proposed to have a dedicated function or class method which would parse
such strings and return a (Network, Address) tuple corresponding to the two
objects that are described by that single notation.

  >>> parse_network_and_host('192.168.1.1/24')
  (Network('192.168.1.0/24'), Address('192.168.1.1'))


Regards

Antoine.



From scott+python-dev at scottdial.com  Mon Sep 28 22:19:15 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Mon, 28 Sep 2009 16:19:15 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC11855.7080406@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>	<4AC03410.4010104@v.loewis.de>	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
Message-ID: <4AC11A43.2010401@scottdial.com>

Martin v. L?wis wrote:
>> Martin v. L?wis <martin <at> v.loewis.de> writes:
>>>> Could you explain what benefit there is for allowing the user to create 
>>>> network objects that don't represent networks? Is there a use-case 
>>>> where these networks-that-aren't-networks are something other than a 
>>>> typo? Under what circumstances would I want to specify a network as 
>>>> 192.168.1.1/24 instead of 192.168.1.0/24?
>>>>
>> [...]
>>> So Python code has to make the computation, and it seems most natural
>>> that the IP library is the piece of code that is able to compute a
>>> network out of that input.
>> The thing is, it doesn't create a network, it creates a hybrid "network plus
>> host" which retains knowledge about the original host (that is, 192.168.1.1
>> rather than simply 192.168.1.0, if you enter "192.168.1.1/24").
>>
>> That's what the OP meant with "networks-that-aren't-networks", and that's what
>> people are objecting to.
> 
> That's not the question that was asked, though - the question asked
> was "Under what circumstances would I want to specify...". I hope
> most people agree that it is desirable to be able to specify a network
> not just by its network address.

I can't help but feel that you and many others jumping on Antoine et al.
have skipped reading or only skimmed this topic of discussion.

Antoine Pitrou wrote:
> We just need a factory function which returns a tuple after parsing:
>
>     >>> addr, net = parse_address_with_mask('192.168.1.1/24')
>     >>> addr == IPv4Address('192.168.1.1')
>     True
>     >>> net == IPv4Network('192.168.1.0/24')
>     True

I still back this suggestion. I would also support a "loose=True" flag
for the IPNetwork constructor, which would allow host bits to be non-zero.

However, I will *never* support a proposal that includes retaining the
host address. I believe I am not alone in that. However, Peter has made
it quite clear that he will *never* change that about the IPNetwork classes.

I would be quite glad to not have ipaddr included in the stdlib. As I
can only imagine code that uses it being quite confusing to maintain
(e.g., "wait is that actually a network or is that someone using
IPNetwork to be an IPAddressWithMask?").

-1.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From solipsis at pitrou.net  Mon Sep 28 22:32:45 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Sep 2009 22:32:45 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC11855.7080406@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
Message-ID: <1254169965.4663.25.camel@localhost>

Le lundi 28 septembre 2009 ? 22:11 +0200, "Martin v. L?wis" a ?crit :
> 
> That's not the question that was asked, though - the question asked
> was "Under what circumstances would I want to specify...". I hope
> most people agree that it is desirable to be able to specify a network
> not just by its network address.

To me it makes no sense to "specify a network not just by its network
address", because by definition a prefix + mask is all that
characterizes a network: that's all you need to specify it entirely. 
Adding a "base address" would be like adding a "port number" attribute
to the Address class: it's abusing an existing class by adding
irrelevant information.

I may be mistaken, but it seems to me that, in the IP world, nowhere
does such a "network + host" hybrid appear. You find networks (prefix +
mask) in routing protocols, and host addresses in applicative protocols.



From rdmurray at bitdance.com  Mon Sep 28 22:36:10 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 28 Sep 2009 16:36:10 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC11855.7080406@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>

On Mon, 28 Sep 2009 at 22:11, "Martin v. L??wis" wrote:
>> Martin v. L??wis <martin <at> v.loewis.de> writes:
>>>> Could you explain what benefit there is for allowing the user to create
>>>> network objects that don't represent networks? Is there a use-case
>>>> where these networks-that-aren't-networks are something other than a
>>>> typo? Under what circumstances would I want to specify a network as
>>>> 192.168.1.1/24 instead of 192.168.1.0/24?
>>>>
>> [...]
>>> So Python code has to make the computation, and it seems most natural
>>> that the IP library is the piece of code that is able to compute a
>>> network out of that input.
>>
>> The thing is, it doesn't create a network, it creates a hybrid "network plus
>> host" which retains knowledge about the original host (that is, 192.168.1.1
>> rather than simply 192.168.1.0, if you enter "192.168.1.1/24").
>>
>> That's what the OP meant with "networks-that-aren't-networks", and that's what
>> people are objecting to.
>
> That's not the question that was asked, though - the question asked
> was "Under what circumstances would I want to specify...". I hope
> most people agree that it is desirable to be able to specify a network
> not just by its network address.

But then it's not a network, it is an ipaddress-plus-mask.  It is exactly
that conflation that we are objecting to.  There is no question about
the use case of _specifying_ a network ip plus a netmask and deriving
a network object from that.  That is unquestionably required by any
useful API.  The argument is about whether the object returned is a
Network object, or a hybrid object representing _both_ an IP address
and a network.  It is the latter, which ipaddr does, which many of us
find problematic and likely to lead to hard to read programs that will
probably produce maintenance errors.

I observe that this line in the current PEP rationale:

     IP addresses and IP networks are distinct.

is not in fact achieved by the reference implementation.  Peter, however,
clearly thinks it is, since it is listed as a design goal of ipaddr.
This is a language disconnect I don't understand, which I think has
been the source of a lot of the difficulties in this thread.

--David

From solipsis at pitrou.net  Mon Sep 28 22:39:43 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 28 Sep 2009 20:39:43 +0000 (UTC)
Subject: [Python-Dev] PEP 389: argparse - optparse compatibility
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<h9q5i7$hv2$1@ger.gmane.org>
	<d11dcfba0909280743n7ce99d03o64396b33bcddc739@mail.gmail.com>
Message-ID: <loom.20090928T223542-163@post.gmane.org>

Steven Bethard <steven.bethard <at> gmail.com> writes:
> 
> I could conceivably add an OptionParser class which::
> [...]

Actually, I think that would be a waste of time. It would be less time-consuming
to keep optparse.py than write a whole new compatibility bridge that you'll have
to debug and maintain anyway, with the added problem of making people annoyed by
whatever difference in behaviour happens to bite them. (*)

So I think the two only reasonable options are:

1. either keep optparse durably
2. or deprecate it

Given your precision about the proposed deprecation period (with the module
vanishing in 3.5), I guess I'm only -0.5 on it, rather than -1.

(*) (but of course I'm not against anyone wasting their time if they want to :-))

Regards

Antoine.



From guido at python.org  Mon Sep 28 22:43:44 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Sep 2009 13:43:44 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> 
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de> 
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
Message-ID: <ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>

On Mon, Sep 28, 2009 at 1:36 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Mon, 28 Sep 2009 at 22:11, "Martin v. L?wis" wrote:
>>>
>>> Martin v. L?wis <martin <at> v.loewis.de> writes:
>>>>>
>>>>> Could you explain what benefit there is for allowing the user to create
>>>>> network objects that don't represent networks? Is there a use-case
>>>>> where these networks-that-aren't-networks are something other than a
>>>>> typo? Under what circumstances would I want to specify a network as
>>>>> 192.168.1.1/24 instead of 192.168.1.0/24?
>>>>>
>>> [...]
>>>>
>>>> So Python code has to make the computation, and it seems most natural
>>>> that the IP library is the piece of code that is able to compute a
>>>> network out of that input.
>>>
>>> The thing is, it doesn't create a network, it creates a hybrid "network
>>> plus
>>> host" which retains knowledge about the original host (that is,
>>> 192.168.1.1
>>> rather than simply 192.168.1.0, if you enter "192.168.1.1/24").
>>>
>>> That's what the OP meant with "networks-that-aren't-networks", and that's
>>> what
>>> people are objecting to.
>>
>> That's not the question that was asked, though - the question asked
>> was "Under what circumstances would I want to specify...". I hope
>> most people agree that it is desirable to be able to specify a network
>> not just by its network address.
>
> But then it's not a network, it is an ipaddress-plus-mask. ?It is exactly
> that conflation that we are objecting to. ?There is no question about
> the use case of _specifying_ a network ip plus a netmask and deriving
> a network object from that. ?That is unquestionably required by any
> useful API. ?The argument is about whether the object returned is a
> Network object, or a hybrid object representing _both_ an IP address
> and a network. ?It is the latter, which ipaddr does, which many of us
> find problematic and likely to lead to hard to read programs that will
> probably produce maintenance errors.
>
> I observe that this line in the current PEP rationale:
>
> ? ?IP addresses and IP networks are distinct.
>
> is not in fact achieved by the reference implementation. ?Peter, however,
> clearly thinks it is, since it is listed as a design goal of ipaddr.
> This is a language disconnect I don't understand, which I think has
> been the source of a lot of the difficulties in this thread.

I say the case has been argued extensively, let's wait for Peter to respond.

I would say that there certainly are precedents in other areas for
keeping the information about the input form around. For example,
occasionally it would be handy if parsing a hex integer returned an
object that was compatible with other integers but somehow kept a hint
that would cause printing it to use hex by default.

I see keeping around the IP address used to create a network object as
a similar edge case. I would probably define the __eq__ method to
implement equivalency (ignoring the form of the input), just as I
would in the case of the (hypothetical) hex integers. If you wanted to
do a comparison that includes the input IP address, you could use (a,
a.ip) == (b, b.ip).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rdmurray at bitdance.com  Mon Sep 28 22:47:26 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 28 Sep 2009 16:47:26 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <1254169965.4663.25.camel@localhost>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de> <1254169965.4663.25.camel@localhost>
Message-ID: <Pine.LNX.4.64.0909281638460.18193@kimball.webabinitio.net>

On Mon, 28 Sep 2009 at 22:32, Antoine Pitrou wrote:
> Le lundi 28 septembre 2009 ?? 22:11 +0200, "Martin v. L??wis" a ??crit :
>>
>> That's not the question that was asked, though - the question asked
>> was "Under what circumstances would I want to specify...". I hope
>> most people agree that it is desirable to be able to specify a network
>> not just by its network address.
>
> To me it makes no sense to "specify a network not just by its network
> address", because by definition a prefix + mask is all that
> characterizes a network: that's all you need to specify it entirely.
> Adding a "base address" would be like adding a "port number" attribute
> to the Address class: it's abusing an existing class by adding
> irrelevant information.
>
> I may be mistaken, but it seems to me that, in the IP world, nowhere
> does such a "network + host" hybrid appear. You find networks (prefix +
> mask) in routing protocols, and host addresses in applicative protocols.

You do find "address plus mask" when specifying an IP address for an
interface.  This is a shorthand for specifying the IP address plus the
network, since the network can be derived from the IP plus the mask.
But it is a _shorthand_, it isn't an entity in its own right.  There is
no such thing as "a network that has an IP", there is only an
_interface configuration_ that has an IP _and_ an associated network.

--David

From ubershmekel at gmail.com  Mon Sep 28 23:14:59 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Tue, 29 Sep 2009 00:14:59 +0300
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
Message-ID: <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>

for a live demo of how getopt is useful and flexible, I like how Audacity
uses it:
http://www.google.com/codesearch/p?hl=en&sa=N&cd=6&ct=rc#_hWFOhGz9lE/mezzo/scons/sconsign.py&q=getopt%20%22import%20getopt%22%20file:%5C.py$&l=264

To answer your question, it goes like this:
    options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])
    for o, a in options:
        if o in ('-a', '--alpha'):
            alpha = a
        elif o in ('-b', '--beta'):
            beta = a
    main(alpha, beta, args)

Notice a few things:
1. There is no chance of the script killing itself. In argparse and optparse
exit() is called on every parsing error (btw because of this it sucks to
debug parse_args in an interpreter).
2. There is no chance the parser will print things I don't want it to print.
3. You can do whatever you want using this flow - call a function, a binary
operator (ie | is used in audacity scons example above)
4. in argparse this flow can be emulated and would be nicer, in ('-a',
'--alpha') becomes == 'alpha'

In a perfect world, getopt would be the low level parser that argparse and
optparse rely on. This is not too far fetched btw, all that needs to be done
is add another optional argument to getopt that would allow '-' and '--' to
be replaced with arbitrary signs, OR moving the parsing code from argparse
into getopt.

On Mon, Sep 28, 2009 at 10:57 PM, Steven Bethard
<steven.bethard at gmail.com>wrote:

> On Mon, Sep 28, 2009 at 12:22 PM, Brett Cannon <brett at python.org> wrote:
> > On Mon, Sep 28, 2009 at 08:49, Steven Bethard <steven.bethard at gmail.com>
> wrote:
> >> On Mon, Sep 28, 2009 at 8:27 AM, Steven D'Aprano <steve at pearwood.info>
> wrote:
> >>> On Tue, 29 Sep 2009 12:28:39 am Steven Bethard wrote:
> >>>> * Would you like argparse to grow an add_getopt_arguments method (as
> >>>> in my other post)?
> >>>
> >>> 0
> >>>
> >>>> * If argparse grew an add_getopt_arguments, would you still want to
> >>>> keep getopt around? And if so, why?
> >>>
> >>> Simplicity of the learning curve. Using it is as simple as:
> >>>
> >>> getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])
> >>
> >> You forgot the for-loop, nested if/else statements and type conversions.
> ;-)
> >>
> >
> > =) I do wonder if people who are advocating for getopt sticking around
> > realize how much extra code must be written to make sure what it gives
> > back to you is in some sane manner.
> >
> > Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])``
> > as an example and simply assume that 'alpha' takes a string as an
> > argument and that it's required and that 'beta' is a boolean flag. To
> > pull everything out you could do::
> >
> >  options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])
> >  options_dict = dict(options)
> >  alpha = options_dict.get('-a', options_dict.get('--alpha', ''))
> >  beta = '-b' in options_dict or '--beta' in options_dict
> >
> >  main(alpha, beta, args)
> >
> > Obviously if one of the getopt supporters has a better way of doing
> > this then please speak up.
> >
> > Now, Steven, can you show how best to do this in argparse?
>
> Here's the same option parsing in argparse:
>
>    parser = argparse.ArgumentParser()
>    parser.add_argument('-a', '--alpha')
>    parser.add_argument('-b', '--beta', action='store_true')
>    args = parser.parse_args()
>
>    main(args.alpha, args.beta)
>
> Or if those final positional arguments were actually meaningful, then
> you would add one more argument like this::
>
>    parser = argparse.ArgumentParser()
>    parser.add_argument('-a', '--alpha')
>    parser.add_argument('-b', '--beta', action='store_true')
>    parser.add_argument('gammas', nargs='*')
>    args = parser.parse_args()
>
>    main(args.alpha, args.beta, args.gammas)
>
> Steve
> --
> Where did you get that preposterous hypothesis?
> Did Steve tell you that?
>        --- The Hiphopopotamus
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/0a77206a/attachment.htm>

From martin at v.loewis.de  Mon Sep 28 23:16:57 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 28 Sep 2009 23:16:57 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909281638460.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>	<loom.20090928T084811-560@post.gmane.org>	<4AC11855.7080406@v.loewis.de>
	<1254169965.4663.25.camel@localhost>
	<Pine.LNX.4.64.0909281638460.18193@kimball.webabinitio.net>
Message-ID: <4AC127C9.5080504@v.loewis.de>

> You do find "address plus mask" when specifying an IP address for an
> interface.  This is a shorthand for specifying the IP address plus the
> network, since the network can be derived from the IP plus the mask.
> But it is a _shorthand_

Just as he says.

Martin

From martin at v.loewis.de  Mon Sep 28 23:29:42 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 28 Sep 2009 23:29:42 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>	<loom.20090928T084811-560@post.gmane.org>	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
Message-ID: <4AC12AC6.80304@v.loewis.de>

> I would say that there certainly are precedents in other areas for
> keeping the information about the input form around. For example,
> occasionally it would be handy if parsing a hex integer returned an
> object that was compatible with other integers but somehow kept a hint
> that would cause printing it to use hex by default.

At the risk of bringing in false analogies: it seems that Python
typically represents values of some type in their canonical form,
rather than remembering the form in which they arrived in the program:
- integer values "forget" how many preceding zeroes they have
- string literals forget which of the characters had been escaped, and
  whether the string was single- or double-quoted
- floating point values forget a lot more about their literal
  representation (including even the literal decimal value)

I guess a close case would be rational numbers: clearly, 3?2 == 6?4;
would a Python library still remember (and repr) the original numerator
and denominator?

Regards,
Martin

From daniel at stutzbachenterprises.com  Mon Sep 28 23:37:38 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 28 Sep 2009 16:37:38 -0500
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC12AC6.80304@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de>
Message-ID: <eae285400909281437w2b9bfe3dq4168c62b4b5d0629@mail.gmail.com>

On Mon, Sep 28, 2009 at 4:29 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> I guess a close case would be rational numbers: clearly, 3?2 == 6?4;
> would a Python library still remember (and repr) the original numerator
> and denominator?
>

No need for a hypothetical, rational numbers were added in Python 2.6:

Python 2.6.2 (r262:71600, Apr 15 2009, 07:20:39)
[GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import fractions
>>> fractions.Fraction(6,4)
Fraction(3, 2)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090928/0c9f5679/attachment.htm>

From cs at zip.com.au  Mon Sep 28 23:44:50 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Tue, 29 Sep 2009 07:44:50 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909272124x27c7e232n6186908e523c318e@mail.gmail.com>
References: <d11dcfba0909272124x27c7e232n6186908e523c318e@mail.gmail.com>
Message-ID: <20090928214450.GA6996@cskk.homeip.net>

On 27Sep2009 21:24, Steven Bethard <steven.bethard at gmail.com> wrote:
| On Sun, Sep 27, 2009 at 9:09 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
| >> If you think getopt and optparse should stick around in 3.X, why is
| >> that? If you think there are things that getopt and optparse do better
| >> than argparse, could you please give some examples?
| >
| > I personally consider getopt superior to both optparse and argparse.
| > Those are terribly verbose in specifying arguments, whereas getopt's
| > sequence-of-letters is really nice and compact.
| 
| Thanks for the concrete example. Although I'm unconvinced that the
| characters you save in the sequence of letters in the getopt.getopt
| call aren't afterwards wasted on type conversions, if/else statements
| and variable assignments in the subsequent loop, it would be pretty
| simple to add to argparse something like::
| 
|     ArgumentParser.add_getopt_arguments(options[, long_options])

Yes please!

I'm also very fond of the succinct getopt sequence-of-letters style;
it works really well for the simple cases.

Disclaimer: I've not used the optparse et al modules because getopt has
covered my needs and my C background made getopt the natural module to
start with. I have written simple getopt-equivalent option parsers a
number of times though. (Of course, that choice also drives me to adapt
my option wishes to stuff that getopt can do:-)

Clarification: this isn't a vote for preferring getopt, it's a vote for
availability of the sequence-of-letters style.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

The most annoying thing about being without my files after our disc crash was
discovering once again how widespread BLINK was on the web.

From phd at phd.pp.ru  Mon Sep 28 23:51:07 2009
From: phd at phd.pp.ru (Oleg Broytman)
Date: Tue, 29 Sep 2009 01:51:07 +0400
Subject: [Python-Dev] PEP 389: argparse - new command line parsing	module
In-Reply-To: <20090928214450.GA6996@cskk.homeip.net>
References: <d11dcfba0909272124x27c7e232n6186908e523c318e@mail.gmail.com>
	<20090928214450.GA6996@cskk.homeip.net>
Message-ID: <20090928215107.GA15173@phd.pp.ru>

On Tue, Sep 29, 2009 at 07:44:50AM +1000, Cameron Simpson wrote:
> |     ArgumentParser.add_getopt_arguments(options[, long_options])
> 
> Yes please!
> 
> I'm also very fond of the succinct getopt sequence-of-letters style;
> it works really well for the simple cases.

   The syntax "a:b" is also widely known; people who are often write small
shell scripts are certainly accustomed to it (I am).

> Clarification: this isn't a vote for preferring getopt, it's a vote for
> availability of the sequence-of-letters style.

   +1

Oleg.
-- 
     Oleg Broytman            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From greg.ewing at canterbury.ac.nz  Tue Sep 29 01:19:30 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 29 Sep 2009 11:19:30 +1200
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4AC089F7.8030004@voidspace.org.uk>
References: <4ABF2047.7050501@gmail.com> <h9njfi$9f4$2@ger.gmane.org>
	<4ABFCDED.8080105@gmail.com> <4ABFD9D6.9070602@voidspace.org.uk>
	<4AC00D2A.6020504@canterbury.ac.nz> <4AC089F7.8030004@voidspace.org.uk>
Message-ID: <4AC14482.1040807@canterbury.ac.nz>

Michael Foord wrote:

> Well, the file type knows what mode it is opened in so catching these 
> errors is really not difficult at all.

Yes, in this particular case it could do some checking of
its own before making the system call. But this still has
nothing to do with the exception hierarchy.

-- 
Greg

From fuzzyman at voidspace.org.uk  Tue Sep 29 01:21:50 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 29 Sep 2009 00:21:50 +0100
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4AC14482.1040807@canterbury.ac.nz>
References: <4ABF2047.7050501@gmail.com>
	<h9njfi$9f4$2@ger.gmane.org>	<4ABFCDED.8080105@gmail.com>
	<4ABFD9D6.9070602@voidspace.org.uk>	<4AC00D2A.6020504@canterbury.ac.nz>
	<4AC089F7.8030004@voidspace.org.uk>
	<4AC14482.1040807@canterbury.ac.nz>
Message-ID: <4AC1450E.1040207@voidspace.org.uk>

Greg Ewing wrote:
> Michael Foord wrote:
>
>> Well, the file type knows what mode it is opened in so catching these 
>> errors is really not difficult at all.
>
> Yes, in this particular case it could do some checking of
> its own before making the system call. But this still has
> nothing to do with the exception hierarchy.
>
You said that having an exception hierarchy was impossible because the 
only information we have is the error number. Some errors could easily 
be caught.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ncoghlan at gmail.com  Mon Sep 28 13:58:22 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 28 Sep 2009 21:58:22 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T185114-628@post.gmane.org>	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>	<loom.20090927T201052-343@post.gmane.org>	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>	<loom.20090927T220341-422@post.gmane.org>	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
Message-ID: <4AC0A4DE.4080402@gmail.com>

Peter Moody wrote:
>> Assuming the Network type loses the notion of a specific host (or host address,
>> or `ip`) attached to it, yes.
> 
> this is "less useful (strictly removing functionality)" and is an
> example of what I explicitly said I was not going to do with ipaddr.

A possible compromise then:

Just change the definition of network equality to only consider the
network address and ignore the ip attribute. Optionally provide a
separate comparison method that pays attention to the ip attribute
(keeping in mind that it isn't particularly hard for people to write out
"net1 == net2 and net1.ip == net2.ip" if they care about the extra state).

Then "IPNetwork" can be largely documented as a true IP network
definition, with the "Host with Netmask" use case handled by having an
IPNetwork object with "net.ip != net.network".

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From andrewm at object-craft.com.au  Tue Sep 29 03:40:18 2009
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Tue, 29 Sep 2009 11:40:18 +1000
Subject: [Python-Dev] PEP 3144 review, and the inclusion process
In-Reply-To: <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T201052-343@post.gmane.org>
	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>
	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>
	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>
	<loom.20090927T220341-422@post.gmane.org>
	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>
	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
	<loom.20090928T000905-169@post.gmane.org>
	<8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com>
Message-ID: <20090929014018.CEED559C0A4@longblack.object-craft.com.au>

>I've never said otherwise. In fact, from an email last night, "If what
>the community requires is the library you've described, then ipaddr is
>not that library." The changes *you* require make ipaddr significantly
>less useful to me. I'm not prepared to make those changes in an
>attempt seek acceptance to the stdlib, especially if the stdlib is in
>such flux that I'll get to do this again in 18 months.

The point is that, having brought it to us, we all now have an interest
in the outcome. Whatever goes into the standard library is going to be
something that we have to live with for a long time, and now is our best
chance to shape the result.

I understand your concern over introducing more classes, however I still
feel my suggested functional decomposition is worth that cost because
I consider the behaviour of my suggested classes to be more intuitive.
I should mention that I am not a computer scientist, and none of this is
motivated by a desire for theoretical purity - just practical experience.

One of my concerns now is that if a code block receives an IPv4Network
instance, it does not know whether this represents a host address
with mask, or a network address. In some contexts, this distinction is
critical, and confounding them can result in delayed error reporting or
erroneous behaviour. Your addition of a strict flag does not completely
address this concern as it assumes the instantiation and use occur in
proximity, which is often not the case in large projects.

I suspect you are also mistaken in thinking my proposed changes make
the module less useful for you - maybe you can describe the problem as
you see it?


As a reminder to people who have come late to this thread, I proposed three
classes per protocol:

    IPv?Address
        A single address

    IPv?AddressWithMask
        A single address with implied IPv?Network

    IPv?Network
        A container-like network address (with strict mask parsing)

Further:

 * Comparisons between classes should be disallowed. 

 * The IPv?AddressWithMask class would have a .address and .mask attributes
   containing IPv?Addresses, and a .network attribute for the containing
   network (as specified by the mask, and lazily constructed).

 * The IPv?Network class would have similar .address and .mask attributes.

In cases where you want to allow lax specification of network addresses,
this would be spelt:

    IPv?AddressWithMask(some_address).network

At first glance, this seems somewhat round-about, however it makes explicit
the potential loss of bits.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From rdmurray at bitdance.com  Tue Sep 29 04:17:54 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 28 Sep 2009 22:17:54 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de> 
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909282115200.18193@kimball.webabinitio.net>

On Mon, 28 Sep 2009 at 13:43, Guido van Rossum wrote:
> On Mon, Sep 28, 2009 at 1:36 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> I would say that there certainly are precedents in other areas for
> keeping the information about the input form around. For example,
> occasionally it would be handy if parsing a hex integer returned an
> object that was compatible with other integers but somehow kept a hint
> that would cause printing it to use hex by default.
>
> I see keeping around the IP address used to create a network object as
> a similar edge case. I would probably define the __eq__ method to
> implement equivalency (ignoring the form of the input), just as I
> would in the case of the (hypothetical) hex integers. If you wanted to
> do a comparison that includes the input IP address, you could use (a,
> a.ip) == (b, b.ip).

Ignoring Antoine's excellent points for the moment, to my mind
one difference between your integer-with-hex-hint and ipaddr's
ip-with-netmask is that there is a one to one mapping between the
canonical hex representations and the integer.

If you want to argue that the _exact_ input string be preserved in the
integer object, then there might be more of a direct analogy.  Except that
ipaddr isn't preserving the input _string_, it's preserving a canonical
representation of that string, so even there the analogy wouldn't be
all that close.  The difference here is that ipaddr is preserving input
information that is irrelevant to the object being returned (a network).

The fractions case is much closer.

Or consider another analogy: the ipaddr case is like the mod function
returning a single object that is supposed to be the remainder,
but also has an extra field that records the original input number.
This might be a useful operator and object type in certain contexts, but
it is something that would be layered on top of the real mod function
that returns a simple integer remainder.  You would never approve this
remainder-with-saved-input as the data type returned by the mod operator.
(I hope :)

Similarly, there should be a basic ipaddr parsing function (and I don't
much care how you spell it, even though I have opinions) that returns
an ip address and a network object when given an ipaddress-plus-netmask
string.  An additional object built upon those fundamental data types that
remembers the input IP would be fine, but the data type that remembers
the IP should _not_ be the fundamental data type in the system.  That
strikes me as poor design, OO or otherwise.

In fact, your "sometimes it would be useful if" phrasing indicates that
your 'integer with hex hint' data type would also not be the fundamental
data type, but instead a subclass of int.

--David

From martin at v.loewis.de  Tue Sep 29 04:18:25 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 29 Sep 2009 04:18:25 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to	ucs4
In-Reply-To: <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>	<4AB67430.2060306@egenix.com>	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>	<4AC07309.1010608@egenix.com>
	<4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net>
Message-ID: <4AC16E71.3040400@v.loewis.de>

James Y Knight wrote:
> On Sep 28, 2009, at 4:25 AM, M.-A. Lemburg wrote:
>> Distributions should really not be put in charge of upstream
>> coding design decisions.
> 
> I don't think you can blame distros for this one....
> 
> From PEP 0261:
>     It is also proposed that one day --enable-unicode will just
>     default to the width of your platforms wchar_t.
> 
> On linux, wchar_t is 4 bytes.
> 
> If there's a consensus amongst python upstream that all the distros
> should be shipping Python with UCS2 unicode strings, you should reach
> out to them and say this, in a rather more clear fashion. Currently,
> most signs point towards UCS4 builds as being the better option.

There is no such consensus. Linux distributions really should build
Python in UCS-4 mode, and I would be in favor of making it the default
to match wchar_t.

Regards,
Martin

From guido at python.org  Tue Sep 29 05:18:43 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 28 Sep 2009 20:18:43 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909282115200.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> 
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de> 
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net> 
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com> 
	<Pine.LNX.4.64.0909282115200.18193@kimball.webabinitio.net>
Message-ID: <ca471dc20909282018u53a9c255o65f3c5ba9d2c156d@mail.gmail.com>

On Mon, Sep 28, 2009 at 7:17 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Mon, 28 Sep 2009 at 13:43, Guido van Rossum wrote:
>>
>> On Mon, Sep 28, 2009 at 1:36 PM, R. David Murray <rdmurray at bitdance.com>
>> wrote:
>> I would say that there certainly are precedents in other areas for
>> keeping the information about the input form around. For example,
>> occasionally it would be handy if parsing a hex integer returned an
>> object that was compatible with other integers but somehow kept a hint
>> that would cause printing it to use hex by default.
>>
>> I see keeping around the IP address used to create a network object as
>> a similar edge case. I would probably define the __eq__ method to
>> implement equivalency (ignoring the form of the input), just as I
>> would in the case of the (hypothetical) hex integers. If you wanted to
>> do a comparison that includes the input IP address, you could use (a,
>> a.ip) == (b, b.ip).
>
> Ignoring Antoine's excellent points for the moment, to my mind
> one difference between your integer-with-hex-hint and ipaddr's
> ip-with-netmask is that there is a one to one mapping between the
> canonical hex representations and the integer.
>
> If you want to argue that the _exact_ input string be preserved in the
> integer object, then there might be more of a direct analogy. ?Except that
> ipaddr isn't preserving the input _string_, it's preserving a canonical
> representation of that string, so even there the analogy wouldn't be
> all that close. ?The difference here is that ipaddr is preserving input
> information that is irrelevant to the object being returned (a network).

I disagree that recording the exact input string would be a better
analogy; as you say the hypothetical hex integer doesn't save the
exact input string -- it remembers that the base was 16 That happens
to be 1 bit of information if you're only considering hex vs. decimal,
but one could imagine a further version that supports any base between
1 and 36 (or however far you want to go).

Whether I typed 01.01.01.01/024 or 1.1.1.1/24 is (presumably)
irrelevant for this case, since the byte values are the same; whether
I typed 1.1.1.1/24 or 1.1.1.0/24 *is* relevant (for Peter). There's
probably another form, 16843009/24, which is also equivalent.

> The fractions case is much closer.

Hardly -- I've never heard of someone who had a use case for
denormalized fractions, but I don't doubt that Peter has a use case
for denormalized IPNetwork objects. (Do you doubt that Peter has such
a use case? If so, we have a much more fundamental disagreement.)

> Or consider another analogy: the ipaddr case is like the mod function
> returning a single object that is supposed to be the remainder,
> but also has an extra field that records the original input number.
> This might be a useful operator and object type in certain contexts, but
> it is something that would be layered on top of the real mod function
> that returns a simple integer remainder. ?You would never approve this
> remainder-with-saved-input as the data type returned by the mod operator.
> (I hope :)

Not a very strong argument, since that use case is purely
hypothetical. I brought up the hex integer example to show that it is
possible to conceive of use cases for objects that record some sort of
information about their creation history. Now you're bending my line
of reasoning by going from the assertion "Guido would never approve of
mod-returning-remainder-with-saved-input" (which happens to be true
until a real use case is found) to "Guido would never approve of *any*
operator that keeps some traces of its creation history" (which is
false -- dict being just one counter-example :-).

> Similarly, there should be a basic ipaddr parsing function (and I don't
> much care how you spell it, even though I have opinions) that returns
> an ip address and a network object when given an ipaddress-plus-netmask
> string. An additional object built upon those fundamental data types that
> remembers the input IP would be fine, but the data type that remembers
> the IP should _not_ be the fundamental data type in the system. ?That
> strikes me as poor design, OO or otherwise.

Poor design is highly subjective, and I simply disagree that one
design is a priori better or worse than the other. This is Python,
where practicality beats purity, so things like expediency of
implementation, frequency of various uses, etc., matter.

Right now, without knowing more about Peter's use case, I'd sat that
__eq__ should ignore the .ip attribute, but that's more based on
trying to get enough people to drop their opposition than on a full
understanding of the use case. I do note that if Peter's use case is
at all common, reducing the number of classes is a worthy goal, and
Python has a bit of a history of preferring a small number of
Swiss-army-knife classes over a multitude of simple classes.

> In fact, your "sometimes it would be useful if" phrasing indicates that
> your 'integer with hex hint' data type would also not be the fundamental
> data type, but instead a subclass of int.

Which proves nothing, see above. As a matter of fact, my "sometimes it
would be useful" was meant as a gentle nudge in the direction of
keeping the .ip attribute.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Tue Sep 29 05:44:30 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 29 Sep 2009 05:44:30 +0200
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
Message-ID: <4AC1829E.50802@v.loewis.de>

> Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])``
> as an example and simply assume that 'alpha' takes a string as an
> argument and that it's required and that 'beta' is a boolean flag. To
> pull everything out you could do::
> 
>   options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])
>   options_dict = dict(options)
>   alpha = options_dict.get('-a', options_dict.get('--alpha', ''))
>   beta = '-b' in options_dict or '--beta' in options_dict
> 
>   main(alpha, beta, args)
> 
> Obviously if one of the getopt supporters has a better way of doing
> this then please speak up.

As Yuvgoog Greenle says, the canonical getopt way is to write

alpha = None
beta = False
options, args = getopt.getopt(sys.argv[1:],"a:b",['alpha=','beta']):
for opt, val in options:
    if arg in ('-a','--alpha'):
        alpha = val
    elif arg in ('-b','--beta'):
        beta = True
main(alpha, beta, args)

Even though this is many more lines, I prefer it over
optparse/argparse: this code has only a single function call,
whereas the argparse version has three function calls to remember.
The actual processing uses standard Python data structures which
I don't need to look up in the documentation.

> Now, Steven, can you show how best to do this in argparse?

This demonstrates my point: you were able to use getopt right away
(even though not in the traditional way), whereas you need to ask
for help on using argparse properly.

> I am
> willing to bet that the total number of lines to do this is not that
> much more and does not require you to know to use 'or' or the dict
> constructor along with dict.get() in order to keep it compact.

See above - getopt users don't care about compactness in
the processing.

> I can
> only imagine what some newbie might try to do in order to be correct
> (if they even try).

Depends on the background of the newbie. If they come from C, they
immediately recognize the way of doing things.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Tue Sep 29 07:20:13 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 29 Sep 2009 17:20:13 +1200
Subject: [Python-Dev] IO module precisions and exception hierarchy
In-Reply-To: <4AC1450E.1040207@voidspace.org.uk>
References: <4ABF2047.7050501@gmail.com> <h9njfi$9f4$2@ger.gmane.org>
	<4ABFCDED.8080105@gmail.com> <4ABFD9D6.9070602@voidspace.org.uk>
	<4AC00D2A.6020504@canterbury.ac.nz> <4AC089F7.8030004@voidspace.org.uk>
	<4AC14482.1040807@canterbury.ac.nz> <4AC1450E.1040207@voidspace.org.uk>
Message-ID: <4AC1990D.8060706@canterbury.ac.nz>

Michael Foord wrote:

> You said that having an exception hierarchy was impossible because the 
> only information we have is the error number.

No, I said that having an exception hierarchy wouldn't
help you to sort out the different meanings of a single
error code.

If you have some *other* way of sorting them out, then
obviously you can represent them using different
exceptions if you want.

-- 
Greg

From steven.bethard at gmail.com  Tue Sep 29 07:22:26 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 28 Sep 2009 22:22:26 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC1829E.50802@v.loewis.de>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<4AC1829E.50802@v.loewis.de>
Message-ID: <d11dcfba0909282222o5fd345een31b5f973501bf078@mail.gmail.com>

On Mon, Sep 28, 2009 at 8:44 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])``
>
> As Yuvgoog Greenle says, the canonical getopt way is to write
[snip getopt code]
> Even though this is many more lines, I prefer it over
> optparse/argparse: this code has only a single function call,
> whereas the argparse version has three function calls to remember.
> The actual processing uses standard Python data structures which
> I don't need to look up in the documentation.
>
>> Now, Steven, can you show how best to do this in argparse?
>
> This demonstrates my point: you were able to use getopt right away
> (even though not in the traditional way), whereas you need to ask
> for help on using argparse properly.

I don't think this is fair at all. I am totally unable to write getopt
code without checking the documentation -- I don't remember the format
string syntax, nor what the function returns. But that's just how
library modules work -- if you don't know the library, you have to
read the documentation. The only reason getopt is easier for you is
that you're already familiar with the API from C. That said, I can
certainly understand that folks who use getopt in C would have an easy
transition to getopt in Python and a harder transition to argparse.

You didn't directly answer my question of whether adding an
"add_getopt_arguments" function would meet your needs, but I infer
from your answer above that it wouldn't because you'd still need to
make multiple function calls which is your primary complaint. So at
this point, I think it's clear that there's nothing I can reasonably
add to argparse to make getopt people more comfortable that isn't just
duplicating what's already in getopt.

So let's just deprecate optparse (as in the PEP), and leave getopt
alone (other than adding a note to the documentation suggesting
argparse as an alternative).
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From greg.ewing at canterbury.ac.nz  Tue Sep 29 08:11:32 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 29 Sep 2009 18:11:32 +1200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909282018u53a9c255o65f3c5ba9d2c156d@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<Pine.LNX.4.64.0909282115200.18193@kimball.webabinitio.net>
	<ca471dc20909282018u53a9c255o65f3c5ba9d2c156d@mail.gmail.com>
Message-ID: <4AC1A514.6000000@canterbury.ac.nz>

Guido van Rossum wrote:
> I've never heard of someone who had a use case for
> denormalized fractions, but I don't doubt that Peter has a use case
> for denormalized IPNetwork objects. (Do you doubt that Peter has such
> a use case? If so, we have a much more fundamental disagreement.)

It would be interesting to know what his use case is. Maybe
it can be addressed in a different way that doesn't require
denormalised network objects.

 > my "sometimes it
> would be useful" was meant as a gentle nudge in the direction of
> keeping the .ip attribute.

I think the point is that it would have to be useful a
*lot* of the time, not just "sometimes", to be worth
having it on every IPNetwork object.

-- 
Greg


From stephen at xemacs.org  Tue Sep 29 11:18:36 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 29 Sep 2009 18:18:36 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909282018u53a9c255o65f3c5ba9d2c156d@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<Pine.LNX.4.64.0909282115200.18193@kimball.webabinitio.net>
	<ca471dc20909282018u53a9c255o65f3c5ba9d2c156d@mail.gmail.com>
Message-ID: <87bpkukt1v.fsf@uwakimon.sk.tsukuba.ac.jp>

Guido van Rossum writes:

 > I don't doubt that Peter has a use case for denormalized IPNetwork
 > objects.

If you know what it is, would you please describe it, or say "that's
proprietary"?  Peter hasn't done either, despite repeated requests.

 > I do note that if Peter's use case is at all common, reducing the
 > number of classes is a worthy goal, and Python has a bit of a
 > history of preferring a small number of Swiss-army-knife classes
 > over a multitude of simple classes.

Even if Peter's use case turns out to be at all common, two things
bother me a lot.

First, 

                   IPv4Network(a) == IPv4Network(b)

has an obvious preferred interpretation as pseudocode.

Second, equality comparison for the "abuse" of a network class to
represent host-with-network-info can be implemented cleanly[1] as

                   x = IPv4Network(a)
                   y = IPv4Network(b)
                   (x.ip, x) == (y.ip, y)

as you've pointed out.  It is self-documenting in that makes it plain
that some things that are not "true networks" are being compared.
OTOH,

         x = IPv4Network(a)
         y = IPv4Network(b)
         (x.network, x.prefixlen) == (y.network, y.prefixlen)
         # alternatively
         # list(x) == list(y)

looks like nothing so much as an incomplete implementation to me.  It
just makes me itch to provide the appropriate definitions of __hash__
and __equal__ for IPv4Network.

Do you feel differently?

Footnotes: 
[1]  It's clean only if we assume you've accepted the "abuse", of course.


From ronaldoussoren at mac.com  Tue Sep 29 12:42:13 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 29 Sep 2009 12:42:13 +0200
Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on
	trunk
In-Reply-To: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>
References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>
Message-ID: <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com>


On 26 Sep, 2009, at 14:46, Barry Scott wrote:

> I'm working with http://svn.python.org/projects/python/trunk on Mac  
> OS X 10.6.1
> using Apples xcode gcc 4.2.1.
>
> When I run the following commands:
>
> 	./configure --enable-framework --with-universal-archs=32-bit | tee  
> build.config.log
> 	make clean all | tee build.make.log
>
> I end up with a x86_64 Python image.
>
> No matter what I use for archs its always the same.
>
> I would expect to see -arch arg to GCC but it is not there.
>
> 	export CFLAG="-arch i386"
>
> did not work either.
>
> Am I doing something wrong or is this broken on trunk?

Use:

     ./configure --enable-framework --enable-universalsdk=/

The --with-universal-archs flag selects whichs architectures should be  
included when you build a universal binary, defaulting to 32-bit.

I'll write some documentation on the build options on OSX,  but don't  
know what's the best location to do so.

I guess I should also add logic to configure that bails out when you  
specifiy --with-universal-archs without asking for a universal build.

Ronald
>
> Barry
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/746987e1/attachment.bin>

From steve at pearwood.info  Tue Sep 29 13:20:03 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 29 Sep 2009 21:20:03 +1000
Subject: [Python-Dev] [OT] Denormalized fractions [was Re: PEP 3144 review.]
In-Reply-To: <ca471dc20909282018u53a9c255o65f3c5ba9d2c156d@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909282115200.18193@kimball.webabinitio.net>
	<ca471dc20909282018u53a9c255o65f3c5ba9d2c156d@mail.gmail.com>
Message-ID: <200909292120.03677.steve@pearwood.info>

On Tue, 29 Sep 2009 01:18:43 pm Guido van Rossum wrote:
> I've never heard of someone who had a use case for
> denormalized fractions

Off-topic, but for what it's worth, I have one -- there's a mathematical 
operator called the mediant:

mediant(a/b, c/d) = (a+c)/(b+d)

It has a few uses, including Farey fractions. Clearly the result you get 
from normalized fractions will be different from denormalized (compare 
mediant(1/2, 3/4) with mediant(5/10, 3/4)). This leads to Simpson's 
Paradox, which is of importance in medical research:

http://en.wikipedia.org/wiki/Simpson's_paradox

Brief summary: consider two medical studies comparing two different 
treatments for an illness, treatment A and B. According to the first 
study, treatment A is better; according to the second study, treatment 
A is also better. But combining the results of the two studies into a 
single comparison paradoxically shows that treatment B is better!

The mediant is fascinating (to maths geeks at least) and important, and 
you need denormalized fractions.



-- 
Steven D'Aprano

From skip at pobox.com  Tue Sep 29 13:48:36 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 29 Sep 2009 06:48:36 -0500
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC1829E.50802@v.loewis.de>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<4AC1829E.50802@v.loewis.de>
Message-ID: <19137.62484.513693.107525@montanaro.dyndns.org>

    Martin> alpha = None
    Martin> beta = False
    Martin> options, args = getopt.getopt(sys.argv[1:],"a:b",['alpha=','beta']):
    Martin> for opt, val in options:
    ...

    Martin> Even though this is many more lines, I prefer it over
    Martin> optparse/argparse: this code has only a single function call,
    ...

Agreed.  I have never completely wrapped my brain around optparse.  Getopt I
just remember.

Skip

From skip at pobox.com  Tue Sep 29 13:56:29 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 29 Sep 2009 06:56:29 -0500
Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on
 trunk
In-Reply-To: <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com>
References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>
	<8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com>
Message-ID: <19137.62957.761923.573377@montanaro.dyndns.org>


    Ronald> I'll write some documentation on the build options on OSX, but
    Ronald> don't know what's the best location to do so.

The top-level README file of the distribution has a "Platform specific
notes" section.  Seems like that would be the most logical place to stuff
such info.

Skip

From solipsis at pitrou.net  Tue Sep 29 15:23:32 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 29 Sep 2009 13:23:32 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Fuzziness_in_io_module_specs_-_PEP_update?=
	=?utf-8?q?=09proposition_V2?=
References: <4AB3DCC9.5050209@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz>
	<4AB5FA81.5060408@gmail.com>
	<eae285400909200743r162dee46pfebf6cf34407badd@mail.gmail.com>
	<4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com>
	<4ABF1B3B.4080106@gmail.com> <h9nihf$9f4$1@ger.gmane.org>
	<ca471dc20909271424l57315d4ft18d4bbe7da8bdd4@mail.gmail.com>
	<h9ops5$hkf$2@ger.gmane.org>
	<ca471dc20909281033w4b9c78d9mf56212c42253bfb2@mail.gmail.com>
Message-ID: <loom.20090929T152217-619@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> All in all I think we should change this before it's too late; it will
> affect a very small number of apps (perhaps none?), but I would rather
> have the right semantics in the future. Also, it's trivial to write
> code that doesn't care (in fact code running under 2.x and 3.x
> probably will have to be written so that it doesn't care) so it's not
> like changing this is going to make life harder for people wanting
> multiple-version support.

Ok, it sounds reasonable to me :)
Now somebody just has to write a patch (it shouldn't be too difficult, since the
position restoring code exists in the 2.x file object).

Regards

Antoine.



From bjourne at gmail.com  Tue Sep 29 17:17:25 2009
From: bjourne at gmail.com (=?ISO-8859-1?Q?Bj=F6rn_Lindqvist?=)
Date: Tue, 29 Sep 2009 17:17:25 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
	to ucs4
In-Reply-To: <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
	<4AB67430.2060306@egenix.com>
	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
	<4AC07309.1010608@egenix.com>
	<4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net>
Message-ID: <740c3aec0909290817r78d8c5e3t2823f8c01aa4642a@mail.gmail.com>

2009/9/28 James Y Knight <foom at fuhm.net>:
>> People building their own Python version will usually also build
>> their own extensions, so I don't really believe that the above
>> scenario is very common.
>
> I'd just like to note that I've run into this trap multiple times. I built a
> custom python, and expected it to work with all the existing, installed,
> extensions (same major version as the system install, just patched). And
> then had to build it again with UCS4, for it to actually work. Of course
> building twice isn't the end of the world, and I'm certainly used to having
> to twiddle build options on software to get it working, but, this *does*
> happen, and *is* a tiny bit irritating.

I've also encountered this trap multiple times. Obviously, the problem
is not rebuilding Python which is quick, but to figure out the correct
configure option to use (--enable-unicode=ucs4). Others have also
spent some time scratching their heads over the strange
PyUnicodeUCS4_FromUnicode error the misconfiguration results in, as
Zooko's links show.

If Python can't infer the unicode setting from the width of the
platforms wchar_t, then perhaps it should be mandatory to specify to
configure whether you want UCS2 or UCS4? For someone clueless like me,
it would be easier to deal with the problem upfront than (much)
further down the line. Explicit being better than implicit and all
that.


-- 
mvh Bj?rn

From mal at egenix.com  Tue Sep 29 17:28:59 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 29 Sep 2009 17:28:59 +0200
Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on
	trunk
In-Reply-To: <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com>
References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>
	<8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com>
Message-ID: <4AC227BB.9000607@egenix.com>


Ronald Oussoren wrote:
> 
> On 26 Sep, 2009, at 14:46, Barry Scott wrote:
> 
>> I'm working with http://svn.python.org/projects/python/trunk on Mac OS
>> X 10.6.1
>> using Apples xcode gcc 4.2.1.
>>
>> When I run the following commands:
>>
>>     ./configure --enable-framework --with-universal-archs=32-bit | tee
>> build.config.log
>>     make clean all | tee build.make.log
>>
>> I end up with a x86_64 Python image.
>>
>> No matter what I use for archs its always the same.
>>
>> I would expect to see -arch arg to GCC but it is not there.
>>
>>     export CFLAG="-arch i386"
>>
>> did not work either.
>>
>> Am I doing something wrong or is this broken on trunk?
> 
> Use:
> 
>     ./configure --enable-framework --enable-universalsdk=/
> 
> The --with-universal-archs flag selects whichs architectures should be
> included when you build a universal binary, defaulting to 32-bit.

The Python default on 10.6 is 64-bit, so wouldn't it be better
to default to that on 10.6 and use 32-bit as default on 10.3/4/5 ?!

> I'll write some documentation on the build options on OSX,  but don't
> know what's the best location to do so.

Please put that information into Mac/README which already has
documentation for how to build universal binaries on Mac.

> I guess I should also add logic to configure that bails out when you
> specifiy --with-universal-archs without asking for a universal build.

That would be useful.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 29 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From solipsis at pitrou.net  Tue Sep 29 17:33:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 29 Sep 2009 15:33:19 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?please_consider_changing_--enable-unicode_?=
	=?utf-8?q?default=09to_ucs4?=
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
	<4AB67430.2060306@egenix.com>
	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
	<4AC07309.1010608@egenix.com>
	<4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net>
	<740c3aec0909290817r78d8c5e3t2823f8c01aa4642a@mail.gmail.com>
Message-ID: <loom.20090929T173025-721@post.gmane.org>

Hello,

> I've also encountered this trap multiple times. Obviously, the problem
> is not rebuilding Python which is quick, but to figure out the correct
> configure option to use (--enable-unicode=ucs4). Others have also
> spent some time scratching their heads over the strange
> PyUnicodeUCS4_FromUnicode error the misconfiguration results in, as
> Zooko's links show.

Isn't this overrated?

First, if you have a Python version that has the wrong version, just print out
its sys.maxunicode and choose the right version according to that (if
sys.maxunicode == 65535, you need to compile an UCS-4 version, otherwise an
UCS-2 version).

Second, once you have encountered this issue, you know what you need the
subsequent times. There are only two possibilities after all.

> If Python can't infer the unicode setting from the width of the
> platforms wchar_t, then perhaps it should be mandatory to specify to
> configure whether you want UCS2 or UCS4? For someone clueless like me,
> it would be easier to deal with the problem upfront than (much)
> further down the line.

I'm not sure why someone "clueless" (your word :-)) wants to compile his own
Python, though.

Regards

Antoine.



From ronaldoussoren at mac.com  Tue Sep 29 17:44:40 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 29 Sep 2009 17:44:40 +0200
Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on
	trunk
In-Reply-To: <4AC227BB.9000607@egenix.com>
References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>
	<8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com>
	<4AC227BB.9000607@egenix.com>
Message-ID: <9015487A-AA0F-4448-8ADF-C2B6E7B65377@mac.com>


On 29 Sep, 2009, at 17:28, M.-A. Lemburg wrote:

>
> Ronald Oussoren wrote:
>>
>> On 26 Sep, 2009, at 14:46, Barry Scott wrote:
>>
>>> I'm working with http://svn.python.org/projects/python/trunk on  
>>> Mac OS
>>> X 10.6.1
>>> using Apples xcode gcc 4.2.1.
>>>
>>> When I run the following commands:
>>>
>>>    ./configure --enable-framework --with-universal-archs=32-bit |  
>>> tee
>>> build.config.log
>>>    make clean all | tee build.make.log
>>>
>>> I end up with a x86_64 Python image.
>>>
>>> No matter what I use for archs its always the same.
>>>
>>> I would expect to see -arch arg to GCC but it is not there.
>>>
>>>    export CFLAG="-arch i386"
>>>
>>> did not work either.
>>>
>>> Am I doing something wrong or is this broken on trunk?
>>
>> Use:
>>
>>    ./configure --enable-framework --enable-universalsdk=/
>>
>> The --with-universal-archs flag selects whichs architectures should  
>> be
>> included when you build a universal binary, defaulting to 32-bit.
>
> The Python default on 10.6 is 64-bit, so wouldn't it be better
> to default to that on 10.6 and use 32-bit as default on 10.3/4/5 ?!

Defaulting to a 32-bit build has several advantages. The first is that  
the defaults match the binary installer on the python.org website, and  
secondly there are still 3th-party libraries that don't work in 64-bit  
mode (mostly GUI libraries, until recently Tk and wxWidgets wrapped  
the Carbon libraries which are not available in 64-bit mode; AFAIK  
both have betas that wrap the Cocoa libraries instead).

To mimick the system default you'd have to default to 32-bit on 10.4,  
4-way universal on 10.5 and 3-way universal on 10.6, and that is  
without considering deployment targets.  All of those are available as  
options, I'd prefer to keep it this way for now to keep things simple.

>
>> I'll write some documentation on the build options on OSX,  but don't
>> know what's the best location to do so.
>
> Please put that information into Mac/README which already has
> documentation for how to build universal binaries on Mac.

I know that, I wrote most of that file ;-).
>
>> I guess I should also add logic to configure that bails out when you
>> specifiy --with-universal-archs without asking for a universal build.
>
> That would be useful.

I've committed a patch to the trunk and 2.6 that does this.

Ronald
>
> -- 
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Sep 29  
> 2009)
>>>> Python/Zope Consulting and Support ...        http:// 
>>>> www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ...             http:// 
>>>> zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http:// 
>>>> python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for  
> free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/9b653822/attachment.bin>

From solipsis at pitrou.net  Tue Sep 29 18:03:16 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 29 Sep 2009 18:03:16 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
Message-ID: <1254240196.5662.5.camel@localhost>


> I've mentioned at least once (and others have mentioned as well) that
> it's very common, when describing the ip address/prefix of a NIC to
> write, "192.168.1.100/24"

You say it yourself : it describes "the ip address/prefix of a NIC".
It isn't the job of a Network class. A Network shouldn't describe a
host, or a particular NIC.

Regards

Antoine.



From peter at hda3.com  Tue Sep 29 17:36:02 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 29 Sep 2009 08:36:02 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
Message-ID: <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>

Responding to a few points here. David and I were discussing these
points off-list, I didn't mean to ignore.

  re: assumed disconnect between the statement "Addresses and Networks
are distinct" and the implementation.

I don't actually see a disconnect.  it seems analogous to stating
lists and ints are distinct yet lists can still contain ints. networks
and addresses are distinct and yet networks contain addresses.

  re: denormalized networks:

I've mentioned at least once (and others have mentioned as well) that
it's very common, when describing the ip address/prefix of a NIC to
write, "192.168.1.100/24"

  re: new comparison methods

this is interesting. I would be open to doing this and making __lt__,
__gt__, __eq__ and friends do the non-ip comparing by default. would
this be sufficient or does the fact that the network has the .ip
attribute break something pythonic?

Cheers,
/peter

On Mon, Sep 28, 2009 at 1:43 PM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Sep 28, 2009 at 1:36 PM, R. David Murray <rdmurray at bitdance.com> wrote:
>> On Mon, 28 Sep 2009 at 22:11, "Martin v. L?wis" wrote:
>>>>
>>>> Martin v. L?wis <martin <at> v.loewis.de> writes:
>>>>>>
>>>>>> Could you explain what benefit there is for allowing the user to create
>>>>>> network objects that don't represent networks? Is there a use-case
>>>>>> where these networks-that-aren't-networks are something other than a
>>>>>> typo? Under what circumstances would I want to specify a network as
>>>>>> 192.168.1.1/24 instead of 192.168.1.0/24?
>>>>>>
>>>> [...]
>>>>>
>>>>> So Python code has to make the computation, and it seems most natural
>>>>> that the IP library is the piece of code that is able to compute a
>>>>> network out of that input.
>>>>
>>>> The thing is, it doesn't create a network, it creates a hybrid "network
>>>> plus
>>>> host" which retains knowledge about the original host (that is,
>>>> 192.168.1.1
>>>> rather than simply 192.168.1.0, if you enter "192.168.1.1/24").
>>>>
>>>> That's what the OP meant with "networks-that-aren't-networks", and that's
>>>> what
>>>> people are objecting to.
>>>
>>> That's not the question that was asked, though - the question asked
>>> was "Under what circumstances would I want to specify...". I hope
>>> most people agree that it is desirable to be able to specify a network
>>> not just by its network address.
>>
>> But then it's not a network, it is an ipaddress-plus-mask. ?It is exactly
>> that conflation that we are objecting to. ?There is no question about
>> the use case of _specifying_ a network ip plus a netmask and deriving
>> a network object from that. ?That is unquestionably required by any
>> useful API. ?The argument is about whether the object returned is a
>> Network object, or a hybrid object representing _both_ an IP address
>> and a network. ?It is the latter, which ipaddr does, which many of us
>> find problematic and likely to lead to hard to read programs that will
>> probably produce maintenance errors.
>>
>> I observe that this line in the current PEP rationale:
>>
>> ? ?IP addresses and IP networks are distinct.
>>
>> is not in fact achieved by the reference implementation. ?Peter, however,
>> clearly thinks it is, since it is listed as a design goal of ipaddr.
>> This is a language disconnect I don't understand, which I think has
>> been the source of a lot of the difficulties in this thread.
>
> I say the case has been argued extensively, let's wait for Peter to respond.
>
> I would say that there certainly are precedents in other areas for
> keeping the information about the input form around. For example,
> occasionally it would be handy if parsing a hex integer returned an
> object that was compatible with other integers but somehow kept a hint
> that would cause printing it to use hex by default.
>
> I see keeping around the IP address used to create a network object as
> a similar edge case. I would probably define the __eq__ method to
> implement equivalency (ignoring the form of the input), just as I
> would in the case of the (hypothetical) hex integers. If you wanted to
> do a comparison that includes the input IP address, you could use (a,
> a.ip) == (b, b.ip).
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

From mal at egenix.com  Tue Sep 29 18:17:15 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 29 Sep 2009 18:17:15 +0200
Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on
	trunk
In-Reply-To: <9015487A-AA0F-4448-8ADF-C2B6E7B65377@mac.com>
References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>	<8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com>	<4AC227BB.9000607@egenix.com>
	<9015487A-AA0F-4448-8ADF-C2B6E7B65377@mac.com>
Message-ID: <4AC2330B.1000405@egenix.com>

Ronald Oussoren wrote:
> 
>>> Use:
>>>
>>>    ./configure --enable-framework --enable-universalsdk=/
>>>
>>> The --with-universal-archs flag selects whichs architectures should be
>>> included when you build a universal binary, defaulting to 32-bit.
>>
>> The Python default on 10.6 is 64-bit, so wouldn't it be better
>> to default to that on 10.6 and use 32-bit as default on 10.3/4/5 ?!
> 
> Defaulting to a 32-bit build has several advantages. The first is that
> the defaults match the binary installer on the python.org website,

What build options does that installer use ? (the web-page doesn't say)

> and
> secondly there are still 3th-party libraries that don't work in 64-bit
> mode (mostly GUI libraries, until recently Tk and wxWidgets wrapped the
> Carbon libraries which are not available in 64-bit mode; AFAIK both have
> betas that wrap the Cocoa libraries instead).
>
> To mimick the system default you'd have to default to 32-bit on 10.4,
> 4-way universal on 10.5 and 3-way universal on 10.6, and that is without
> considering deployment targets.  All of those are available as options,
> I'd prefer to keep it this way for now to keep things simple.

Hmm, so I guess the only way to support them all is by building extensions
using 4-way universal on 10.5. No wonder they are called "fat" binaries ;-)

>>> I'll write some documentation on the build options on OSX,  but don't
>>> know what's the best location to do so.
>>
>> Please put that information into Mac/README which already has
>> documentation for how to build universal binaries on Mac.
> 
> I know that, I wrote most of that file ;-).

Which is why I was surprised you asked :-)

>>> I guess I should also add logic to configure that bails out when you
>>> specifiy --with-universal-archs without asking for a universal build.
>>
>> That would be useful.
> 
> I've committed a patch to the trunk and 2.6 that does this.

Thanks.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 29 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ronaldoussoren at mac.com  Tue Sep 29 18:22:25 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 29 Sep 2009 18:22:25 +0200
Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on
	trunk
In-Reply-To: <4AC2330B.1000405@egenix.com>
References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>
	<8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com>
	<4AC227BB.9000607@egenix.com>
	<9015487A-AA0F-4448-8ADF-C2B6E7B65377@mac.com>
	<4AC2330B.1000405@egenix.com>
Message-ID: <15E9DA97-B2CF-4F01-8375-F2169E598CB6@mac.com>


On 29 Sep, 2009, at 18:17, M.-A. Lemburg wrote:

> Ronald Oussoren wrote:
>>
>>>> Use:
>>>>
>>>>   ./configure --enable-framework --enable-universalsdk=/
>>>>
>>>> The --with-universal-archs flag selects whichs architectures  
>>>> should be
>>>> included when you build a universal binary, defaulting to 32-bit.
>>>
>>> The Python default on 10.6 is 64-bit, so wouldn't it be better
>>> to default to that on 10.6 and use 32-bit as default on 10.3/4/5 ?!
>>
>> Defaulting to a 32-bit build has several advantages. The first is  
>> that
>> the defaults match the binary installer on the python.org website,
>
> What build options does that installer use ? (the web-page doesn't  
> say)

The installer is build using the script in Mac/BuildScript, and uses -- 
enable-framework --with-universalsk.

This creates a 32-bit fat build that runs on 10.3.9 or later.

>
>> and
>> secondly there are still 3th-party libraries that don't work in 64- 
>> bit
>> mode (mostly GUI libraries, until recently Tk and wxWidgets wrapped  
>> the
>> Carbon libraries which are not available in 64-bit mode; AFAIK both  
>> have
>> betas that wrap the Cocoa libraries instead).
>>
>> To mimick the system default you'd have to default to 32-bit on 10.4,
>> 4-way universal on 10.5 and 3-way universal on 10.6, and that is  
>> without
>> considering deployment targets.  All of those are available as  
>> options,
>> I'd prefer to keep it this way for now to keep things simple.
>
> Hmm, so I guess the only way to support them all is by building  
> extensions
> using 4-way universal on 10.5. No wonder they are called "fat"  
> binaries ;-)

I like the technology though, much more convenient than having  
parallel directory tries as on Linux.

>
>>>> I'll write some documentation on the build options on OSX,  but  
>>>> don't
>>>> know what's the best location to do so.
>>>
>>> Please put that information into Mac/README which already has
>>> documentation for how to build universal binaries on Mac.
>>
>> I know that, I wrote most of that file ;-).
>
> Which is why I was surprised you asked :-)

I hoped to find a document on docs.python.org that explains how to  
install Python, but sadly enough there isn't. Mac/README contains the  
right information, but isn't easily found if you're searching on the  
web or even if you're looking for documentation in the source tree.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/304004e4/attachment.bin>

From guido at python.org  Tue Sep 29 19:00:39 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Sep 2009 10:00:39 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <1254240196.5662.5.camel@localhost>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> 
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de> 
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net> 
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com> 
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> 
	<1254240196.5662.5.camel@localhost>
Message-ID: <ca471dc20909291000i5098e311o547f374f06e34091@mail.gmail.com>

On Tue, Sep 29, 2009 at 9:03 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>> I've mentioned at least once (and others have mentioned as well) that
>> it's very common, when describing the ip address/prefix of a NIC to
>> write, "192.168.1.100/24"
>
> You say it yourself : it describes "the ip address/prefix of a NIC".
> It isn't the job of a Network class. A Network shouldn't describe a
> host, or a particular NIC.

Hey Antoine,

Can we drop the pedantic discussion about what "should" or "shouldn't"
be the job of a "Network" class, and just proceed to a pragmatic
compromise? Peter has already posted that he is okay with __eq__ and
friends ignoring the .ip attribute, which sounds good enough to me.
His use case (which he mentioned to me off-list) is simply that if the
denormalized .ip attribute weren't saved as part of the IPNetwork
class, in many cases he'd have to keep track of it separately, which
just feels clumsy.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From zookog at gmail.com  Tue Sep 29 19:03:25 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Tue, 29 Sep 2009 11:03:25 -0600
Subject: [Python-Dev] please consider changing --enable-unicode default
	to ucs4
In-Reply-To: <4AC07309.1010608@egenix.com>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>
	<loom.20090920T162358-891@post.gmane.org>
	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>
	<4AB67430.2060306@egenix.com>
	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>
	<4AC07309.1010608@egenix.com>
Message-ID: <cd6401a0909291003h31076a34o1ad9770ae5dfebbc@mail.gmail.com>

Dear MAL and python-dev:

I failed to explain the problem that users are having.  I will try
again, and this time I will omit my ideas about how to improve things
and just focus on describing the problem.

Some users are having trouble using Python packages containing binary
extensions on Linux.  I want to provide such binary Python packages
for Linux for the pycryptopp project
(http://allmydata.org/trac/pycryptopp ) and the zfec project
(http://allmydata.org/trac/zfec ).  I also want to make it possible
for users to install the Tahoe-LAFS project (http://allmydata.org )
without having a compiler or Python header files.  (You'd be surprised
at how often Tahoe-LAFS users try to do this on Linux.  Linux is no
longer only for people who have the knowledge and patience to compile
software themselves.)  Tahoe-LAFS also depends on many packages that
are maintained by other people and are not packaged or distributed by
me -- pyOpenSSL, simplejson, etc..

There have been several hurdles in the way that we've overcome, and no
doubt there will be more, but the current hurdle is that there are two
"formats" for Python extension modules that are used on Linux -- UCS2
and UCS4.  If a user gets a Python package containing a compiled
extension module which was built for the wrong UCS2/4 setting, he will
get mysterious (to him) "undefined symbol" errors at import time.

On Mon, Sep 28, 2009 at 2:25 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>
> The Python default is UCS2 for a good reason: it's a good trade-off
> between memory consumption, functionality and performance.

I'm sure you are right about this.  At some point I will try to
measure the performance implications in the context of our
application.  I don't think it will be an issue for us, as so far no
users have complained about any performance or functionality problems
that were traceable to the choice of UCS2/4.

> As already mentioned, I also don't understand how the changing
> the Python default on Linux would help your users in any way -
> if you let distutils compile your extensions, it's automatically
> going to use the right Unicode setting for you (as well as your
> users).

My users are using some Python packages built by me and some built by
others.  The binary packages they get from others could have the
incompatible UCS2/4 setting.  Also some of my users might be using a
python configured with the opposite setting of the python interpreter
that I use to build packages.

> Unfortunately, this automatic support doesn't help you when
> shipping e.g. setuptools eggs, but this is a tool problem,
> not one of Python: setuptools completely ignores the fact
> that there are two ways to build Python.

This is the setuptools/distribute issue that I mentioned:
http://bugs.python.org/setuptools/issue78 .  If that issue were solved
then if a user tried to install a specific package, for example with a
command-line like "easy_install
http://allmydata.org/source/tahoe/deps/tahoe-dep-eggs/pyOpenSSL-0.8-py2.5-linux-i686.egg",
then instead of getting an undefined symbol error at import time, they
would get an error message to the effect of "This package is not
compatible with your Python interpreter." at install time.  That would
be good because it would be less confusing to the users.

However, if they were using the default setuptools/distribute
dependency-satisfaction feature, e.g. because they are installing a
package and that package is marked as
"install_requires=['pyOpenSSL']", then setuptools/distribute would do
its fallback behavior in which it attempts to compile the package from
source when it can't find a compatible binary package.  This would
probably confuse the users at least as much as the undefined symbol
error currently does.

In any case, improving the tools to handle incompatible packages
nicely would not make more packages compatible.  Let's do both!
Improve tools to handle incompatible packages nicely, and encourage
everyone who compiles python on Linux to use the same UCS2/4
setting.

Thank you for your attention.

Regards,

Zooko

From glyph at twistedmatrix.com  Tue Sep 29 20:19:10 2009
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Tue, 29 Sep 2009 14:19:10 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909291000i5098e311o547f374f06e34091@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
	<1254240196.5662.5.camel@localhost>
	<ca471dc20909291000i5098e311o547f374f06e34091@mail.gmail.com>
Message-ID: <d9047f780909291119v11c43e77ue19e8b0d634bd44e@mail.gmail.com>

On Tue, Sep 29, 2009 at 1:00 PM, Guido van Rossum <guido at python.org> wrote:

> On Tue, Sep 29, 2009 at 9:03 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
>
> You say it yourself : it describes "the ip address/prefix of a NIC".
> > It isn't the job of a Network class. A Network shouldn't describe a
> > host, or a particular NIC.
>
> Hey Antoine,
>
> Can we drop the pedantic discussion about what "should" or "shouldn't"
> be the job of a "Network" class, and just proceed to a pragmatic
> compromise? Peter has already posted that he is okay with __eq__ and
> friends ignoring the .ip attribute, which sounds good enough to me.
> His use case (which he mentioned to me off-list) is simply that if the
> denormalized .ip attribute weren't saved as part of the IPNetwork
> class, in many cases he'd have to keep track of it separately, which
> just feels clumsy.
>

I apologize in advance for missing a message that answers my question; I've
done my best to read all the related traffic in the various threads that
discuss this, but I'm sure I missed something.

I don't see what's particularly "pragmatic", in terms of functionality,
about confusing the responsibility of the Network class.  Networks don't
have addresses, in the sense that is being discussed here.  Allowing them to
have an IP presents a misleading model, and will encourage applications to
be passing around networks where they should be passing around hosts.  And
yes, the discussion is pedantic, in that some people are certain to learn
about the model of IP networking by reading the documentation of this module
if it gets into the stdlib.  I personally learned all about async networking
from reading the asyncore, select, and socket modules in python 1.5.2, lo
these many years past.

The discussion seems to be centered around the inconvenience of adding a
separate IPNetworkWithHost class that can encapsulate both of these things.
Peter seems to think that this is hugely inconvenient, but classes are
cheap.  If we were talking about IPNetwork.from_string() instead of
IPNetwork(), it seems to me that it would even be acceptable for it to
return a IPNetwork subclass if the address were not canonical (i.e. without
the bits already masked off and zeroed).  Perhaps there should be such a
method, or even just a free function, parse_mask(), as that would allow for
dealing with other user-input use-cases that have been brought up in this
thread.  I don't understand why we can't just add that class and make
everybody happy.  IPNetwork could even have a .canonicalize() method which
would return itself, and the subclass implementation in IPNetworkWithHost
return the corresponding IPNetwork.  (I wish I could come up with a better
name, but I certainly agree that there are cases where a IPNetworkWithHost
is what I would want.)

In addition to the somebody-must-have-mentioned-this-already feeling that I
got, I hesitated to post this message because it doesn't actually seem that
important to me.  While I'm firmly convinced that Network.ip is a design
mistake, it's not like the rest of Python, or for that matter any software,
is completely perfect.  In fact I think this mistake is significantly less
bad than some of the others already present in Python.  If Peter remains
unconvinced, I do think that we should put it in the stdlib, move on, and
get to fixing some of the other stuff we agree needs fixing rather than
continuing to re-hash this.  Primarily because, as far as I can tell, if
hashing and equality are defined the way that everyone seems to be agreeing
they be defined (ignoring the .ip attribute) then those of us who think .ip
is a design error can use the library and safely ignore it completely.

So, I promise not to contribute further to the problem; I won't post again
in this thread against someone who is actually going to do some work here
wants to solicit a clarification of my opinion or some more ideas :).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/1bec9bb9/attachment.htm>

From stephen at xemacs.org  Tue Sep 29 20:30:56 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 30 Sep 2009 03:30:56 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
Message-ID: <873a65li1r.fsf@uwakimon.sk.tsukuba.ac.jp>

Let me first say that the module is overall quite nice; the
implementation seems concise and very efficient, and the API is
well-balanced.  I'm sorry there's been such controversy (and for my
part in it, as I managed to completely misunderstand what you
absolutely require), when AFAICT the only "line in the sand" is being
drawn around the semantics of "net1 == net2".

Peter Moody writes:

 > I don't actually see a disconnect.  it seems analogous to stating
 > lists and ints are distinct yet lists can still contain ints. networks
 > and addresses are distinct and yet networks contain addresses.

AIUI, the disconnect is that lists do not contain a distinguished
element such that if that element differs, the lists differ.  Ie, the
disconnect is the very presence of the .ip attribute in IPv?Network.
ISTM that all of the vehement opposition is related to this single
feature.

 >   re: denormalized networks:
 > 
 > I've mentioned at least once (and others have mentioned as well) that
 > it's very common, when describing the ip address/prefix of a NIC to
 > write, "192.168.1.100/24"

Please be careful to distinguish between "denormalized descriptions of
networks" (strings like your example above) and "denormalized network
objects" (ie, objects where net.ip != net.network).  I'm happy with
the former, with a slight preference for a separate method to do the
parsing, as in Antoine's proposal.  The latter makes me "unhappy", but
I'll deal with that simply by refusing to use that feature (see below).

 >   re: new comparison methods
 > 
 > this is interesting. I would be open to doing this and making __lt__,
 > __gt__, __eq__ and friends do the non-ip comparing by default. would
 > this be sufficient or does the fact that the network has the .ip
 > attribute break something pythonic?

What do you mean, "by default"?  Surely you don't mean an equality
method with some sort of flag!?

Personally I want "net1 == net2" to have the same value as
"(net1.network, net1.prefixlen) == (net2.network, net2.prefixlen)"
when net1 and net2 are IPv?Networks, period.  For me, this is the only
sticking point; if it's not true, I'll never use this module, and I'd
oppose adding it to stdlib.

I don't care if the .ip attribute *exists* (and I don't care if you
provide separate .denormalized_compare() methods) but I find the
presence of .ip in IPv?Network objects counterintuitive.  Enough so
that if I need such an object, I will create a new class (with a name
that expresses its purpose to me) to manage it in my app, if such a
class isn't provided by ipaddr.py.

That would be no great burden to me since AFAICS apps that need such
behavior are quite specialized.  In any case, it seems that many
developers (you, GvR, and MvL, at least) aren't bothered by
overloading the IPv?Network class as you do in ipaddr.py.  (In fact,
IIUC you don't even see using it for denormalized networks as
overloading.)  I see no point in getting in your way.


From peter at hda3.com  Tue Sep 29 20:28:07 2009
From: peter at hda3.com (Peter Moody)
Date: Tue, 29 Sep 2009 11:28:07 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <873a65li1r.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
	<873a65li1r.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <8517e9350909291128s2a4ce554qecaa8f8fb1b7488e@mail.gmail.com>

On Tue, Sep 29, 2009 at 11:30 AM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> Let me first say that the module is overall quite nice; the
> implementation seems concise and very efficient, and the API is
> well-balanced. ?I'm sorry there's been such controversy (and for my
> part in it, as I managed to completely misunderstand what you
> absolutely require), when AFAICT the only "line in the sand" is being
> drawn around the semantics of "net1 == net2".
>
> Peter Moody writes:
>
> ?> I don't actually see a disconnect. ?it seems analogous to stating
> ?> lists and ints are distinct yet lists can still contain ints. networks
> ?> and addresses are distinct and yet networks contain addresses.
>
> AIUI, the disconnect is that lists do not contain a distinguished
> element such that if that element differs, the lists differ. ?Ie, the
> disconnect is the very presence of the .ip attribute in IPv?Network.
> ISTM that all of the vehement opposition is related to this single
> feature.
>
> ?> ? re: denormalized networks:
> ?>
> ?> I've mentioned at least once (and others have mentioned as well) that
> ?> it's very common, when describing the ip address/prefix of a NIC to
> ?> write, "192.168.1.100/24"
>
> Please be careful to distinguish between "denormalized descriptions of
> networks" (strings like your example above) and "denormalized network
> objects" (ie, objects where net.ip != net.network). ?I'm happy with
> the former, with a slight preference for a separate method to do the
> parsing, as in Antoine's proposal. ?The latter makes me "unhappy", but
> I'll deal with that simply by refusing to use that feature (see below).
>
> ?> ? re: new comparison methods
> ?>
> ?> this is interesting. I would be open to doing this and making __lt__,
> ?> __gt__, __eq__ and friends do the non-ip comparing by default. would
> ?> this be sufficient or does the fact that the network has the .ip
> ?> attribute break something pythonic?
>
> What do you mean, "by default"? ?Surely you don't mean an equality
> method with some sort of flag!?

no, I mean setting it up so that if I want to compare two network
objects and take into account their .ip attribute, that I use a
.compare_with_extra_bits() method or something.  __eq__, __ne__,
__lt__, __le__, __gt__, __ge__ (any more?) would all only compare
.network and .prefixlen (as you have in your example).

Cheers,
/peter

> Personally I want "net1 == net2" to have the same value as
> "(net1.network, net1.prefixlen) == (net2.network, net2.prefixlen)"
> when net1 and net2 are IPv?Networks, period. ?For me, this is the only
> sticking point; if it's not true, I'll never use this module, and I'd
> oppose adding it to stdlib.
>
> I don't care if the .ip attribute *exists* (and I don't care if you
> provide separate .denormalized_compare() methods) but I find the
> presence of .ip in IPv?Network objects counterintuitive. ?Enough so
> that if I need such an object, I will create a new class (with a name
> that expresses its purpose to me) to manage it in my app, if such a
> class isn't provided by ipaddr.py.
>
> That would be no great burden to me since AFAICS apps that need such
> behavior are quite specialized. ?In any case, it seems that many
> developers (you, GvR, and MvL, at least) aren't bothered by
> overloading the IPv?Network class as you do in ipaddr.py. ?(In fact,
> IIUC you don't even see using it for denormalized networks as
> overloading.) ?I see no point in getting in your way.
>
>

From martin at v.loewis.de  Tue Sep 29 20:36:42 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 29 Sep 2009 20:36:42 +0200
Subject: [Python-Dev] please consider changing --enable-unicode default
 to ucs4
In-Reply-To: <loom.20090929T173025-721@post.gmane.org>
References: <cd6401a0909200702t29772a1ana5d40eb39b881b36@mail.gmail.com>	<loom.20090920T162358-891@post.gmane.org>	<cd6401a0909200917gd68a6acqe7c9dff3bbbc8605@mail.gmail.com>	<4AB67430.2060306@egenix.com>	<cd6401a0909271043n2bbc206ex6988ffb69d3fb772@mail.gmail.com>	<4AC07309.1010608@egenix.com>	<4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net>	<740c3aec0909290817r78d8c5e3t2823f8c01aa4642a@mail.gmail.com>
	<loom.20090929T173025-721@post.gmane.org>
Message-ID: <4AC253BA.3040204@v.loewis.de>

>> I've also encountered this trap multiple times. Obviously, the problem
>> is not rebuilding Python which is quick, but to figure out the correct
>> configure option to use (--enable-unicode=ucs4). Others have also
>> spent some time scratching their heads over the strange
>> PyUnicodeUCS4_FromUnicode error the misconfiguration results in, as
>> Zooko's links show.
> 
> Isn't this overrated?
> 
> First, if you have a Python version that has the wrong version, just print out
> its sys.maxunicode and choose the right version according to that (if
> sys.maxunicode == 65535, you need to compile an UCS-4 version, otherwise an
> UCS-2 version).

To do so, you have to know that there is such a configuration option in
the first place, and that the error message you get (missing symbols)
has anything to do with it. This is quite puzzling to users.

Once people know what the problem is, fixing it is indeed easy.

> I'm not sure why someone "clueless" (your word :-)) wants to compile his own
> Python, though.

People install a lot of software that they don't understand. In fact,
most people who ever install software don't know how it is written,
and cannot enumerate the meaning of all configuration options that the
software possesses. In Unix, there is a long tradition that "installing
software" means "building from source"; if you have a configure script,
you expect that it either works out of the box, or gives an error
message if it finds that something is wrong with the environment.

So it is quite normal that people who don't understand how the Python
interpreter works (or that it has a Unicode type) install Python.

Regards,
Martin

From brett at python.org  Tue Sep 29 08:34:24 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 28 Sep 2009 23:34:24 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC1829E.50802@v.loewis.de>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com> 
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com> 
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com> 
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com> 
	<4AC1829E.50802@v.loewis.de>
Message-ID: <bbaeab100909282334rb680a11w71615d281e1f0df3@mail.gmail.com>

On Mon, Sep 28, 2009 at 20:44, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])``
>> as an example and simply assume that 'alpha' takes a string as an
>> argument and that it's required and that 'beta' is a boolean flag. To
>> pull everything out you could do::
>>
>> ? options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])
>> ? options_dict = dict(options)
>> ? alpha = options_dict.get('-a', options_dict.get('--alpha', ''))
>> ? beta = '-b' in options_dict or '--beta' in options_dict
>>
>> ? main(alpha, beta, args)
>>
>> Obviously if one of the getopt supporters has a better way of doing
>> this then please speak up.
>
> As Yuvgoog Greenle says, the canonical getopt way is to write
>
> alpha = None
> beta = False
> options, args = getopt.getopt(sys.argv[1:],"a:b",['alpha=','beta']):
> for opt, val in options:
> ? ?if arg in ('-a','--alpha'):
> ? ? ? ?alpha = val
> ? ?elif arg in ('-b','--beta'):
> ? ? ? ?beta = True
> main(alpha, beta, args)
>
> Even though this is many more lines, I prefer it over
> optparse/argparse: this code has only a single function call,
> whereas the argparse version has three function calls to remember.
> The actual processing uses standard Python data structures which
> I don't need to look up in the documentation.
>
>> Now, Steven, can you show how best to do this in argparse?
>
> This demonstrates my point: you were able to use getopt right away
> (even though not in the traditional way), whereas you need to ask
> for help on using argparse properly.
>

Actually, I had to read the docs for getopt. And I chose to not even
try argparse when the creator of the module is cc'ed on the email and
can obviously do a better example using his own code then I could.

-Brett

From barry at python.org  Tue Sep 29 21:10:56 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 29 Sep 2009 15:10:56 -0400
Subject: [Python-Dev] bug 3890
Message-ID: <7FE4E7BF-9434-4B60-90F4-252A5B16D910@python.org>

I'd like to include the patch for bug 3890 in Python 2.6.3.  There is  
a patch by Amaury that has gotten the blessing of Bill Janssen,  
however the patch has no test.

Is there anybody who can come up with a test in the next few hours so  
we can get this into 2.6.3rc1?

If not, is the fix important enough to include without the test?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/840883be/attachment.pgp>

From rdmurray at bitdance.com  Tue Sep 29 21:24:38 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 29 Sep 2009 15:24:38 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <d9047f780909291119v11c43e77ue19e8b0d634bd44e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
	<1254240196.5662.5.camel@localhost>
	<ca471dc20909291000i5098e311o547f374f06e34091@mail.gmail.com>
	<d9047f780909291119v11c43e77ue19e8b0d634bd44e@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909291437540.18193@kimball.webabinitio.net>

On Tue, 29 Sep 2009 at 14:19, Glyph Lefkowitz wrote:
> In addition to the somebody-must-have-mentioned-this-already feeling that I
> got, I hesitated to post this message because it doesn't actually seem that
> important to me.  While I'm firmly convinced that Network.ip is a design
> mistake, it's not like the rest of Python, or for that matter any software,
> is completely perfect.  In fact I think this mistake is significantly less
> bad than some of the others already present in Python.  If Peter remains
> unconvinced, I do think that we should put it in the stdlib, move on, and
> get to fixing some of the other stuff we agree needs fixing rather than
> continuing to re-hash this.  Primarily because, as far as I can tell, if
> hashing and equality are defined the way that everyone seems to be agreeing
> they be defined (ignoring the .ip attribute) then those of us who think .ip
> is a design error can use the library and safely ignore it completely.

I think Glyph is petty much on target here.  I think I dislike the .ip
attribute more than he does, but I'm willing to let go of it now
that the equality issue has been addressed so that the class acts
like a Network object _except_ for the ip attribute.

I do want to show why I think the ip attribute is a bad idea, using
a concrete example, rather than the hand waving we've done up to this
point.  This is a stripped down example of something that could be a
real application.  I started writing this to show what the difference
between using the ipaddr classes versus the completely separate address
and network classes would look like, but ended up with something where
there would be very little difference in the final program.

Given this simple configuration file:

demo.ini
--------------------------------------------------------------
[gatewayrouter]
inside =  e1/0 192.168.1.1/24
outside = e1/1 172.16.200.1/23
dmz =     e1/2 192.168.2.1/24

[dmzhosts]
webserver =  172.16.200.2 192.168.2.30 22 80 442
mailserver = 172.16.200.3 182.168.2.31 22 25 587
--------------------------------------------------------------


Here is a (runable) program to generate the skeleton of a Cisco router
configuration file from that configuration (I've left out a bunch
of additinoal Cisco configuration I'd include if this were a real
application):

--------------------------------------------------------------
import ipaddr
import ConfigParser

config = ConfigParser.SafeConfigParser()
config.read('demo.ini')
output = open('gatewayrouter.config', 'w')


class InterfaceData(object):

     def __init__(self, interfacespec):
         self.interfacename, ipwithnetmask = interfacespec.split()
         self.network = ipaddr.IPv4Network(ipwithnetmask)
         self.ip = self.network.ip
         self.netmask = self.network.netmask


class DMZHost(object):

     def __init__(self, hostdata):
         data = hostdata.split()
         outsideip, insideip = data[0:2]
         self.ports = data[2:]
         self.outsideip = ipaddr.IPv4Address(outsideip)
         self.insideip = ipaddr.IPv4Address(insideip)


interfaces = {}
for name in ('inside', 'outside', 'dmz'):
     interfacedata = InterfaceData(config.get('gatewayrouter', name))
     interfaces[name] = interfacedata
     globals()[name] = interfacedata.network

dmzhosts = {}
for host, hostdata in config.items('dmzhosts'):
     dmzhosts[host] = DMZHost(hostdata)

for idata in interfaces.values():
     print >>output, 'interface {}'.format(idata.interfacename)
     print >>output, 'ip address {} {}'.format(idata.ip, idata.netmask)

print >>output, 'ip extended access-list outside_in'
for host in dmzhosts.values():
     for port in host.ports:
         print >>output, 'permit ip any host {} eq {}'.format(
             host.outsideip, port)
for icmp in ('echo', 'echo-reply', 'host-unreachable', 'time-exceeded'):
     print >>output, 'permit icmp any host {} {}'.format(
         interfaces['inside'].ip, icmp)
     print >>output, 'permit icmp any {} {} {}'.format(
         dmz.network, dmz.hostmask, icmp)

print >>output, 'ip extended access-list nat'
print >>output, 'permit ip {} {} any'.format(inside.network, inside.hostmask)
print >>output, 'permit ip {} {} any'.format(dmz.network, dmz.hostmask)

print >>output, ('ip nat inside source access-list nat '
     'interface {} overload').format(interfaces['outside'].interfacename)
for host in dmzhosts.values():
     print >>output, 'ip nat inside source static {} {}'.format(
         host.insideip, host.outsideip)
--------------------------------------------------------------

There are several things to note about this code.  The first is that I
wanted my own wrapper classes to deal with the data, IPv4Network/Address
by themselves weren't enough, nor would an IPWithNetmask class have been.
I also notice that I immediately pulled out the 'ip' address from the
IPv4Network object to a top level attribute in my wrapper, so for me
a ParseIPWithMask that returned separate IPAddress and an IPNetwork
classes would have been more natural.  On the other hand, I also pulled
out the 'netmask', which I would still have needed to do.

There's one place in this code where the inclusion of the 'ip' information
in the IPNetwork class could have been used.  In the line that allows ICMP
traffic to the router's outside port, I could have written 'inside.ip'
instead of interfaces['inside'].ip.  I think that would have confused me
when I came back to read the code later, though, since 'inside' is
otherwise a network object.

But the real 'ah ha' moment in writing this program came when I wrote
the lines for the 'nat' access list.  I first wrote the line like
this:

print >>output, 'permit ip {} {} any'.format(inside.ip, inside.hostmask)

because I wanted the base IP of the inside network to go into the
access-list statement.  The code would have run, but it would have
produced incorrect output.  It also felt unnatural to write '.network'
there, since the object, in my mind, _was_ a network, which is presumably
why I wrote 'inside.ip' at first, before catching myself.  But that I
can certainly live with, just as I can live with '.ip' if I have to :)

--David

From rdmurray at bitdance.com  Tue Sep 29 21:31:31 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 29 Sep 2009 15:31:31 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909291437540.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
	<1254240196.5662.5.camel@localhost>
	<ca471dc20909291000i5098e311o547f374f06e34091@mail.gmail.com>
	<d9047f780909291119v11c43e77ue19e8b0d634bd44e@mail.gmail.com>
	<Pine.LNX.4.64.0909291437540.18193@kimball.webabinitio.net>
Message-ID: <Pine.LNX.4.64.0909291529560.18193@kimball.webabinitio.net>

On Tue, 29 Sep 2009 at 15:24, R. David Murray wrote:
> There's one place in this code where the inclusion of the 'ip' information
> in the IPNetwork class could have been used.  In the line that allows ICMP
> traffic to the router's outside port, I could have written 'inside.ip'
> instead of interfaces['inside'].ip.  I think that would have confused me
> when I came back to read the code later, though, since 'inside' is
> otherwise a network object.

Er, that should have been 'outside' in both the code and the paragraph
above, of course.  Which would also have run but produced incorrect
output...make of that what you will :)

--David

From mal at egenix.com  Tue Sep 29 21:55:35 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 29 Sep 2009 21:55:35 +0200
Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on
	trunk
In-Reply-To: <15E9DA97-B2CF-4F01-8375-F2169E598CB6@mac.com>
References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org>	<8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com>	<4AC227BB.9000607@egenix.com>	<9015487A-AA0F-4448-8ADF-C2B6E7B65377@mac.com>	<4AC2330B.1000405@egenix.com>
	<15E9DA97-B2CF-4F01-8375-F2169E598CB6@mac.com>
Message-ID: <4AC26637.6090401@egenix.com>

Ronald Oussoren wrote:
> 
> On 29 Sep, 2009, at 18:17, M.-A. Lemburg wrote:
> 
>> Ronald Oussoren wrote:
>>>
>>>>> Use:
>>>>>
>>>>>   ./configure --enable-framework --enable-universalsdk=/
>>>>>
>>>>> The --with-universal-archs flag selects whichs architectures should be
>>>>> included when you build a universal binary, defaulting to 32-bit.
>>>>
>>>> The Python default on 10.6 is 64-bit, so wouldn't it be better
>>>> to default to that on 10.6 and use 32-bit as default on 10.3/4/5 ?!
>>>
>>> Defaulting to a 32-bit build has several advantages. The first is that
>>> the defaults match the binary installer on the python.org website,
>>
>> What build options does that installer use ? (the web-page doesn't say)
> 
> The installer is build using the script in Mac/BuildScript, and uses
> --enable-framework --with-universalsk.
> 
> This creates a 32-bit fat build that runs on 10.3.9 or later.

Ok, so the installed has 32-bit x86 and ppc code, just like for
the older releases.

>>
>>> and
>>> secondly there are still 3th-party libraries that don't work in 64-bit
>>> mode (mostly GUI libraries, until recently Tk and wxWidgets wrapped the
>>> Carbon libraries which are not available in 64-bit mode; AFAIK both have
>>> betas that wrap the Cocoa libraries instead).
>>>
>>> To mimick the system default you'd have to default to 32-bit on 10.4,
>>> 4-way universal on 10.5 and 3-way universal on 10.6, and that is without
>>> considering deployment targets.  All of those are available as options,
>>> I'd prefer to keep it this way for now to keep things simple.
>>
>> Hmm, so I guess the only way to support them all is by building
>> extensions
>> using 4-way universal on 10.5. No wonder they are called "fat"
>> binaries ;-)
> 
> I like the technology though, much more convenient than having parallel
> directory tries as on Linux.

True, but it also makes it harder to get rid off parts you
don't need and the often necessary separate combine step
(using lipo) can get tedious at times, e.g. when trying
to build OpenSSL or other software that uses different
configuration data or optimizations depending on the underlying
architecture.

>>>>> I'll write some documentation on the build options on OSX,  but don't
>>>>> know what's the best location to do so.
>>>>
>>>> Please put that information into Mac/README which already has
>>>> documentation for how to build universal binaries on Mac.
>>>
>>> I know that, I wrote most of that file ;-).
>>
>> Which is why I was surprised you asked :-)
> 
> I hoped to find a document on docs.python.org that explains how to
> install Python, but sadly enough there isn't. Mac/README contains the
> right information, but isn't easily found if you're searching on the web
> or even if you're looking for documentation in the source tree.

True. The download page used to have a bit of information on how
to build Python, but that got trimmed away it seems. The Python
docs only have this page:

Unix: http://docs.python.org/using/unix.html
Window: http://docs.python.org/using/windows.html#installing-python
Mac: http://docs.python.org/using/mac.html#getting-and-installing-macpython

Neither of those pages has the in-depth information of the README
files and, frankly, I wouldn't be looking for an installation guide
in a "Using Python" section either.

Perhaps something for a sprint to change...

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 29 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From hnassrat at gmail.com  Tue Sep 29 22:20:35 2009
From: hnassrat at gmail.com (Hatem Nassrat)
Date: Tue, 29 Sep 2009 17:20:35 -0300
Subject: [Python-Dev] Announcing PEP 3136
Message-ID: <20090929202035.GA5492@gmail.com>

Tue Jul 3 10:14:17 CEST 2007, Guido van Rossum wrote:
> On 6/30/07, Matt Chisholm <matt-python at theory.org> wrote:
> > I've created and submitted a new PEP proposing support for labels in
> > Python's break and continue statements.  Georg Brandl has graciously
> > added it to the PEP list as PEP 3136:
> >
> > http://www.python.org/dev/peps/pep-3136/
> 
> I think this is a good summary of various proposals that have been
> floated in the past, plus some new ones. As a PEP, it falls short
> because it doesn't pick a solution but merely offers a large menu of
> possible options. Also, there is nothing about implementation yet.
> 
> However, I'm rejecting it on the basis that code so complicated to
> require this feature is very rare. In most cases there are existing
> work-arounds that produce clean code, for example using 'return'.

I agree that this feature will only serve as a quick hack and in many
cases it would be misused and ugly code will be the result. However,
it seems that there are some shortcuts that have sneaked into python
(I am a fairly new Python programmer, only since late 2.4, so don't
shoot me). The specific one of which I speak about is:

while_stmt ::=  "while" expression ":" suite
                ["else" ":" suite]

for_stmt ::=  "for" target_list "in" expression_list ":" suite
              ["else" ":" suite]

try1_stmt ::=  "try" ":" suite
               ("except" [expression ["as" target]] ":" suite)+
               ["else" ":" suite]
               ["finally" ":" suite]

All these else's seem like shortcuts to me. I did find a use for them,
once I found out they existed, but I get butterflies whenever I do.

-- 
Hatem Nassrat

From p.f.moore at gmail.com  Tue Sep 29 22:31:58 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 29 Sep 2009 21:31:58 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
Message-ID: <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>

2009/9/28 Yuvgoog Greenle <ubershmekel at gmail.com>:
> 1. There is no chance of the script killing itself. In argparse and optparse
> exit() is called on every parsing error (btw because of this it sucks to
> debug parse_args in an interpreter).

That one does worry me. I'd rather argparse (or any library function)
didn't call sys.exit on my behalf - it should raise an exception. Is
it actually true that argparse exits? (I can imagine that it might if
--help was specified, for example. An exception may not be right here,
but I still don't like the idea of a straight exit - I've used too
many C libraries that think they know when I want to exit).

> 2. There is no chance the parser will print things I don't want it to print.

That may also be bad - for example, Windows GUI-mode programs raise an
error if they write to stdout/stderr. I could imagine using argparse
for such a program, and wanting to do something with --help other than
write to stdout and exit (a message box, for example). And yet, I'd
want access to the text argparse would otherwise write to stdout.

Paul.

From sesquile at gmail.com  Tue Sep 29 22:38:23 2009
From: sesquile at gmail.com (m h)
Date: Tue, 29 Sep 2009 14:38:23 -0600
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
Message-ID: <e36b84ee0909291338x35230cd9i634a764c8438c05b@mail.gmail.com>

On Tue, Sep 29, 2009 at 2:31 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/9/28 Yuvgoog Greenle <ubershmekel at gmail.com>:
>> 1. There is no chance of the script killing itself. In argparse and optparse
>> exit() is called on every parsing error (btw because of this it sucks to
>> debug parse_args in an interpreter).
>
> That one does worry me. I'd rather argparse (or any library function)
> didn't call sys.exit on my behalf - it should raise an exception. Is
> it actually true that argparse exits? (I can imagine that it might if
> --help was specified, for example. An exception may not be right here,
> but I still don't like the idea of a straight exit - I've used too
> many C libraries that think they know when I want to exit).
>

+1

From guido at python.org  Tue Sep 29 22:47:32 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Sep 2009 13:47:32 -0700
Subject: [Python-Dev] Announcing PEP 3136
In-Reply-To: <20090929202035.GA5492@gmail.com>
References: <20090929202035.GA5492@gmail.com>
Message-ID: <ca471dc20909291347j7ffb202dv6f7a5b8fd0f2a9bd@mail.gmail.com>

On Tue, Sep 29, 2009 at 1:20 PM, Hatem Nassrat <hnassrat at gmail.com> wrote:
> Tue Jul 3 10:14:17 CEST 2007, Guido van Rossum wrote:
>> On 6/30/07, Matt Chisholm <matt-python at theory.org> wrote:
>> > I've created and submitted a new PEP proposing support for labels in
>> > Python's break and continue statements. ?Georg Brandl has graciously
>> > added it to the PEP list as PEP 3136:
>> >
>> > http://www.python.org/dev/peps/pep-3136/
>>
>> I think this is a good summary of various proposals that have been
>> floated in the past, plus some new ones. As a PEP, it falls short
>> because it doesn't pick a solution but merely offers a large menu of
>> possible options. Also, there is nothing about implementation yet.
>>
>> However, I'm rejecting it on the basis that code so complicated to
>> require this feature is very rare. In most cases there are existing
>> work-arounds that produce clean code, for example using 'return'.
>
> I agree that this feature will only serve as a quick hack and in many
> cases it would be misused and ugly code will be the result. However,
> it seems that there are some shortcuts that have sneaked into python
> (I am a fairly new Python programmer, only since late 2.4, so don't
> shoot me). The specific one of which I speak about is:
>
> while_stmt ::= ?"while" expression ":" suite
> ? ? ? ? ? ? ? ?["else" ":" suite]
>
> for_stmt ::= ?"for" target_list "in" expression_list ":" suite
> ? ? ? ? ? ? ?["else" ":" suite]
>
> try1_stmt ::= ?"try" ":" suite
> ? ? ? ? ? ? ? ("except" [expression ["as" target]] ":" suite)+
> ? ? ? ? ? ? ? ["else" ":" suite]
> ? ? ? ? ? ? ? ["finally" ":" suite]
>
> All these else's seem like shortcuts to me. I did find a use for them,
> once I found out they existed, but I get butterflies whenever I do.

In English, butterflies are usually a good thing (they mean you'e in love).

These else clauses (assuming you're talking about that) have been in
the language pretty much forever. The combined except/finally clause
is newer, it was added because Java allows it and it was actually a
pretty common usage.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From eric at trueblade.com  Tue Sep 29 22:56:38 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 29 Sep 2009 16:56:38 -0400
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
Message-ID: <4AC27486.3020106@trueblade.com>

Paul Moore wrote:
> 2009/9/28 Yuvgoog Greenle <ubershmekel at gmail.com>:
>> 1. There is no chance of the script killing itself. In argparse and optparse
>> exit() is called on every parsing error (btw because of this it sucks to
>> debug parse_args in an interpreter).
> 
> That one does worry me. I'd rather argparse (or any library function)
> didn't call sys.exit on my behalf - it should raise an exception. Is
> it actually true that argparse exits? (I can imagine that it might if
> --help was specified, for example. An exception may not be right here,
> but I still don't like the idea of a straight exit - I've used too
> many C libraries that think they know when I want to exit).

You can override ArgumentParser.error() to raise an exception.

>> 2. There is no chance the parser will print things I don't want it to print.
> 
> That may also be bad - for example, Windows GUI-mode programs raise an
> error if they write to stdout/stderr. I could imagine using argparse
> for such a program, and wanting to do something with --help other than
> write to stdout and exit (a message box, for example). And yet, I'd
> want access to the text argparse would otherwise write to stdout.

It looks like this might not be so easy to do. I'd suggest adding a 
file-like object to the constructor, defaulting to sys.stderr; or maybe 
an ArgumentParser.print() method that can be overridden.

Eric.

From steven.bethard at gmail.com  Tue Sep 29 22:57:22 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 29 Sep 2009 13:57:22 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
Message-ID: <d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>

On Tue, Sep 29, 2009 at 1:31 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/9/28 Yuvgoog Greenle <ubershmekel at gmail.com>:
>> 1. There is no chance of the script killing itself. In argparse and optparse
>> exit() is called on every parsing error (btw because of this it sucks to
>> debug parse_args in an interpreter).
>
> That one does worry me. I'd rather argparse (or any library function)
> didn't call sys.exit on my behalf - it should raise an exception. Is
> it actually true that argparse exits? (I can imagine that it might if
> --help was specified, for example. An exception may not be right here,
> but I still don't like the idea of a straight exit - I've used too
> many C libraries that think they know when I want to exit).

This is behavior that argparse inherits from optparse, but I believe
it's still what 99.9% of users want.  If you're writing a command line
interface, you don't want a stack trace when there's an error message
(which is what you'd get if argparse just raised exceptions) you want
an exit with an error code. That's what command line applications are
supposed to do.

If you're not using argparse to write command line applications, then
I don't feel bad if you have to do a tiny bit of extra work to take
care of that use case. In this particular situation, all you have to
do is subclass ArgumentParser and override exit() to do whatever you
think it should do.

>> 2. There is no chance the parser will print things I don't want it to print.
>
> That may also be bad - for example, Windows GUI-mode programs raise an
> error if they write to stdout/stderr. I could imagine using argparse
> for such a program, and wanting to do something with --help other than
> write to stdout and exit (a message box, for example). And yet, I'd
> want access to the text argparse would otherwise write to stdout.

There is only a single method in argparse that prints things,
_print_message(). So if you want it to do something else, you can
simply override it in a subclass. I can make that method public if
this is a common use case.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From tjreedy at udel.edu  Tue Sep 29 23:05:27 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 29 Sep 2009 17:05:27 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>	<4AC03410.4010104@v.loewis.de>	<loom.20090928T084811-560@post.gmane.org>	<4AC11855.7080406@v.loewis.de>	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
Message-ID: <h9tsqn$6hn$1@ger.gmane.org>

This is my first post in this thread. I do not currently expect to 
directly use ipaddr, but I am interested in Python having good modules 
and a friendly development environment.

It strikes me that a module that meets the desiderata of broad community 
support is going to have the 'baggage' of lots of users, who will not 
want current code broken. So that criterion conflicts with the desire to 
do major redesign.

Peter Moody wrote:

>   re: assumed disconnect between the statement "Addresses and Networks
> are distinct" and the implementation.
> 
> I don't actually see a disconnect.

I can, at least somewhat. But I do not think philosophical agreement is 
either possible or needed for practical compromise. There will still 
remain the issue of how the doc is organized and worded (see below).

>   re: denormalized networks:
> 
> I've mentioned at least once (and others have mentioned as well) that
> it's very common, when describing the ip address/prefix of a NIC to
> write, "192.168.1.100/24"
> 
>   re: new comparison methods
> 
> this is interesting. I would be open to doing this and making __lt__,
> __gt__, __eq__ and friends do the non-ip comparing by default. would
> this be sufficient or does the fact that the network has the .ip
> attribute break something pythonic?

To my naive viewpoint, yes and mostly no, and if Guido say yes and no, 
that would also be good enough for me. I do not think the cognitive load 
of .ip on learning the module is any greater than having a third class, 
especially if it is somewhat isolated in the doc as suggested below.

My suggestion for the doc, based on my understanding derived from 
reading this thread (and therefore subject to correction ;-) is 
something as follows:

-----

NN. IPADDR

NN.1 Adresses ....

NN.2 Networks

Conceptually, a network is a power of 2 range of addresses whose last k 
bits range from 0...0 to 1...1. A network object is similar to a Py3 
range object. Networks are often (usually? always?) defined by an 
address plus an indication of the bits to either keep or ignore.

As a convenience for certain uses, the defining address is included with 
the network instance as its .ip attribute (see NN.3 Address + network). 
However, standard comparisons ignore this attribute, as do the methods 
discussed in this section.

...

NN.3 (or possibly NN.2.1) Address (or Host) + Network

As mentioned above, Network instances include their defining address as 
a .ip attribute so that they can alternatively be used as Host+Network. 
The following functions and methods use this attribute. ...

-------

In other words, I suggest the the doc be organized more or less as it 
would be if there were a third class. I believe this would help both 
those who do not need or want this usage, as well as those who do.

Terry Jan Reedy


From guido at python.org  Tue Sep 29 23:23:39 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Sep 2009 14:23:39 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <h9tsqn$6hn$1@ger.gmane.org>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> 
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de> 
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net> 
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com> 
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> 
	<h9tsqn$6hn$1@ger.gmane.org>
Message-ID: <ca471dc20909291423r2ddd6e0bse7ec91a733a87c31@mail.gmail.com>

On Tue, Sep 29, 2009 at 2:05 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> this is interesting. I would be open to doing this and making __lt__,
>> __gt__, __eq__ and friends do the non-ip comparing by default. would
>> this be sufficient or does the fact that the network has the .ip
>> attribute break something pythonic?
>
> To my naive viewpoint, yes and mostly no, and if Guido say yes and no, that
> would also be good enough for me.

Indeed I say yes and no.

> I do not think the cognitive load of .ip
> on learning the module is any greater than having a third class, especially
> if it is somewhat isolated in the doc as suggested below.

Here I disagree. I want users to have few choices of data types, since
they usually make the wrong choice (Ever tried to figure out whether
to use an ArrayList or a LinkedList in Java?)

So if there are two different but similar classes to represent a
network, distinguished only by whether they retain the denormalized
input address or not, users have a 50% chance of picking the wrong
one, while if there is only one class, there is a 100% chance of
picking the right one. (There is still the IPAddress class, but the
differences between IPAddress and IPNetwork are much  clearer.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From v+python at g.nevcal.com  Wed Sep 30 00:04:30 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 29 Sep 2009 15:04:30 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
Message-ID: <4AC2846E.8060304@g.nevcal.com>

On approximately 9/29/2009 1:57 PM, came the following characters from 
the keyboard of Steven Bethard:
> On Tue, Sep 29, 2009 at 1:31 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>> 2009/9/28 Yuvgoog Greenle <ubershmekel at gmail.com>:
>>> 1. There is no chance of the script killing itself. In argparse and optparse
>>> exit() is called on every parsing error (btw because of this it sucks to
>>> debug parse_args in an interpreter).
>> That one does worry me. I'd rather argparse (or any library function)
>> didn't call sys.exit on my behalf - it should raise an exception. Is
>> it actually true that argparse exits? (I can imagine that it might if
>> --help was specified, for example. An exception may not be right here,
>> but I still don't like the idea of a straight exit - I've used too
>> many C libraries that think they know when I want to exit).
> 
> This is behavior that argparse inherits from optparse, but I believe
> it's still what 99.9% of users want.  If you're writing a command line
> interface, you don't want a stack trace when there's an error message
> (which is what you'd get if argparse just raised exceptions) you want
> an exit with an error code. That's what command line applications are
> supposed to do.
> 
> If you're not using argparse to write command line applications, then
> I don't feel bad if you have to do a tiny bit of extra work to take
> care of that use case. In this particular situation, all you have to
> do is subclass ArgumentParser and override exit() to do whatever you
> think it should do.
> 
>>> 2. There is no chance the parser will print things I don't want it to print.
>> That may also be bad - for example, Windows GUI-mode programs raise an
>> error if they write to stdout/stderr. I could imagine using argparse
>> for such a program, and wanting to do something with --help other than
>> write to stdout and exit (a message box, for example). And yet, I'd
>> want access to the text argparse would otherwise write to stdout.
> 
> There is only a single method in argparse that prints things,
> _print_message(). So if you want it to do something else, you can
> simply override it in a subclass. I can make that method public if
> this is a common use case.


Documenting both of these options would forestall people from thinking 
it is only useful for console applications.  An example of using 
argparse with Tk (I think that is the only GUI that ships with Python) 
would also be good.  In making the method public you might want to 
rename it to something other than print_message.  If all the messages 
are errors, including "error" in the name would be good.  If not, 
classifying the errors vs non-errors as an extra parameter might be good 
(in other words... inform the user and continue, or inform the user and 
exit).  Separating the message from the classification is not good, 
because that leads to dialog boxes having only an "OK" button, and 
depending on the message, it can be really inappropriate to display an 
"OK" button... buttons named "Sorry", "Alas!", and "Exit" or "Quit" are 
often more appropriate, even when there is no user choice possible.

Clearly if someone is writing a GUI, they are willing to write lots of 
lines of code to do things... a couple more well-documented lines to 
integrate argparse into their chosen GUI doesn't seem onerous. 
Especially if they can cut, paste, and hack from the above-suggested 
example code, like they do for the rest of their GUI code.

Well, at least, cut, paste, and hack is how I write GUIs when I bother.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From greg.ewing at canterbury.ac.nz  Wed Sep 30 00:40:51 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 30 Sep 2009 10:40:51 +1200
Subject: [Python-Dev] [OT] Denormalized fractions [was Re: PEP 3144
	review.]
In-Reply-To: <200909292120.03677.steve@pearwood.info>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909282115200.18193@kimball.webabinitio.net>
	<ca471dc20909282018u53a9c255o65f3c5ba9d2c156d@mail.gmail.com>
	<200909292120.03677.steve@pearwood.info>
Message-ID: <4AC28CF3.6040900@canterbury.ac.nz>

Steven D'Aprano wrote:
> there's a mathematical operator called the mediant:
> 
> mediant(a/b, c/d) = (a+c)/(b+d)

That's a function of four arguments, not two!

-- 
Greg


From greg.ewing at canterbury.ac.nz  Wed Sep 30 00:51:11 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 30 Sep 2009 10:51:11 +1200
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <19137.62484.513693.107525@montanaro.dyndns.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<4AC1829E.50802@v.loewis.de>
	<19137.62484.513693.107525@montanaro.dyndns.org>
Message-ID: <4AC28F5F.9020003@canterbury.ac.nz>

skip at pobox.com wrote:
> I have never completely wrapped my brain around optparse.  Getopt I
> just remember.

Seems to me that optparse and argparse are fairly similar
in their APIs, and that argparse isn't going to be significantly
easier to fit in one's brain than optparse.

There's an art to coming up with an API that makes simple
things easy and other things possible. I'm not convinced that
argparse represents a subsantial enough advancement in
that art to justify replacing optparse with it in the stdlib,
and thereby forcing everyone to learn a similar-but-different
API.

-- 
Greg

From p.f.moore at gmail.com  Wed Sep 30 00:51:23 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 29 Sep 2009 23:51:23 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
Message-ID: <79990c6b0909291551j4f526f32i3dc42264f67c5c8e@mail.gmail.com>

2009/9/29 Steven Bethard <steven.bethard at gmail.com>:
> If you're not using argparse to write command line applications, then
> I don't feel bad if you have to do a tiny bit of extra work to take
> care of that use case. In this particular situation, all you have to
> do is subclass ArgumentParser and override exit() to do whatever you
> think it should do.
[...]
> There is only a single method in argparse that prints things,
> _print_message(). So if you want it to do something else, you can
> simply override it in a subclass. I can make that method public if
> this is a common use case.

Thanks, that's fine for me (as things stand, no need to publicly
expose and document _print_message).

BTW, the helpful and responsive way you reply to queries is much appreciated.

I'm +1 on the PEP (although I see why people want getopt to stay, so
I'm happy to leave that and only deprecate optparse).

Paul.

From janzert at janzert.com  Wed Sep 30 01:05:31 2009
From: janzert at janzert.com (Janzert)
Date: Tue, 29 Sep 2009 19:05:31 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T185114-628@post.gmane.org>	<8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com>	<loom.20090927T201052-343@post.gmane.org>	<8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com>	<9676ED37-B679-4948-B658-257314BCD46B@fuhm.net>	<8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com>	<loom.20090927T220341-422@post.gmane.org>	<8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com>	<loom.20090927T224041-529@post.gmane.org>
	<8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com>
Message-ID: <h9u3s4$q6t$1@ger.gmane.org>

Peter Moody wrote:
> On Sun, Sep 27, 2009 at 1:49 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Peter Moody <peter <at> hda3.com> writes:
>>>>>> def parse_net_and_addr(s):
>>>>>>  return (IPNetwork(s), IPAddress(s.split('/')[0]))
>>>>> I've only heard talk of new classes and new methods, not new
>>>>> constructor functions.
>>>> Well, "method" in that context meant "class method" since the results aren't
>>>> dependent on a particular instance. Of course, both a class method or a
>>>> module-level function would be fine.
>>> so this is not the response I got when I asked what was required
>>> before. Would adding this constructor function satisfy your concerns
>>> (given sensible strict settings in the constructor, etc)?
>> Assuming the Network type loses the notion of a specific host (or host address,
>> or `ip`) attached to it, yes.
> 
> this is "less useful (strictly removing functionality)" and is an
> example of what I explicitly said I was not going to do with ipaddr.
> 
> Cheers,
> /peter
> 

Not that I should have any say, but I'm also -1 on any proposal that
conflates an "address with mask" and "network" into the same object.

Janzert


From greg.ewing at canterbury.ac.nz  Wed Sep 30 01:20:18 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 30 Sep 2009 11:20:18 +1200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
Message-ID: <4AC29632.9050903@canterbury.ac.nz>

Peter Moody wrote:

> I've mentioned at least once (and others have mentioned as well) that
> it's very common, when describing the ip address/prefix of a NIC to
> write, "192.168.1.100/24"

Yes, but if you want to *retain* all of that information,
the object you have isn't just a "network", it's more
like "network plus the address of its interface". So
some people think that using the name "IPNetwork" for
this object is confusing.

-- 
Greg

From steven.bethard at gmail.com  Wed Sep 30 01:38:53 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 29 Sep 2009 16:38:53 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC2846E.8060304@g.nevcal.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
	<4AC2846E.8060304@g.nevcal.com>
Message-ID: <d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>

On Tue, Sep 29, 2009 at 3:04 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On approximately 9/29/2009 1:57 PM, came the following characters from the
> keyboard of Steven Bethard:
>> If you're not using argparse to write command line applications, then
>> I don't feel bad if you have to do a tiny bit of extra work to take
>> care of that use case. In this particular situation, all you have to
>> do is subclass ArgumentParser and override exit() to do whatever you
>> think it should do.
[snip]
>> There is only a single method in argparse that prints things,
>> _print_message(). So if you want it to do something else, you can
>> simply override it in a subclass. I can make that method public if
>> this is a common use case.
>
> Documenting both of these options would forestall people from thinking it is
> only useful for console applications.

I'm totally fine with people thinking it is only useful for console
applications. That's what it's intended for. That said, if there are
people out there who want to use it for other applications, I'm happy
to make things easier for them if I know concretely what they want.

> An example of using argparse with Tk
> (I think that is the only GUI that ships with Python) would also be good.

I don't really use GUI libraries, so I wouldn't be able to come up
with such an example. I'd also rather not make API changes based on
speculative use cases, so before I spend time documenting these
things, I'd really like to hear from someone who has already, say,
used getopt or optparse in conjunction with a GUI library, and what
feedback they have about that.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From barry at python.org  Wed Sep 30 01:39:16 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 29 Sep 2009 19:39:16 -0400
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC28F5F.9020003@canterbury.ac.nz>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<4AC1829E.50802@v.loewis.de>
	<19137.62484.513693.107525@montanaro.dyndns.org>
	<4AC28F5F.9020003@canterbury.ac.nz>
Message-ID: <C260195A-06AC-4B1C-8D67-68DCCBAB7122@python.org>

On Sep 29, 2009, at 6:51 PM, Greg Ewing wrote:

> skip at pobox.com wrote:
>> I have never completely wrapped my brain around optparse.  Getopt I
>> just remember.
>
> Seems to me that optparse and argparse are fairly similar
> in their APIs, and that argparse isn't going to be significantly
> easier to fit in one's brain than optparse.

There's no question it is if you're doing more complicated stuff, like  
extending it or using subcommands.  After I converted my code from  
optparse to argparse, I ended up with less stuff that was more regular  
and easier to understand.  It convinced me that argparse is a win.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/8c05d48c/attachment.pgp>

From v+python at g.nevcal.com  Wed Sep 30 01:59:22 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 29 Sep 2009 16:59:22 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	
	<200909290127.03215.steve@pearwood.info>	
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>	
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>	
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>	
	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
Message-ID: <4AC29F5A.1040809@g.nevcal.com>

On approximately 9/29/2009 4:38 PM, came the following characters from 
the keyboard of Steven Bethard:
> On Tue, Sep 29, 2009 at 3:04 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
>> On approximately 9/29/2009 1:57 PM, came the following characters from the
>> keyboard of Steven Bethard:
>>> If you're not using argparse to write command line applications, then
>>> I don't feel bad if you have to do a tiny bit of extra work to take
>>> care of that use case. In this particular situation, all you have to
>>> do is subclass ArgumentParser and override exit() to do whatever you
>>> think it should do.
> [snip]
>>> There is only a single method in argparse that prints things,
>>> _print_message(). So if you want it to do something else, you can
>>> simply override it in a subclass. I can make that method public if
>>> this is a common use case.
>> Documenting both of these options would forestall people from thinking it is
>> only useful for console applications.
> 
> I'm totally fine with people thinking it is only useful for console
> applications. That's what it's intended for. That said, if there are
> people out there who want to use it for other applications, I'm happy
> to make things easier for them if I know concretely what they want.


Hmm.  Maybe that is partly why (sadly) so many GUI programs don't offer 
much in  the way of command line options.  Of course, many of their 
users in Windowsland wouldn't know how to tweak the shortcut to invoke 
them with options anyway, which might be another part.  Fortunately, 
there are some good Windows programs that do offer rich GUIs and also 
rich command line options (e.g. IrfanView and ImgBurn)


>> An example of using argparse with Tk
>> (I think that is the only GUI that ships with Python) would also be good.
> 
> I don't really use GUI libraries, so I wouldn't be able to come up
> with such an example. I'd also rather not make API changes based on
> speculative use cases, so before I spend time documenting these
> things, I'd really like to hear from someone who has already, say,
> used getopt or optparse in conjunction with a GUI library, and what
> feedback they have about that.


I'm not a Tk user, just coming to Python from Perl, where I used 
Win32::GUI, but when I was looking for a functional and portable GUI 
development package, Perl didn't have one (still doesn't have one that 
supports Unicode and printing), and Python does... so here I come to 
Python and Qt.  From my experience in Perl GUI Windowsland, the primary 
issue with command line parsing is displaying the message in a dialog 
instead of STDOUT.  But the message and the user choices have to be 
known at the same time to design the dialog box.  And, there is nothing 
so aggravating as to be shown an error message, and the only option is a 
button that says "OK", when it clearly isn't OK.  So the tone/type of 
the messages also needs to be known, even when there are no user choices.

The --help option could display the help message, and offer OK.

Many errors (particularly unrecoverable ones) should display the error 
message, and offer an Exit button, or just the close box.

A few (although probably only highly customized user options) might want 
to give the user multiple recovery options.

So, I guess I'm in the unfortunate position of seeing the need, but not 
having used these specific technologies enough to offer an example 
either.  So far, I've only used optparse (just now hearing about 
argparse in this thread) for command line programs in Python, and I am 
still just experimenting with Qt, and don't have enough familiarity with 
it yet to think that what I'm doing is best practices.

I think it would be sad to a new replacement for optparse that didn't 
GUI usage, though, at least in a limited form.  The concepts I describe 
seem sufficient from experience in other environments, and I would think 
they would be sufficient in Python too, but I'm too new here to state 
that definitely.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From ncoghlan at gmail.com  Wed Sep 30 02:05:17 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Sep 2009 10:05:17 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <19136.52362.562678.628471@montanaro.dyndns.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<bbaeab100909271457o7db763ecn9d1c22221e1207b4@mail.gmail.com>
	<4ABFE075.2070309@voidspace.org.uk>
	<F5B27C82-50C3-4838-B507-53DF82C35D4C@gmail.com>
	<4AC0AD5C.1000008@gmail.com>
	<19136.52362.562678.628471@montanaro.dyndns.org>
Message-ID: <4AC2A0BD.1070006@gmail.com>

skip at pobox.com wrote:
>     Nick> +1 here as well (although we should definitely find a way to use
>     Nick> str.format strings instead of %-format ones... come to think of
>     Nick> it, does even the logging module support str.format style
>     Nick> formatting in Py3k?)
> 
> Assuming argparse currently works with versions of Python < 2.6 I see no
> reason to make such a change.  This would just introduce needless
> differences between the version delivered with Python and the PyPI version
> and make it more difficult for the author to keep the two code bases in
> sync.

Sorry, my phrasing was poor - I should have said "as well as" rather
than "instead of". For both existing argparse users and to ease
conversion from optparse to argparse, %-formatting support obviously
needs to remain.

We already have a problem with existing APIs not supporting the new
string formatting - let's not make it worse by adding *new* APIs that
only support the *old* formatting technique.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From steven.bethard at gmail.com  Wed Sep 30 02:27:31 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 29 Sep 2009 17:27:31 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
Message-ID: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>

There's a lot of code already out there (in the standard library and
other places) that uses %-style formatting, when in Python 3.0 we
should be encouraging {}-style formatting. We should really provide
some sort of transition plan. Consider an example from the logging
docs:

logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

We'd like to support both this style as well as the following style:

logging.Formatter("{asctime} - {name} - {levelname} - {message}")

Perhaps we'd eventually deprecate the %-style formatting, but at least
in the intervening time, we'd have to support both. I see a few
possibilities:

* Add a new class, NewFormatter, which uses the {}-style formatting.
This is ugly because it makes the formatting we're trying to encourage
look like the alternative implementation instead of the standard one.
It also means we have to come up with new names for every API that
uses format strings.

* Have Formatter try to guess whether it got %-style formatting or
{}-style formatting, and then delegate to the appropriate one. I don't
know how accurately we can guess. We also end up still relying on both
formatting styles under the hood.

* Have Formatter convert all %-style formatting strings to {}-style
formatting strings (automatically). This still involves some guessing,
and involves some serious hacking to translate from one to the other
(maybe it wouldn't even always be possible?). But at least we'd only
be using {}-style formatting under the hood.

Thoughts?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From ncoghlan at gmail.com  Wed Sep 30 02:25:48 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Sep 2009 10:25:48 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC28F5F.9020003@canterbury.ac.nz>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<4AC1829E.50802@v.loewis.de>	<19137.62484.513693.107525@montanaro.dyndns.org>
	<4AC28F5F.9020003@canterbury.ac.nz>
Message-ID: <4AC2A58C.9080609@gmail.com>

Greg Ewing wrote:
> skip at pobox.com wrote:
>> I have never completely wrapped my brain around optparse.  Getopt I
>> just remember.
> 
> Seems to me that optparse and argparse are fairly similar
> in their APIs, and that argparse isn't going to be significantly
> easier to fit in one's brain than optparse.
> 
> There's an art to coming up with an API that makes simple
> things easy and other things possible. I'm not convinced that
> argparse represents a subsantial enough advancement in
> that art to justify replacing optparse with it in the stdlib,
> and thereby forcing everyone to learn a similar-but-different
> API.

As someone that has written multiple optparse based utility scripts, I
would say that yes, argparse is a *huge* improvement. Several things
that I implemented for my own use in a rather clumsy fashion
(subcommands, aggregated parsers, non-interspersed arguments) have far
more elegant support built directly into argparse.

For the getopt-vs-opt/argparse discussion, I believe the major
distinction is in the relative balance between procedural and
declarative style of argument handling.

getopt is very procedural - you define a minimal amount regarding the
options you accept, but then do the bulk of the command line processing
yourself

optparse is declarative to some degree, but forces you to drop back to a
procedural style to handle arguments and subcommands.

argparse takes things even further in a declarative direction by adding
explicit support for positional arguments and subcommands.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From greg.ewing at canterbury.ac.nz  Wed Sep 30 02:38:26 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 30 Sep 2009 12:38:26 +1200
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
Message-ID: <4AC2A882.7070108@canterbury.ac.nz>

Paul Moore wrote:
> I'd rather argparse (or any library function)
> didn't call sys.exit on my behalf - it should raise an exception.

Actually, sys.exit() *does* raise an exception (i.e.
SystemExit) that you can catch if you want.

-- 
Greg

From ncoghlan at gmail.com  Wed Sep 30 02:44:07 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Sep 2009 10:44:07 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC12AC6.80304@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<loom.20090927T131544-612@post.gmane.org>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>	<4AC03410.4010104@v.loewis.de>	<loom.20090928T084811-560@post.gmane.org>	<4AC11855.7080406@v.loewis.de>	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de>
Message-ID: <4AC2A9D7.4080603@gmail.com>

Martin v. L?wis wrote:
>> I would say that there certainly are precedents in other areas for
>> keeping the information about the input form around. For example,
>> occasionally it would be handy if parsing a hex integer returned an
>> object that was compatible with other integers but somehow kept a hint
>> that would cause printing it to use hex by default.
> 
> At the risk of bringing in false analogies: it seems that Python
> typically represents values of some type in their canonical form,
> rather than remembering the form in which they arrived in the program:
> - integer values "forget" how many preceding zeroes they have
> - string literals forget which of the characters had been escaped, and
>   whether the string was single- or double-quoted
> - floating point values forget a lot more about their literal
>   representation (including even the literal decimal value)
> 
> I guess a close case would be rational numbers: clearly, 3?2 == 6?4;
> would a Python library still remember (and repr) the original numerator
> and denominator?

For a concrete example of an object which remembers details about its
creation that it ignores when determining equality, we have decimal.Decimal:

.>> from decimal import Decimal as d
.>> x = d("3.0")
.>> y = d("3.00")
.>> x
d("3.0")
.>> y
d("3.00")
.>> repr(x) == repr(y)
False
.>> x.as_tuple() == y.as_tuple()
False
.>> x == y
True

(It was actually thinking of this example which led to me suggesting
that the equivalence classes of IPNetwork just needed adjusting rather
than the .ip attribute being removed completely)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Wed Sep 30 02:56:22 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 29 Sep 2009 20:56:22 -0400
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC2A58C.9080609@gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<4AC1829E.50802@v.loewis.de>	<19137.62484.513693.107525@montanaro.dyndns.org>
	<4AC28F5F.9020003@canterbury.ac.nz> <4AC2A58C.9080609@gmail.com>
Message-ID: <ACE2EC76-4B63-44BF-AFDA-1E9448F37B54@python.org>

On Sep 29, 2009, at 8:25 PM, Nick Coghlan wrote:

> getopt is very procedural - you define a minimal amount regarding the
> options you accept, but then do the bulk of the command line  
> processing
> yourself

Right, and we shouldn't underestimate the cognitive load this  
imposes.  All those twisty if-conditions are de-facto part of getopt's  
API.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/cc02ff8b/attachment-0001.pgp>

From ncoghlan at gmail.com  Wed Sep 30 03:07:28 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Sep 2009 11:07:28 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909291423r2ddd6e0bse7ec91a733a87c31@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>	<loom.20090928T084811-560@post.gmane.org>	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
	<h9tsqn$6hn$1@ger.gmane.org>
	<ca471dc20909291423r2ddd6e0bse7ec91a733a87c31@mail.gmail.com>
Message-ID: <4AC2AF50.6010806@gmail.com>

Guido van Rossum wrote:
> On Tue, Sep 29, 2009 at 2:05 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> I do not think the cognitive load of .ip
>> on learning the module is any greater than having a third class, especially
>> if it is somewhat isolated in the doc as suggested below.
> 
> Here I disagree. I want users to have few choices of data types, since
> they usually make the wrong choice (Ever tried to figure out whether
> to use an ArrayList or a LinkedList in Java?)
> 
> So if there are two different but similar classes to represent a
> network, distinguished only by whether they retain the denormalized
> input address or not, users have a 50% chance of picking the wrong
> one, while if there is only one class, there is a 100% chance of
> picking the right one. (There is still the IPAddress class, but the
> differences between IPAddress and IPNetwork are much  clearer.)

Note that Terry was just talking about the documentation there. That is,
he was suggesting that the documentation first describe the "normal"
IPNetwork methods which are unaffected by the .ip attribute and then, in
a separate section, describe the methods that are specific to
"denormalised" networks where "net.ip != net.network". Such an
arrangement makes sense to me.

At the risk of bikeshedding a bit, I'm still somewhat uncomfortable with
the "net.network" and "net.ip" attribute names. RDMs example application
elicited the reason for that discomfort pretty well: the current naming
seems like an invitation to write code using 'net.ip' that should have
used 'net.network' instead. Such code will then work correctly most of
the time (i.e. when only given normalised IPNetwork objects) but will
fail when given a denormalised one.

I believe that discomfort could be eliminated best by changing the name
of the ".ip" attribute to ".host_ip" to make it clear that it is
referring to the IP address of the host that was used to derive the
network definition rather than referring to the network ID itself.
Shortening ".network" to ".net_ip" would also help (this latter change
would also eliminate the mental disconnect caused by an attribute called
.network returning an IPAddress instance).

Those two naming suggestions are far less fundamental than fixing the
definition of the IPNetwork equivalent classes though - and it sounds
like Peter is happy with the proposed change on that front.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From steven.bethard at gmail.com  Wed Sep 30 03:16:33 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 29 Sep 2009 18:16:33 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <1afaf6160909291754r51197dd3jdcd05782bef11ebd@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<1afaf6160909291754r51197dd3jdcd05782bef11ebd@mail.gmail.com>
Message-ID: <d11dcfba0909291816o13af2df7n61c52609c67ecaee@mail.gmail.com>

On Tue, Sep 29, 2009 at 5:54 PM, Benjamin Peterson <benjamin at python.org> wrote:
> * Provide a flag to Formatter which controls whether new or old
> formatting is used. Emit a warning when it's not true.

So then the transition strategy is something like:

version N: Add formatting flag which uses {}-style formatting on True
and %-style formatting on False, which defaults to False
version N + 1: Deprecate False value for formatting flag (all code
should start specifying flag=True)
version N + 2: Raise error on False value for formatting flag (all
code must specify flag=True)
version N + 3: Deprecate formatting flag
version N + 4: Remove formatting flag

?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From ncoghlan at gmail.com  Wed Sep 30 03:18:59 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Sep 2009 11:18:59 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <ACE2EC76-4B63-44BF-AFDA-1E9448F37B54@python.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<4AC1829E.50802@v.loewis.de>	<19137.62484.513693.107525@montanaro.dyndns.org>
	<4AC28F5F.9020003@canterbury.ac.nz> <4AC2A58C.9080609@gmail.com>
	<ACE2EC76-4B63-44BF-AFDA-1E9448F37B54@python.org>
Message-ID: <4AC2B203.9010002@gmail.com>

Barry Warsaw wrote:
> On Sep 29, 2009, at 8:25 PM, Nick Coghlan wrote:
> 
>> getopt is very procedural - you define a minimal amount regarding the
>> options you accept, but then do the bulk of the command line processing
>> yourself
> 
> Right, and we shouldn't underestimate the cognitive load this imposes. 
> All those twisty if-conditions are de-facto part of getopt's API.

People assess the mental cost differently though - for getopt, they tend
to assign the cost to the script they're currently writing, while for
optparse/argparse they assign the cost to those two libraries being
difficult to learn :)

Keeping getopt around *and* including a "add_getopt_arguments" method in
argparse is probably the best of both worlds, in that it allows for
relatively straightforward evolution of an application:

1. Start with getopt
2. As the getopt argument parsing gets twisty and arcane and maintaining
the help string becomes a nightmare, move to argparse with the
"add_getopt_arguments" method.
3. Over time, convert more arguments to proper argparse arguments with
full documentation.

(Note that getting a good help string "for free" is one of the biggest
gains I currently find in using optparse over getopt. This is especially
so once you start making use of option groups)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Wed Sep 30 03:21:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Sep 2009 11:21:49 +1000
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC2A882.7070108@canterbury.ac.nz>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<4AC2A882.7070108@canterbury.ac.nz>
Message-ID: <4AC2B2AD.4060600@gmail.com>

Greg Ewing wrote:
> Paul Moore wrote:
>> I'd rather argparse (or any library function)
>> didn't call sys.exit on my behalf - it should raise an exception.
> 
> Actually, sys.exit() *does* raise an exception (i.e.
> SystemExit) that you can catch if you want.

I was going to mention that. Between catching SystemExit, the '-i'
switch to the interpeter and the PYTHONINSPECT environment variable,
investigating applications that call sys.exit isn't *that* difficult.
It's a far cry from the instant exits of a C level abort call.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From steven.bethard at gmail.com  Wed Sep 30 03:26:04 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 29 Sep 2009 18:26:04 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC2B203.9010002@gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<4AC1829E.50802@v.loewis.de>
	<19137.62484.513693.107525@montanaro.dyndns.org>
	<4AC28F5F.9020003@canterbury.ac.nz> <4AC2A58C.9080609@gmail.com>
	<ACE2EC76-4B63-44BF-AFDA-1E9448F37B54@python.org>
	<4AC2B203.9010002@gmail.com>
Message-ID: <d11dcfba0909291826r7ef67f67la37fa2691689b6f2@mail.gmail.com>

On Tue, Sep 29, 2009 at 6:18 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Keeping getopt around *and* including a "add_getopt_arguments" method in
> argparse is probably the best of both worlds, in that it allows for
> relatively straightforward evolution of an application:
>
> 1. Start with getopt
> 2. As the getopt argument parsing gets twisty and arcane and maintaining
> the help string becomes a nightmare, move to argparse with the
> "add_getopt_arguments" method.
> 3. Over time, convert more arguments to proper argparse arguments with
> full documentation.

One thing that wouldn't be good in this transition is that
"add_getopt_arguments" can only generate the part of the help string
that says "-a" and "--author" exist as options -- it can't add the
additional bit of text that says what they do because that's not
provided to the getopt API.

I suspect this makes the transition less convenient because with
getopt you were probably already manually maintaining a usage message
that had at least some of this information, and switching to
"add_getopt_arguments" would mean throwing this information away.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From barry at python.org  Wed Sep 30 03:30:04 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 29 Sep 2009 21:30:04 -0400
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC2B203.9010002@gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<4AC1829E.50802@v.loewis.de>	<19137.62484.513693.107525@montanaro.dyndns.org>
	<4AC28F5F.9020003@canterbury.ac.nz> <4AC2A58C.9080609@gmail.com>
	<ACE2EC76-4B63-44BF-AFDA-1E9448F37B54@python.org>
	<4AC2B203.9010002@gmail.com>
Message-ID: <FA8C9A65-A319-4164-B5F2-D2E1E6A5B018@python.org>

On Sep 29, 2009, at 9:18 PM, Nick Coghlan wrote:

> Keeping getopt around *and* including a "add_getopt_arguments"  
> method in
> argparse is probably the best of both worlds, in that it allows for
> relatively straightforward evolution of an application:
>
> 1. Start with getopt
> 2. As the getopt argument parsing gets twisty and arcane and  
> maintaining
> the help string becomes a nightmare, move to argparse with the
> "add_getopt_arguments" method.
> 3. Over time, convert more arguments to proper argparse arguments with
> full documentation.

Maybe.  I haven't been following this entire thread, but I don't see  
much point in making it easy to go from getopt to argparse.  The two  
are so different that I think you're either going to jump all in or  
not.  Maybe that's just me though as I really don't see much use for  
getopt any more (even for throwaway scripts).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/6dfb7d8d/attachment-0001.pgp>

From skip at pobox.com  Wed Sep 30 03:43:10 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 29 Sep 2009 20:43:10 -0500 (CDT)
Subject: [Python-Dev] test_thread tracebacks
Message-ID: <20090930014310.78E571250FE5@montanaro.dyndns.org>

It's been awhile since I rebuilt Python and ran the test suite.  This
evening I noticed this on my Mac (OS X 10.5):

    test_thread
    Unhandled exception in thread started by <bound method ThreadRunningTests.task of <test.test_thread.ThreadRunningTests testMethod=test_stack_size>>
    Traceback (most recent call last):
      File "/Users/skip/src/python/trunk/Lib/test/test_thread.py", line 51, in task
        self.done_mutex.release()
    thread.error: release unlocked lock
    Unhandled exception in thread started by <bound method ThreadRunningTests.task of <test.test_thread.ThreadRunningTests testMethod=test_stack_size>>
    Traceback (most recent call last):
      File "/Users/skip/src/python/trunk/Lib/test/test_thread.py", line 51, in task
        self.done_mutex.release()
    thread.error: release unlocked lock
    Unhandled exception in thread started by <bound method ThreadRunningTests.task of <test.test_thread.ThreadRunningTests testMethod=test_stack_size>>
    Traceback (most recent call last):
      File "/Users/skip/src/python/trunk/Lib/test/test_thread.py", line 51, in task
        self.done_mutex.release()
    thread.error: release unlocked lock
    Unhandled exception in thread started by <bound method ThreadRunningTests.task of <test.test_thread.ThreadRunningTests testMethod=test_starting_threads>>
    Traceback (most recent call last):
      File "/Users/skip/src/python/trunk/Lib/test/test_thread.py", line 51, in task
        self.done_mutex.release()
    thread.error: release unlocked lock

Oddly enough, this didn't cause the test to fail.

Is this a known problem?  Should I open a ticket?

Skip

From fdrake at gmail.com  Wed Sep 30 03:49:09 2009
From: fdrake at gmail.com (Fred Drake)
Date: Tue, 29 Sep 2009 21:49:09 -0400
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <FA8C9A65-A319-4164-B5F2-D2E1E6A5B018@python.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com> 
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com> 
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com> 
	<4AC1829E.50802@v.loewis.de>
	<19137.62484.513693.107525@montanaro.dyndns.org> 
	<4AC28F5F.9020003@canterbury.ac.nz> <4AC2A58C.9080609@gmail.com> 
	<ACE2EC76-4B63-44BF-AFDA-1E9448F37B54@python.org>
	<4AC2B203.9010002@gmail.com> 
	<FA8C9A65-A319-4164-B5F2-D2E1E6A5B018@python.org>
Message-ID: <9cee7ab80909291849o45b73613g1fb303bfe6e33c25@mail.gmail.com>

On Tue, Sep 29, 2009 at 9:30 PM, Barry Warsaw <barry at python.org> wrote:
> Maybe. ?I haven't been following this entire thread, but I don't see much
> point in making it easy to go from getopt to argparse.

I'm with you on this; specific getopt uses are more likely to be
switched to opt/argparse than to shift gradually via hybrid APIs.

> The two are so
> different that I think you're either going to jump all in or not. ?Maybe
> that's just me though as I really don't see much use for getopt any more
> (even for throwaway scripts).

Heh.  I never liked getopt anyway.  :-)


  -Fred

-- 
Fred L. Drake, Jr.    <fdrake at gmail.com>
"Chaos is the score upon which reality is written." --Henry Miller

From barry at python.org  Wed Sep 30 04:07:52 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 29 Sep 2009 22:07:52 -0400
Subject: [Python-Dev] test_thread tracebacks
In-Reply-To: <20090930014310.78E571250FE5@montanaro.dyndns.org>
References: <20090930014310.78E571250FE5@montanaro.dyndns.org>
Message-ID: <760DC0C1-64AB-491E-8C7F-7257249049D8@python.org>

On Sep 29, 2009, at 9:43 PM, skip at pobox.com wrote:

> It's been awhile since I rebuilt Python and ran the test suite.  This
> evening I noticed this on my Mac (OS X 10.5):

What version of Python?
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/8de96b4a/attachment.pgp>

From rdmurray at bitdance.com  Wed Sep 30 04:12:54 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 29 Sep 2009 22:12:54 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC2AF50.6010806@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
	<h9tsqn$6hn$1@ger.gmane.org>
	<ca471dc20909291423r2ddd6e0bse7ec91a733a87c31@mail.gmail.com>
	<4AC2AF50.6010806@gmail.com>
Message-ID: <Pine.LNX.4.64.0909292211300.18193@kimball.webabinitio.net>

On Wed, 30 Sep 2009 at 11:07, Nick Coghlan wrote:
> At the risk of bikeshedding a bit, I'm still somewhat uncomfortable with
> the "net.network" and "net.ip" attribute names. RDMs example application
> elicited the reason for that discomfort pretty well: the current naming
> seems like an invitation to write code using 'net.ip' that should have
> used 'net.network' instead. Such code will then work correctly most of
> the time (i.e. when only given normalised IPNetwork objects) but will
> fail when given a denormalised one.
>
> I believe that discomfort could be eliminated best by changing the name
> of the ".ip" attribute to ".host_ip" to make it clear that it is
> referring to the IP address of the host that was used to derive the
> network definition rather than referring to the network ID itself.
> Shortening ".network" to ".net_ip" would also help (this latter change
> would also eliminate the mental disconnect caused by an attribute called
> .network returning an IPAddress instance).

+1

--David (RDM)

From nad at acm.org  Wed Sep 30 04:32:42 2009
From: nad at acm.org (Ned Deily)
Date: Tue, 29 Sep 2009 19:32:42 -0700
Subject: [Python-Dev] test_thread tracebacks
References: <20090930014310.78E571250FE5@montanaro.dyndns.org>
	<760DC0C1-64AB-491E-8C7F-7257249049D8@python.org>
Message-ID: <nad-265A22.19324229092009@news.gmane.org>

In article <760DC0C1-64AB-491E-8C7F-7257249049D8 at python.org>,
 Barry Warsaw <barry at python.org> wrote:
> On Sep 29, 2009, at 9:43 PM, skip at pobox.com wrote:
> > It's been awhile since I rebuilt Python and ran the test suite.  This
> > evening I noticed this on my Mac (OS X 10.5):

It's not a new problem:

http://bugs.python.org/issue6186

-- 
 Ned Deily,
 nad at acm.org


From guido at python.org  Wed Sep 30 04:48:04 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Sep 2009 19:48:04 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909292211300.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de> 
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net> 
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com> 
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> 
	<h9tsqn$6hn$1@ger.gmane.org>
	<ca471dc20909291423r2ddd6e0bse7ec91a733a87c31@mail.gmail.com> 
	<4AC2AF50.6010806@gmail.com>
	<Pine.LNX.4.64.0909292211300.18193@kimball.webabinitio.net>
Message-ID: <ca471dc20909291948v710bfa5fg2e7a07b6f10e6f00@mail.gmail.com>

On Tue, Sep 29, 2009 at 7:12 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Wed, 30 Sep 2009 at 11:07, Nick Coghlan wrote:
>>
>> At the risk of bikeshedding a bit, I'm still somewhat uncomfortable with
>> the "net.network" and "net.ip" attribute names. RDMs example application
>> elicited the reason for that discomfort pretty well: the current naming
>> seems like an invitation to write code using 'net.ip' that should have
>> used 'net.network' instead. Such code will then work correctly most of
>> the time (i.e. when only given normalised IPNetwork objects) but will
>> fail when given a denormalised one.
>>
>> I believe that discomfort could be eliminated best by changing the name
>> of the ".ip" attribute to ".host_ip" to make it clear that it is
>> referring to the IP address of the host that was used to derive the
>> network definition rather than referring to the network ID itself.
>> Shortening ".network" to ".net_ip" would also help (this latter change
>> would also eliminate the mental disconnect caused by an attribute called
>> .network returning an IPAddress instance).
>
> +1

+1 on that it's a bikeshed.

+0 on renaming .ip to something longer.

-0 on renaming .network.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nad at acm.org  Wed Sep 30 04:50:14 2009
From: nad at acm.org (Ned Deily)
Date: Tue, 29 Sep 2009 19:50:14 -0700
Subject: [Python-Dev] test_thread tracebacks
References: <20090930014310.78E571250FE5@montanaro.dyndns.org>
	<760DC0C1-64AB-491E-8C7F-7257249049D8@python.org>
	<nad-265A22.19324229092009@news.gmane.org>
Message-ID: <nad-E13BA8.19501429092009@news.gmane.org>

In article <nad-265A22.19324229092009 at news.gmane.org>,
 Ned Deily <nad at acm.org> wrote:
> In article <760DC0C1-64AB-491E-8C7F-7257249049D8 at python.org>,
>  Barry Warsaw <barry at python.org> wrote:
> > On Sep 29, 2009, at 9:43 PM, skip at pobox.com wrote:
> > > It's been awhile since I rebuilt Python and ran the test suite.  This
> > > evening I noticed this on my Mac (OS X 10.5):
> It's not a new problem:
> 
> http://bugs.python.org/issue6186

... and minutes after I posted this, I saw another occurrence of it, 
this time with 2.6.3rc1 on 10.5 ppc, so, whatever the problem or 
severity, it is still around in 2.6, at least.

-- 
 Ned Deily,
 nad at acm.org


From stephen at xemacs.org  Wed Sep 30 05:17:49 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 30 Sep 2009 12:17:49 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC29632.9050903@canterbury.ac.nz>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<loom.20090927T131544-612@post.gmane.org>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
	<4AC29632.9050903@canterbury.ac.nz>
Message-ID: <87ws3hjf36.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:
 > Peter Moody wrote:
 > 
 > > I've mentioned at least once (and others have mentioned as well) that
 > > it's very common, when describing the ip address/prefix of a NIC to
 > > write, "192.168.1.100/24"
 > 
 > Yes, but if you want to *retain* all of that information,
 > the object you have isn't just a "network", it's more
 > like "network plus the address of its interface". So
 > some people think that using the name "IPNetwork" for
 > this object is confusing.

Especially since to my mind networks don't have interfaces.  Hosts
have interfaces.  "IPInterface", anyone? ... no, let's not go there,
stick a fork in it, it's done.

From stephen at xemacs.org  Wed Sep 30 05:24:29 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 30 Sep 2009 12:24:29 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8517e9350909291128s2a4ce554qecaa8f8fb1b7488e@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com>
	<873a65li1r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<8517e9350909291128s2a4ce554qecaa8f8fb1b7488e@mail.gmail.com>
Message-ID: <87vdj1jes2.fsf@uwakimon.sk.tsukuba.ac.jp>

Peter Moody writes:

 > > ?> this is interesting. I would be open to doing this and making __lt__,
 > > ?> __gt__, __eq__ and friends do the non-ip comparing by default. would
 > > ?> this be sufficient or does the fact that the network has the .ip
 > > ?> attribute break something pythonic?
 > >
 > > What do you mean, "by default"? ?Surely you don't mean an equality
 > > method with some sort of flag!?
 > 
 > no, I mean setting it up so that if I want to compare two network
 > objects and take into account their .ip attribute, that I use a
 > .compare_with_extra_bits() method or something.  __eq__, __ne__,
 > __lt__, __le__, __gt__, __ge__ (any more?) would all only compare
 > .network and .prefixlen (as you have in your example).

OK.  My position is that I'd appreciate it if you would review some of
the existing bikeshedding to see if there are any acceptable tweaks to
make more people a little happier, but as of now I'm +1 on the design.


From martin at v.loewis.de  Wed Sep 30 05:15:49 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 30 Sep 2009 05:15:49 +0200
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
Message-ID: <4AC2CD65.30905@v.loewis.de>

Steven Bethard wrote:
> There's a lot of code already out there (in the standard library and
> other places) that uses %-style formatting, when in Python 3.0 we
> should be encouraging {}-style formatting. 

I don't agree that we should do that. I see nothing wrong with using
% substitution.

> We should really provide some sort of transition plan.

-1.

> Consider an example from the logging
> docs:
> 
> logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
> 
> We'd like to support both this style as well as the following style:
> 
> logging.Formatter()

Now, that's different. IIUC, you are not actually performing the
substitution here, but only at a later place.

So changing to the new formatting mechanism is an API change.
I agree that the new placeholder syntax needs to be supported.

> * Add a new class, NewFormatter, which uses the {}-style formatting.
> This is ugly because it makes the formatting we're trying to encourage
> look like the alternative implementation instead of the standard one.

It's also ugly because the class has the word "new" in its name, which
no class should ever have. In a few years, it would still be around,
but not new anymore.

> * Have Formatter convert all %-style formatting strings to {}-style
> formatting strings (automatically). This still involves some guessing,
> and involves some serious hacking to translate from one to the other
> (maybe it wouldn't even always be possible?). But at least we'd only
> be using {}-style formatting under the hood.

I don't see the point of having a converter. The tricky part, as you
say, is the guessing. Whether the implementation then converts the
string or has two alternative formatting algorithms is an implementation
detail. I would favor continued use of the actual % substitution
code.

I would propose that the format argument gets an argument name,
according to the syntax it is written in. For PEP 3101 format,
I would call the argument "format" (like the method name of the
string type), i.e.

logging.Formatter(
  format="{asctime} - {name} - {levelname} - {message}")

For the % formatting, I suggest "dicttemplate" (assuming that
you *have* to use dictionary %(key)s style currently).

The positional parameter would also mean dicttemplate, and
would be deprecated (eventually requiring a keyword-only
parameter).

Regards,
Martin



From nad at acm.org  Wed Sep 30 06:29:32 2009
From: nad at acm.org (Ned Deily)
Date: Tue, 29 Sep 2009 21:29:32 -0700
Subject: [Python-Dev] Python 2.6.3
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
Message-ID: <nad-FC243A.21293229092009@news.gmane.org>

In article <9D506035-7C2D-4929-A134-E88EEB7B7D9E at python.org>,
 Barry Warsaw <barry at python.org> wrote:
> On Sep 9, 2009, at 1:29 PM, Ned Deily wrote:
> > The recent release of OS X 10.6 (Snow Leopard) has triggered a fair
> > amount of 2.6 bug tracker activity, since 10.6 now includes 2.6  
> > (2.6.1)
> > and a 64-bit version at that.   A number of patches have either just
> > been checked-in over the past couple of weeks or are getting some
> > exposure before check-in.  Given the timing and the (appropriate)
> > infrequency of 2.6.x releases, I think it would be unfortunate to push
> > 2.6.3 out the door without ensuring that it works well on 10.6.
> > Therefore, I propose that 2.6.3 should have 10.6 compatibility as a
> > "release goal".
> > Without trying to put Ronald on the spot (too much!), it would be a  
> > good
> > idea to get his assessment where things stand wrt 2.6 on 10.6 before
> > setting a final release date.
> 
> I'm hoping that Python won't have any issues building and running on  
> 10.6, but I don't have it yet so I can't personally test it out.
> 
> How would you quantify "works well"?  Do you have any thoughts on  
> tests you'd run other than the standard test suite?  If 2.6.3 is shown  
> to pass its test suite on 10.5.x, is that good enough?  Are the  
> specific bug fixes necessary for 10.6?

To follow-up, I've now run the usual "standard" OS X installer builds 
and regression tests using 2.6.3rc1 plus a few additional tests (IDLE 
and package installation) and have found no regressions from 2.6.2 and 
verified that some release-blocker problems have indeed been fixed.  In 
particular, Issue6957 regarding extensions module building on OS X 10.6 
is fixed as discussed in the issue.

The tests involved the target python.org installer configuration:

- deployment target = 10.3, 2-way i386/ppc, Tk 10.4, built on 10.5
     -> regrtest on 10.4 ppc, 10.5 ppc, 10.6 i386

plus the following additional configurations:

- deployment target = 10.3, 2-way i386/ppc, Tk 10.4, built on 10.4:
     -> regrtest on 10.4 ppc, 10.5 ppc, 10.6 i386

- deployment target = 10.5, 4-way i386/x86_64/ppc/ppc64, Tk 10.4, built 
on 10.5:
     -> regrtest on 10.5 ppc, 10.6 i386, 10.6 x86_64

plus, a quick check of the currently unsupported configuration, with no 
new regressions noted:

- deployment target = 10.6, 2-way i386/x86_64, Tk 10.5, built on 10.6:
     -> regrtest on 10.6 x86_64

As noted in various tracker issues, there are still a few unresolved 
issues with universal builds built on 10.6 (e.g. 32-bit vs 64-bit 
selection, IDLE with Tk 8.5) which is why this last installer build 
configuration is currently unsupported.

In my opinion, the standard python.org OS X installer for 2.6.3 now 
"works well" on 10.4, 10.5, and 10.6 and is ready for release.

-- 
 Ned Deily,
 nad at acm.org


From steven.bethard at gmail.com  Wed Sep 30 06:35:10 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 29 Sep 2009 21:35:10 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC2CD65.30905@v.loewis.de>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
Message-ID: <d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>

On Tue, Sep 29, 2009 at 8:15 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Steven Bethard wrote:
>> Consider an example from the logging
>> docs:
>>
>> logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
>>
>> We'd like to support both this style as well as the following style:
>>
>> logging.Formatter()
>
> Now, that's different. IIUC, you are not actually performing the
> substitution here, but only at a later place.
>
> So changing to the new formatting mechanism is an API change.
> I agree that the new placeholder syntax needs to be supported.

Just to be clear, I don't think logging is the only place these kind
of things happen. Some others I found looking around:

BaseHTTPServer.BaseHTTPRequestHandler.error_message_format
http://docs.python.org/library/basehttpserver.html#BaseHTTPServer.BaseHTTPRequestHandler.error_message_format

BaseHTTPServer.BaseHTTPRequestHandler.log_message
http://docs.python.org/3.1/library/http.server.html#http.server.BaseHTTPRequestHandler.log_message

email.generator.DecodedGenerator
http://docs.python.org/library/email.generator.html#email.generator.DecodedGenerator

There may be more.

> I would propose that the format argument gets an argument name,
> according to the syntax it is written in. For PEP 3101 format,
> I would call the argument "format" (like the method name of the
> string type), i.e.
>
> logging.Formatter(
> ?format="{asctime} - {name} - {levelname} - {message}")
>
> For the % formatting, I suggest "dicttemplate" (assuming that
> you *have* to use dictionary %(key)s style currently).
>
> The positional parameter would also mean dicttemplate, and
> would be deprecated (eventually requiring a keyword-only
> parameter).

This is a nice solution for the cases where we can be confident that
the parameter is currently only used positionally. However, at least
in Python 3.1, "fmt" is already documented as a keyword parameter:

http://docs.python.org/3.1/library/logging.html#logging.Formatter

I guess we could follow the same approach though, and have fmt= be the
%-style formatting, and use some other keyword argument for {}-style
formatting.

We've got a similar problem for the
BaseHTTPRequestHandler.error_message_format attribute. I guess we'd
want to introduce some other attribute which is the error message
format for the {}-style formatting?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From mhagger at alum.mit.edu  Wed Sep 30 06:36:42 2009
From: mhagger at alum.mit.edu (Michael Haggerty)
Date: Wed, 30 Sep 2009 06:36:42 +0200
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>
Message-ID: <4AC2E05A.70904@alum.mit.edu>

Steven Bethard wrote:
> On Tue, Sep 29, 2009 at 1:31 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>> 2009/9/28 Yuvgoog Greenle <ubershmekel at gmail.com>:
>>> 1. There is no chance of the script killing itself. In argparse and optparse
>>> exit() is called on every parsing error (btw because of this it sucks to
>>> debug parse_args in an interpreter).
> [...]
> 
> This is behavior that argparse inherits from optparse, but I believe
> it's still what 99.9% of users want.  If you're writing a command line
> interface, you don't want a stack trace when there's an error message
> (which is what you'd get if argparse just raised exceptions) you want
> an exit with an error code. That's what command line applications are
> supposed to do.
> 
> If you're not using argparse to write command line applications, then
> I don't feel bad if you have to do a tiny bit of extra work to take
> care of that use case. In this particular situation, all you have to
> do is subclass ArgumentParser and override exit() to do whatever you
> think it should do.
> 
>>> 2. There is no chance the parser will print things I don't want it to print.
> [...]
> 
> There is only a single method in argparse that prints things,
> _print_message(). So if you want it to do something else, you can
> simply override it in a subclass. I can make that method public if
> this is a common use case.

Instead of forcing the user to override the ArgumentParser class to
change how errors are handled, I suggest adding a separate method
ArgumentParser.parse_args_with_exceptions() that raises exceptions
instead of writing to stdout/stderr and never calls sys.exit().  Then
implement ArgumentParser.parse_args() as a wrapper around
parse_args_with_exceptions():

class ArgparseError(Exception):
    """argparse-specific exception type."""
    pass

class ArgumentError(ArgparseError):
    # ...

class ArgumentParser(...):
    # ...
    def parse_args_with_exceptions(...):
       # like the old parse_args(), except raises exceptions instead of
       # writing to stdout/stderr or calling sys.exit()...

    def parse_args(self, *args, **kwargs):
        try:
            self.parse_args_with_exceptions(*args, **kwargs)
        except ArgparseError as e:
            self.print_usage(_sys.stderr)
            self.exit(status=2,
                      message=(_('%s: error: %s\n') % (self.prog, e,)))
        # perhaps catch other exceptions that need special handling...

    def error(self, message):
        raise ArgparseError(message)

The exception classes should hold enough information to be useful to
non-command-line users, and obviously contain error messages that are
output to stderr by default.  This would allow non-command-line users to
call parse_args_with_exceptions() and handle the exceptions however they
like.

Michael

From foom at fuhm.net  Wed Sep 30 07:04:05 2009
From: foom at fuhm.net (James Y Knight)
Date: Wed, 30 Sep 2009 01:04:05 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
Message-ID: <A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>

I'm resending a message I sent in June, since it seems the same thread  
has come up again, and I don't believe anybody actually responded  
(positively or negatively) to the suggestion back then.

http://mail.python.org/pipermail/python-dev/2009-June/090176.html

On Jun 21, 2009, at 5:40 PM, Eric Smith wrote:
> I've basically come to accept that %-formatting can never go away,  
> unfortunately. There are too many places where %-formatting is used,  
> for example in logging Formatters. %-formatting either has to exist  
> or it has to be emulated.

It'd possibly be helpful if there were builtin objects which forced  
the format style to be either newstyle or oldstyle, independent of  
whether % or format was called on it.

E.g.
x = newstyle_formatstr("{} {} {}")
x % (1,2,3) == x.format(1,2,3) == "1 2 3"

and perhaps, for symmetry:
y = oldstyle_formatstr("%s %s %s")
y.format(1,2,3) == x % (1,2,3) == "1 2 3"

This allows the format string "style" decision is to be made external  
to the API actually calling the formatting function. Thus, it need not  
matter as much whether the logging API uses % or .format() internally  
-- that only affects the *default* behavior when a bare string is  
passed in.

This could allow for a controlled switch towards the new format string  
format, with a long deprecation period for users to migrate:

1) introduce the above feature, and recommend in docs that people only  
ever use new-style format strings, wrapping the string in  
newstyle_formatstr() when necessary for passing to an API which uses %  
internally.
2) A long time later...deprecate str.__mod__; don't deprecate  
newstyle_formatstr.__mod__.
3) A while after that (maybe), remove str.__mod__ and replace all  
calls in Python to % (used as a formatting operator) with .format() so  
that the default is to use newstyle format strings for all APIs from  
then on.


From ncoghlan at gmail.com  Wed Sep 30 07:48:32 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Sep 2009 15:48:32 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<4AC2CD65.30905@v.loewis.de>	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
Message-ID: <4AC2F130.3060202@gmail.com>

James Y Knight wrote:
> I'm resending a message I sent in June, since it seems the same thread
> has come up again, and I don't believe anybody actually responded
> (positively or negatively) to the suggestion back then.
> 
> http://mail.python.org/pipermail/python-dev/2009-June/090176.html
> 
> On Jun 21, 2009, at 5:40 PM, Eric Smith wrote:
>> I've basically come to accept that %-formatting can never go away,
>> unfortunately. There are too many places where %-formatting is used,
>> for example in logging Formatters. %-formatting either has to exist or
>> it has to be emulated.
> 
> It'd possibly be helpful if there were builtin objects which forced the
> format style to be either newstyle or oldstyle, independent of whether %
> or format was called on it.
> 
> E.g.
> x = newstyle_formatstr("{} {} {}")
> x % (1,2,3) == x.format(1,2,3) == "1 2 3"
> 
> and perhaps, for symmetry:
> y = oldstyle_formatstr("%s %s %s")
> y.format(1,2,3) == x % (1,2,3) == "1 2 3"
> 
> This allows the format string "style" decision is to be made external to
> the API actually calling the formatting function. Thus, it need not
> matter as much whether the logging API uses % or .format() internally --
> that only affects the *default* behavior when a bare string is passed in.
> 
> This could allow for a controlled switch towards the new format string
> format, with a long deprecation period for users to migrate:
> 
> 1) introduce the above feature, and recommend in docs that people only
> ever use new-style format strings, wrapping the string in
> newstyle_formatstr() when necessary for passing to an API which uses %
> internally.
> 2) A long time later...deprecate str.__mod__; don't deprecate
> newstyle_formatstr.__mod__.
> 3) A while after that (maybe), remove str.__mod__ and replace all calls
> in Python to % (used as a formatting operator) with .format() so that
> the default is to use newstyle format strings for all APIs from then on.

I must have missed this suggestion when it went past the first time. I
certainly like this approach - it has the virtue of only having to solve
the problem once, and then application developers can use it to adapt
any existing use of %-mod formatting to str.format formatting.

Something like:

class formatstr(str):
  def __mod__(self, other):
    if isinstance(other, dict):
      return self.format(**dict)
    if isinstance(other, tuple)
      return self.format(*other)
    return self.format(other)

APIs that did their own parsing based on %-formatting codes would still
break, as would any that explicitly called "str" on the object (or
otherwise stripped the subclass away, such as via "'%s' % fmt"), but
most things should pass a string subclass through transparently.

I wouldn't bother with a deprecation plan for 'normal' %-formatting
though. I don't think it is going to be practical to actually get rid of
that short of creating Python 4.0.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From steven.bethard at gmail.com  Wed Sep 30 09:35:18 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 30 Sep 2009 00:35:18 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
Message-ID: <d11dcfba0909300035s7e67264l925b3b8c5388da3e@mail.gmail.com>

On Tue, Sep 29, 2009 at 10:04 PM, James Y Knight <foom at fuhm.net> wrote:
> It'd possibly be helpful if there were builtin objects which forced the
> format style to be either newstyle or oldstyle, independent of whether % or
> format was called on it.
>
> E.g.
> x = newstyle_formatstr("{} {} {}")
> x % (1,2,3) == x.format(1,2,3) == "1 2 3"
>
> and perhaps, for symmetry:
> y = oldstyle_formatstr("%s %s %s")
> y.format(1,2,3) == x % (1,2,3) == "1 2 3"
>
> This allows the format string "style" decision is to be made external to the
> API actually calling the formatting function. Thus, it need not matter as
> much whether the logging API uses % or .format() internally -- that only
> affects the *default* behavior when a bare string is passed in.
>
> This could allow for a controlled switch towards the new format string
> format, with a long deprecation period for users to migrate:
>
> 1) introduce the above feature, and recommend in docs that people only ever
> use new-style format strings, wrapping the string in newstyle_formatstr()
> when necessary for passing to an API which uses % internally.
> 2) A long time later...deprecate str.__mod__; don't deprecate
> newstyle_formatstr.__mod__.
> 3) A while after that (maybe), remove str.__mod__ and replace all calls in
> Python to % (used as a formatting operator) with .format() so that the
> default is to use newstyle format strings for all APIs from then on.

So I understand how this might help a user to move from %-style
formatting to {}-style formatting, but it's still not clear to me how
to use this to evolve an API. In particular, if the goal is for the
API to move from accepting %-style format strings to {}-style format
strings, how should that API use newstyle_formatstr or
oldstyle_formatstr to make this transition?

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From dickinsm at gmail.com  Wed Sep 30 10:31:25 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 30 Sep 2009 09:31:25 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC2A9D7.4080603@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
Message-ID: <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>

On Wed, Sep 30, 2009 at 1:44 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Martin v. L?wis wrote:
>>> I would say that there certainly are precedents in other areas for
>>> keeping the information about the input form around. For example,
>>> occasionally it would be handy if parsing a hex integer returned an
>>> object that was compatible with other integers but somehow kept a hint
>>> that would cause printing it to use hex by default.
>>
>> At the risk of bringing in false analogies: it seems that Python
>> typically represents values of some type in their canonical form,
>> rather than remembering the form in which they arrived in the program:
>> - integer values "forget" how many preceding zeroes they have
>> - string literals forget which of the characters had been escaped, and
>> ? whether the string was single- or double-quoted
>> - floating point values forget a lot more about their literal
>> ? representation (including even the literal decimal value)
>>
>> I guess a close case would be rational numbers: clearly, 3?2 == 6?4;
>> would a Python library still remember (and repr) the original numerator
>> and denominator?
>
> For a concrete example of an object which remembers details about its
> creation that it ignores when determining equality, we have decimal.Decimal:
>
> .>> from decimal import Decimal as d
> .>> x = d("3.0")
> .>> y = d("3.00")
> .>> x
> d("3.0")
> .>> y
> d("3.00")
> .>> repr(x) == repr(y)
> False
> .>> x.as_tuple() == y.as_tuple()
> False
> .>> x == y
> True
[snipped]

[More on the Decimal analogy below.]

Please could someone who understands the uses of IPNetwork better than
I do explain why the following wouldn't be a significant problem, if __eq__
and __hash__ were modified to disregard the .ip attribute as suggested:

>>> linus = IPv4Network('172.16.200.1/24')
>>> snoopy = IPv4Network('172.16.200.3/24')
>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'}
>>> fqdn[linus]  # expecting 'linus.peanuts.net'
'snoopy.peanuts.net'

Is this just a problem of education, teaching the users not to abuse
IPv4Network this way?  Or is this just an unlikely use of IPv4Network?
Or have I misunderstood the proposal altogether?

As for Decimal, I see that as another whole kettle of tuna:  equality for
Decimal couldn't reasonably have been done any other way---if it weren't
mandated by the standard, there would still be a very strong expectation
that == would mean numeric equality.  That is, I see the == operator
as having two distinct but mostly compatible uses in Python:  it's
used for numeric equality, *and* it's used as the equivalence relation for
determining container membership.  Mostly these two different meanings
get along fine, though they lead to some fun when trying to ensure
that x == y implies hash(x) == hash(y) for x and y two different numeric
types.

But since Decimals and floats aren't used as set elements or dict keys
that often, the fact that you can't store Decimal('1.0') and Decimal('1.00')
together in a set doesn't often get in the way.  I'd expect putting
IPv4Network objects in a set or dict to be more common.

Mark

From p.f.moore at gmail.com  Wed Sep 30 11:03:15 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 30 Sep 2009 10:03:15 +0100
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <4AC2A882.7070108@canterbury.ac.nz>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>
	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>
	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>
	<4AC2A882.7070108@canterbury.ac.nz>
Message-ID: <79990c6b0909300203u20e719e5w1d492d124b96d0d2@mail.gmail.com>

2009/9/30 Greg Ewing <greg.ewing at canterbury.ac.nz>:
> Paul Moore wrote:
>>
>> I'd rather argparse (or any library function)
>> didn't call sys.exit on my behalf - it should raise an exception.
>
> Actually, sys.exit() *does* raise an exception (i.e.
> SystemExit) that you can catch if you want.

That's a good point...
Paul

From p.f.moore at gmail.com  Wed Sep 30 11:18:38 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 30 Sep 2009 10:18:38 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0909300035s7e67264l925b3b8c5388da3e@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<d11dcfba0909300035s7e67264l925b3b8c5388da3e@mail.gmail.com>
Message-ID: <79990c6b0909300218o50cf6car91216b512266b0a2@mail.gmail.com>

2009/9/30 Steven Bethard <steven.bethard at gmail.com>:
> On Tue, Sep 29, 2009 at 10:04 PM, James Y Knight <foom at fuhm.net> wrote:
>> It'd possibly be helpful if there were builtin objects which forced the
>> format style to be either newstyle or oldstyle, independent of whether % or
>> format was called on it.
> So I understand how this might help a user to move from %-style
> formatting to {}-style formatting, but it's still not clear to me how
> to use this to evolve an API. In particular, if the goal is for the
> API to move from accepting %-style format strings to {}-style format
> strings, how should that API use newstyle_formatstr or
> oldstyle_formatstr to make this transition?

IIUC, the API doesn't change. It's just that the internal code goes as follows:

1. (Current) Use %. str and oldformat objects work as now, newformat
objects work (using .format).
2. Convert the code to use .format - oldformat and newformat objects
work as before, str objects must be in new format.

The problem is, there's a point at which bare str arguments change
behaviour. So unless people wrap their arguments when calling the API,
there's still a point when things break (albeit with a simple
workaround available).

So maybe we need transition steps - wrap bare str objects in oldformat
classes, and warn, then wrap str objects in newformat objects and
warn, then stop wrapping.

That sounds to me like "normal" usage (bare strings) will be
annoyingly painful for a substantial transition period.

I'm assuming that the oldformat and newformat classes are intended to
be transitional things, and there's no intention that users should be
using the wrapper objects always. (And of course better names than
"oldformat" and "newformat" are needed - as Martin pointed out, having
"old" and "new" in the names is inappropriate). Otherwise, I'm a
strong -1 on the whole idea.

Paul.

From p.f.moore at gmail.com  Wed Sep 30 11:52:29 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 30 Sep 2009 10:52:29 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
Message-ID: <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>

2009/9/30 Mark Dickinson <dickinsm at gmail.com>:
> Please could someone who understands the uses of IPNetwork better than
> I do explain why the following wouldn't be a significant problem, if __eq__
> and __hash__ were modified to disregard the .ip attribute as suggested:
>
>>>> linus = IPv4Network('172.16.200.1/24')
>>>> snoopy = IPv4Network('172.16.200.3/24')
>>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'}
>>>> fqdn[linus] ?# expecting 'linus.peanuts.net'
> 'snoopy.peanuts.net'

I certainly don't understand IPv4Network better than you :-) But that
just looks wrong to me - linus and snoopy are hosts not networks, so
making them IPv4Network classes seems wrong. I'd instinctively make
them IPv4Address objects (which, I believe, would work).

Paul.

From dickinsm at gmail.com  Wed Sep 30 12:18:24 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 30 Sep 2009 11:18:24 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
Message-ID: <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>

On Wed, Sep 30, 2009 at 10:52 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/9/30 Mark Dickinson <dickinsm at gmail.com>:
>> Please could someone who understands the uses of IPNetwork better than
>> I do explain why the following wouldn't be a significant problem, if __eq__
>> and __hash__ were modified to disregard the .ip attribute as suggested:
>>
>>>>> linus = IPv4Network('172.16.200.1/24')
>>>>> snoopy = IPv4Network('172.16.200.3/24')
>>>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'}
>>>>> fqdn[linus] ?# expecting 'linus.peanuts.net'
>> 'snoopy.peanuts.net'
>
> I certainly don't understand IPv4Network better than you :-) But that
> just looks wrong to me - linus and snoopy are hosts not networks, so
> making them IPv4Network classes seems wrong. I'd instinctively make
> them IPv4Address objects (which, I believe, would work).

Okay, so maybe this is an abuse of IPv4Network.  But I'd (mis?)understood
that the retention of the .ip attribute was precisely a convenience to allow
this sort of use.  If not, then what's it for?  I've read the PEP and almost
all of this thread, but I can't help feeling I'm still missing something.  If
someone could point out the obvious to me I'd be grateful.

I don't have any opinion on whether the ip attribute should be retained
or not; but retaining it *and* ignoring it in comparisons just seems a
bit odd.

Mark

From eric at trueblade.com  Wed Sep 30 13:15:45 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 30 Sep 2009 07:15:45 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC2CD65.30905@v.loewis.de>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
Message-ID: <4AC33DE1.3040304@trueblade.com>

Martin v. L?wis wrote:
> Steven Bethard wrote:
>> There's a lot of code already out there (in the standard library and
>> other places) that uses %-style formatting, when in Python 3.0 we
>> should be encouraging {}-style formatting. 
> 
> I don't agree that we should do that. I see nothing wrong with using
> % substitution.

It's a maintenance burden. There are several outstanding bugs with it, 
admittedly not of any great significance. I've been putting time into 
fixing at least one of them. When Mark and I did short-float-repr, at 
least half of my time was consumed with %-formatting, mostly because of 
how it does memory management.

On the plus side, %-formatting is (and always will be) faster than 
str.format(). Its very limitations make it possible for it to be fast.

I'd note that PEP 3101 calls str.format() a replacement for 
%-formatting, not an alternate mechanism to achieve the same end.

>> * Add a new class, NewFormatter, which uses the {}-style formatting.
>> This is ugly because it makes the formatting we're trying to encourage
>> look like the alternative implementation instead of the standard one.
> 
> It's also ugly because the class has the word "new" in its name, which
> no class should ever have. In a few years, it would still be around,
> but not new anymore.
> 
>> * Have Formatter convert all %-style formatting strings to {}-style
>> formatting strings (automatically). This still involves some guessing,
>> and involves some serious hacking to translate from one to the other
>> (maybe it wouldn't even always be possible?). But at least we'd only
>> be using {}-style formatting under the hood.
> 
> I don't see the point of having a converter. The tricky part, as you
> say, is the guessing. Whether the implementation then converts the
> string or has two alternative formatting algorithms is an implementation
> detail. I would favor continued use of the actual % substitution
> code.

Having a converter and guessing are 2 distinct issues. I believe a 
convert from %-formatting specification strings to str.format() strings 
is possible. I point this out not because I think a converter solves 
this problem, but because in the past there's been a debate on whether a 
converter is even possible.

Eric.


From dickinsm at gmail.com  Wed Sep 30 13:33:03 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 30 Sep 2009 12:33:03 +0100
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC33BA1.8050203@gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
	<4AC33BA1.8050203@gmail.com>
Message-ID: <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com>

On Wed, Sep 30, 2009 at 12:06 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Mark Dickinson wrote:
>> Okay, so maybe this is an abuse of IPv4Network. ?But I'd (mis?)understood
>> that the retention of the .ip attribute was precisely a convenience to allow
>> this sort of use. ?If not, then what's it for? ?I've read the PEP and almost
>> all of this thread, but I can't help feeling I'm still missing something. ?If
>> someone could point out the obvious to me I'd be grateful.
>
> You're not missing anything that I'm aware of - unlike the use case for
> accepting a denormalised network definition in the IPNetwork constructor
> (which has been made quite clear in the list discussion, even if it is
> still a bit vague in the PEP), the use case for *retaining* the host
> information on the network object hasn't been well articulated at all.
>
> The closest anyone has come to describing a use case is an indirect
> comment via Guido that leaving out the attribute would involve real code
> having to find somewhere else to stash the original address details
> (e.g. by passing around an IPAddres/IPNetwork tuple rather than just an
> IPNetwork object).

Ah, thanks---I'd missed that bit.  So the .ip attribute is mainly for
backwards compatibility with existing uses/users of ipaddr.  I guess
that makes sense, then.  In particular, if it's suggested that new code
shouldn't make use of the .ip attribute, then the list/dict membership
problems described above can't arise.

> However, while I'd still be a little happier if the .ip attribute went
> away all together and another means was found to conveniently associate
> an IPAddress and an IPNetwork, keeping it doesn't bother me anywhere
> near as much as having network equivalence defined in terms of something
> other than the network ID and the netmask.

Makes sense.

Thanks,

Mark

From stephen at xemacs.org  Wed Sep 30 14:13:38 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 30 Sep 2009 21:13:38 +0900
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
Message-ID: <87pr98k4ul.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Dickinson writes:

 > >> Please could someone who understands the uses of IPNetwork
 > >> better than I do explain why the following wouldn't be a
 > >> significant problem, if __eq__ and __hash__ were modified to
 > >> disregard the .ip attribute as suggested:

 > >>>>> linus = IPv4Network('172.16.200.1/24')
 > >>>>> snoopy = IPv4Network('172.16.200.3/24')
 > >>>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'}
 > >>>>> fqdn[linus] ?# expecting 'linus.peanuts.net'

 > >> 'snoopy.peanuts.net'

Well, for one thing it would be broken anyway if __eq__ and __hash__
paid attention to the .ip attribute, because they *also* pay attention
to the .prefixlen (and I guess .network, redundantly) attribute.
Presumably the use in real code would not be fqdn[linus], you already
know about linus.  Rather, it would be equivalent to say

>>>>> fqdn[IPv4Network('172.16.200.1/16')]
KeyError

where an IP and network parsed out of some packet were used to
construct the key.  Of course it would work almost all the time, and
only break when, say, old-style class was used to infer the prefix
length, and was wrong.  So, no, I don't think we want to do this.



From skip at pobox.com  Wed Sep 30 14:18:11 2009
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 30 Sep 2009 07:18:11 -0500
Subject: [Python-Dev] test_thread tracebacks
In-Reply-To: <760DC0C1-64AB-491E-8C7F-7257249049D8@python.org>
References: <20090930014310.78E571250FE5@montanaro.dyndns.org>
	<760DC0C1-64AB-491E-8C7F-7257249049D8@python.org>
Message-ID: <19139.19587.615917.647283@montanaro.dyndns.org>


    >> It's been awhile since I rebuilt Python and ran the test suite.  This
    >> evening I noticed this on my Mac (OS X 10.5):

Sorry, trunk.

Skip

From barry at python.org  Wed Sep 30 14:21:18 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 30 Sep 2009 08:21:18 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC2CD65.30905@v.loewis.de>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
Message-ID: <DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>

On Sep 29, 2009, at 11:15 PM, Martin v. L?wis wrote:

> I would propose that the format argument gets an argument name,
> according to the syntax it is written in. For PEP 3101 format,
> I would call the argument "format" (like the method name of the
> string type), i.e.
>
> logging.Formatter(
>  format="{asctime} - {name} - {levelname} - {message}")
>
> For the % formatting, I suggest "dicttemplate" (assuming that
> you *have* to use dictionary %(key)s style currently).
>
> The positional parameter would also mean dicttemplate, and
> would be deprecated (eventually requiring a keyword-only
> parameter).

Although I hate the name 'dicttemplate', this seems like the best  
solution to me.  Maybe it's good that 'dicttemplate' is so ugly though  
so that people will naturally prefer 'format' :).  But I like this  
because there's not really any magic, it's explicit, and the decision  
is made by the coder at the call site.  The implementation does not  
need to guess at all.

If this is adopted, it should become a common idiom across Python so  
that once you've learned how to transition between the format strings,  
you pretty much know how to do it for any supporting API.  So we  
should adopt it across all of the standard library.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090930/115766a2/attachment-0001.pgp>

From p.f.moore at gmail.com  Wed Sep 30 14:29:11 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 30 Sep 2009 13:29:11 +0100
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
	<DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
Message-ID: <79990c6b0909300529u4383e02esa21b464b957fb015@mail.gmail.com>

2009/9/30 Barry Warsaw <barry at python.org>:
> Although I hate the name 'dicttemplate', this seems like the best solution
> to me. ?Maybe it's good that 'dicttemplate' is so ugly though so that people
> will naturally prefer 'format' :). ?But I like this because there's not
> really any magic, it's explicit, and the decision is made by the coder at
> the call site. ?The implementation does not need to guess at all.

Ignoring the "make it ugly to persuade people not to use it" aspect,
why not just use 'template'?

Paul

From barry at python.org  Wed Sep 30 14:34:02 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 30 Sep 2009 08:34:02 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <nad-FC243A.21293229092009@news.gmane.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-FC243A.21293229092009@news.gmane.org>
Message-ID: <21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org>

On Sep 30, 2009, at 12:29 AM, Ned Deily wrote:

> In my opinion, the standard python.org OS X installer for 2.6.3 now
> "works well" on 10.4, 10.5, and 10.6 and is ready for release.

Fantastic, thanks.  Martin's uploaded the Windows binaries so I'll  
make the announcement now.  No commits to the 2.6 tree are allowed  
without checking with me first (via IRC, bug tracker or email).  I'll  
make an exception for svnmerge bookkeeping.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090930/6b1f2785/attachment.pgp>

From barry at python.org  Wed Sep 30 14:37:58 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 30 Sep 2009 08:37:58 -0400
Subject: [Python-Dev] Python 2.6.3rc1 available
Message-ID: <B132EC23-B905-4BEC-B845-5E19BD4BE9A1@python.org>

The first (and hopefully last) release candidate for Python 2.6.3 is  
now available via

http://www.python.org/download/releases/2.6.3/

Source releases and Windows binaries are currently available, and Mac  
OS X binaries should be forthcoming.

Nearly 100 bugs have been fixed since 2.6.2.  Barring any unforeseen  
problems, we will make the final 2.6.3 release this Friday, October  
2nd.  Please give this release candidate a spin and let us know if you  
encounter any show stopping problems.

Enjoy,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090930/571902d7/attachment.pgp>

From vinay_sajip at yahoo.co.uk  Wed Sep 30 15:27:41 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 30 Sep 2009 13:27:41 +0000 (UTC)
Subject: [Python-Dev] Python logging and 1.5.2 compatibility
Message-ID: <loom.20090930T152706-630@post.gmane.org>

I'm planning to "officially" drop support for Python 1.5.2 in the logging
package.

When the logging package was introduced in Python 2.3, many Linux distros were
shipping 1.5.2 as the system's Python, even though 2.2 had been out for a
while. So it seemed important to support 1.5.2 for those sysadmins who wanted
to use logging with their system Python.

The Linux landscape has changed a bit since then. Most Linux distros ship with
much more recent versions of Python, and so I no longer see 1.5.2 support as
important.

Dropping support for 1.5.2 means that future changes to logging will not be
concerned with 1.5.2 compatibility. For example, boolean values which were 0/1
in the logging package will at some point be replaced by False/True, and newer
language features will start to be used when changes are made. There are no
plans for a specific "cleanup" exercise at the moment. In fact some changes
made a while ago inadvertently broke 1.5.2 compatibility, but no-one's
complained. So I'm assuming the whole thing is really a non-issue, and this
post is just to keep everyone in the picture.

A 1.5.2-compatible version of the package is still available via
http://www.red-dove.com/python_logging.html if anyone needs it. This version
is not actively maintained, but that shouldn't be an issue.

Regards,

Vinay Sajip


From solipsis at pitrou.net  Wed Sep 30 15:39:22 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 30 Sep 2009 13:39:22 +0000 (UTC)
Subject: [Python-Dev] Python logging and 1.5.2 compatibility
References: <loom.20090930T152706-630@post.gmane.org>
Message-ID: <h9vn2a$3lg$1@ger.gmane.org>

Le Wed, 30 Sep 2009 13:27:41 +0000, Vinay Sajip a ?crit?:
> I'm planning to "officially" drop support for Python 1.5.2 in the
> logging package.

Thanks for the announcement.
So, what is the minimum supported version now?

Regards

Antoine.



From rdmurray at bitdance.com  Wed Sep 30 15:41:26 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 30 Sep 2009 09:41:26 -0400 (EDT)
Subject: [Python-Dev] Python logging and 1.5.2 compatibility
In-Reply-To: <loom.20090930T152706-630@post.gmane.org>
References: <loom.20090930T152706-630@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0909300938550.18193@kimball.webabinitio.net>

On Wed, 30 Sep 2009 at 13:27, Vinay Sajip wrote:
> I'm planning to "officially" drop support for Python 1.5.2 in the logging
> package.

What's the minimum version of Python that the logging module now officially
supports?

--David (RDM)

From chris at simplistix.co.uk  Wed Sep 30 16:24:32 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 30 Sep 2009 15:24:32 +0100
Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc,
 if both object have implementations
In-Reply-To: <19129.39295.970536.916767@montanaro.dyndns.org>
References: <4AB8D9F2.6050807@simplistix.co.uk>	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>	<4AB8E136.8040706@simplistix.co.uk>	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
Message-ID: <4AC36A20.4060102@simplistix.co.uk>

skip at pobox.com wrote:
>     Dino> For IronPython we wrote a set of tests which go through and define
>     Dino> the various operator methods in all sorts of combinations on both
>     Dino> new-style and old-style classes as well as subclasses of those
>     Dino> classes and then do the comparisons w/ logging.
> 
> It would be very nice if these complex corner cases had a set of test
> cases which could be run by all implementations (CPython, Jython,
> IronPython, PyPy, etc).  I don't know.  Maybe the CPython test suite serves
> that purpose, but it seems like it would be helpful if this sort of
> "validation suite" was maintained as a separate project all implementations
> could use and contribute to.

+1, and supported by documentation.

Something like a manuel-based table test would seem ideal for this:

http://packages.python.org/manuel/manuel/table-example.html

..but what are the chances of something like that getting into the 
python core docs?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Wed Sep 30 16:28:58 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 30 Sep 2009 15:28:58 +0100
Subject: [Python-Dev] sharing stdlib across python implementations
In-Reply-To: <4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>	<4AB8E136.8040706@simplistix.co.uk>	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>	<19129.39295.970536.916767@montanaro.dyndns.org>
	<4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com>
Message-ID: <4AC36B2A.6020406@simplistix.co.uk>

Frank Wierzbicki wrote:
> Talk has started up again on the stdlib-sig list about finding a core
> stdlib + tests that can be shared by all implementations, potentially
> living apart from CPython.  I have volunteered to put together a PEP
> on the subject, with Jessie Noller and Brett Canon are helping me out.
>  When I have something worth showing, I'll start the real PEP process.

I'm on on stdlib-sig and I'm afraid I don't have the bandwidth to start 
on it, but I'd just like to throw in (yet again) that it would be great 
if the stdlib was actually a set of separate python packages with their 
own version metadata so that packaging tools could manage them, and 
upgrade them independently of python packages when there are bug fixes. 
If that were the case, then pure python packages in the stdlib, of which 
there are many, *really* could be used across python implementations 
with no changes whatsoever...

The big changes I can see from here would be moving the tests to the 
packages from the central tests directory, and adding a setup.py file or 
some other form of metadata providion for each package. Not that big now 
that I've written it ;-)

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From vinay_sajip at yahoo.co.uk  Wed Sep 30 16:32:44 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 30 Sep 2009 14:32:44 +0000 (UTC)
Subject: [Python-Dev] Python logging and 1.5.2 compatibility
References: <loom.20090930T152706-630@post.gmane.org>
	<h9vn2a$3lg$1@ger.gmane.org>
Message-ID: <loom.20090930T160811-948@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> So, what is the minimum supported version now?

Well, each version is tested with the release of Python which it's part of, and
in theory it would be possible for the code in logging for that release to use
features only available in that release of Python. So, I'm not sure if it now
makes sense to support a minimum version. When changing the package I intend to
be conservative in terms of using the simplest, rather than using the latest
language features (say, coroutines) just because they're available.

The reason it was done for 1.5.2 was that there was already a community of users
of the "standalone" package (which was 1.5.2-compatible) before it got accepted
into Python 2.3, and it didn't make sense at the time to maintain two different
codebases - one 1.5.2-compatible and one not. Keeping it compatible meant that
1.5.2-dependent users/sysadmins could just copy the whole package and have it
work, and take advantage of bug-fixes and enhancements as they appeared.

PEP 291 (Backward Compatibility for Standard Library) mentions logging as having
1.5.2 compatibility, and I'm thinking that it can be removed from that PEP
altogether. I don't think it's still necessary to hold back from using new
features just to support 1.5.2, given that a working implementation is available
if any 1.5.2 user should need it, and that the 1.5.2-dependent community will be
pretty minimal. A lot of water has passed under the bridge.

There are some new features which have been added in recent versions (say,
WatchedFileHandler in 2.6, NullHandler in 2.7) and these should still work if
copied and pasted into a codebase which runs with an older version of Python.
Certain projects which need to support multiple Python versions - Django, say -
can take advantage of this by e.g. offering a copy of these handlers in their
own codebase for use when constrained to work with an older Python version.

Regards,

Vinay Sajip


From steve at pearwood.info  Wed Sep 30 16:34:27 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 1 Oct 2009 00:34:27 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
Message-ID: <200910010034.27427.steve@pearwood.info>

On Wed, 30 Sep 2009 03:04:05 pm James Y Knight wrote:

> It'd possibly be helpful if there were builtin objects which forced
> the format style to be either newstyle or oldstyle, independent of
> whether % or format was called on it.
>
> E.g.
> x = newstyle_formatstr("{} {} {}")
> x % (1,2,3) == x.format(1,2,3) == "1 2 3"

People will want this formatstr object to behave like strings, with 
concatenation, slicing, etc.:

>>> x = newstyle_formatstr("x{} {} : ")
>>> y = newstyle_formatstr("{}")
>>> (x[1:] + y) % (1, 2, 3)
'1 2 : 3'

Instead of having to support one type with %-formatting and 
{}-formatting (str), now the std lib will have two classes 
with %-formatting and {}-formatting. How is this an improvement?

Moving along, let's suppose the newstyle_formatstr is introduced. What's 
the intention then? Do we go through the std lib and replace every call 
to (say)

    somestring % args 

with 

    newstyle_formatstr(somestring) % args 

instead? That seems terribly pointless to me -- it does nothing about 
getting rid of % but adds a layer of indirection which slows down the 
code. Things are no better if the replacement code is:

    newstyle_formatstr(somestring).format(*args)

(or similar). If we can do that, then why not just go all the way and 
use this as the replacement instead?

    somestring.format(*args)




> and perhaps, for symmetry:
> y = oldstyle_formatstr("%s %s %s")
> y.format(1,2,3) == x % (1,2,3) == "1 2 3"

Now we have three classes that support both % and {} formatting. Great.


[...]
> This could allow for a controlled switch towards the new format
> string format, with a long deprecation period for users to migrate:
>
> 1) introduce the above feature, and recommend in docs that people
> only ever use new-style format strings, wrapping the string in
> newstyle_formatstr() when necessary for passing to an API which uses
> % internally.

And how are people supposed to know what the API uses internally?

Personally, I think your chances of getting people to write:

logging.Formatter(newstyle_formatstr("%(asctime)s - %(name)s - %(level)s - %(msg)s"))

instead of 

logging.Formatter("%(asctime)s - %(name)s - %(level)s - %(msg)s")

is slim to none -- especially when the second call still works. You'd 
better off putting the call to newstyle_formatstr() inside 
logging.Formatter, and not even telling the users.

Instead of wrapping strings in a class that makes .__mod__() 
and .format() behave the same, at some cost on every call presumably, 
my preferred approach would be a converter function (perhaps taken from 
2to3?) which modified strings like "%(asctime)s" to "{asctime}". That 
cost gets paid *once*, rather than on every call.

(Obviously the details will need to be ironed out, and it will depend on 
the external API. If the external API depends on the caller using % 
explicitly, then this approach may not work.)


> 2) A long time later...deprecate str.__mod__; 

How long? I hope that's not until I'm dead and buried.



-- 
Steven D'Aprano

From solipsis at pitrou.net  Wed Sep 30 16:43:07 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 30 Sep 2009 14:43:07 +0000 (UTC)
Subject: [Python-Dev] sharing stdlib across python implementations
References: <4AB8D9F2.6050807@simplistix.co.uk>	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>	<4AB8E136.8040706@simplistix.co.uk>	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>	<19129.39295.970536.916767@montanaro.dyndns.org>
	<4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com>
	<4AC36B2A.6020406@simplistix.co.uk>
Message-ID: <loom.20090930T163756-153@post.gmane.org>

Chris Withers <chris <at> simplistix.co.uk> writes:
> 
> I'm on on stdlib-sig and I'm afraid I don't have the bandwidth to start 
> on it, but I'd just like to throw in (yet again) that it would be great 
> if the stdlib was actually a set of separate python packages with their 
> own version metadata so that packaging tools could manage them, and 
> upgrade them independently of python packages when there are bug fixes. 

This sounds like a bad idea to me. Each Python release is tested and debugged as
a whole. If you have a lot of possible combinations (module A version 1.1 with
module B version 1.2, etc.), it becomes impossible for us to ensure proper QA
for the whole and as a result the quality might become lower, rather than higher.

(of course, if we rigorously enforce APIs and preserve compatibility, this might
not be a real issue; but our compatibility story is a bit irregular, IMHO)

Regards

Antoine.



From chris at simplistix.co.uk  Wed Sep 30 16:45:56 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 30 Sep 2009 15:45:56 +0100
Subject: [Python-Dev] sharing stdlib across python implementations
In-Reply-To: <loom.20090930T163756-153@post.gmane.org>
References: <4AB8D9F2.6050807@simplistix.co.uk>	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>	<4AB8E136.8040706@simplistix.co.uk>	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>	<19129.39295.970536.916767@montanaro.dyndns.org>	<4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com>	<4AC36B2A.6020406@simplistix.co.uk>
	<loom.20090930T163756-153@post.gmane.org>
Message-ID: <4AC36F24.5020107@simplistix.co.uk>

Antoine Pitrou wrote:
> This sounds like a bad idea to me. Each Python release is tested and debugged as
> a whole. If you have a lot of possible combinations (module A version 1.1 with
> module B version 1.2, etc.), it becomes impossible for us to ensure proper QA
> for the whole and as a result the quality might become lower, rather than higher.

I'd certainly still see there being "blessed" python releases.
(The Zope community does this in the form of "known good sets" and 
these, for me, would be "the python releases")

However, it's pretty frustrating to not be able to upgrade a pure-python 
package with a critical bug (as happened recently to me with httplib) 
because it's part of the stdlib.

It's also frustrating to have to if-then-else about a package or 
project's dependencies when using dependency management tools like 
buildout when libraries such as elementtree and argparse become part of 
the stdlib when they weren't before...

> (of course, if we rigorously enforce APIs and preserve compatibility, this might
> not be a real issue; but our compatibility story is a bit irregular, IMHO)

Breaking it apart like this would *make* the compatibility story better...

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Wed Sep 30 16:57:27 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 30 Sep 2009 15:57:27 +0100
Subject: [Python-Dev] a new setuptools release?
In-Reply-To: <20090923204737.A97DC3A4079@sparrow.telecommunity.com>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>
	<4AB8D910.8050601@simplistix.co.uk>
	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>
	<4AB92ADE.9060700@simplistix.co.uk>
	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>
	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>
	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>
	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<20090923204737.A97DC3A4079@sparrow.telecommunity.com>
Message-ID: <4AC371D7.9000904@simplistix.co.uk>

P.J. Eby wrote:
> Here's what actually happened, if anyone cares.  Tarek and friends 
> announced a fork of setuptools.  I reviewed the work and saw that -- for 
> the most part -- I was happy with it, and opined as how I might be 
> willing to bless the the "package inquisition" team as official 
> maintainers of the 0.6 branch of setuptools, so that I could work on the 
> fun bits I've long planned for 0.7, but never felt free to start on 
> while there was so much still needing to be done on 0.6.

If this offer is still available, I'd lake to take you up on it.
I'd be more than willing to merge changes on the 0.6 distribute branch 
back into the setuptools codebase and do whatever is needed to get a new 
setuptools release out.

Why? Because there are a *lot* of copies of ez_setup.py and buildout's 
bootstrap.py that will need replacing if it doesn't happen. I think it'd 
be better for the python community all round if setuptools just 
continued in maintenance mode until whatever-ends-up-in-the-core exists 
and people want to use...

I'm happy to submit to whatever supervision is needed for you to trust 
me to do this, and I promise to be as careful as I can with this. I 
*know* how important this is and want to make it work...

> All I want is for good stuff to happen for setuptools users and Python 
> users in general, so I don't think all the suspicion and backbiting is 
> merited. 

Fine, if that's true, I apologize (even spelled correctly!) for any 
previous involvement in this, but please help me help you achieve your 
aims...

To put this into a way that makes sense to me: I'm volunteering to keep 
distribute 0.6 and setuptools 0.6 in sync, no more, no less, and try and 
keep that as uncontroversial as possible, and get setuptools 0.6 
releases out to match distribute 0.6 releases as soon as I can.

Again, I feel I need to stress that the *only* reason I want to do this 
is to bring the benefits of the distribute work to the existing 
setuptools codebase, with appropriate attribution if that makes a 
difference.

Apologies if any of this is offensive to anyone. For once (really!) I 
really mean that :-)

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Wed Sep 30 17:04:26 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 30 Sep 2009 16:04:26 +0100
Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format
In-Reply-To: <loom.20090923T191242-680@post.gmane.org>
References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com>	<4AB8D910.8050601@simplistix.co.uk>	<94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com>	<4AB92ADE.9060700@simplistix.co.uk>	<bbaeab100909221302v62b9f227v84de978fd838b514@mail.gmail.com>	<87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp>	<94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com>	<87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp>	<94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com>
	<loom.20090923T191242-680@post.gmane.org>
Message-ID: <4AC3737A.8020701@simplistix.co.uk>

Antoine Pitrou wrote:
> As far as I'm concerned, anything which looks intuitive enough (e.g. ini-style)
> and not overly complicated is fine. The details of the syntax aren't really
> important as long as they make sense, and don't get in the way.

This is a variant of "as simple as possible and required but no 
simpler", which I'm sure is what we're all aiming for. The problem is 
that it's fiendishly difficult ;-)

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

From olemis at gmail.com  Wed Sep 30 17:18:09 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 30 Sep 2009 10:18:09 -0500
Subject: [Python-Dev] sharing stdlib across python implementations
In-Reply-To: <4AC36B2A.6020406@simplistix.co.uk>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
	<4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com>
	<4AC36B2A.6020406@simplistix.co.uk>
Message-ID: <24ea26600909300818h4663eafbqba5971d4b4d3d752@mail.gmail.com>

On Wed, Sep 30, 2009 at 9:28 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> Frank Wierzbicki wrote:
>>
>> Talk has started up again on the stdlib-sig list about finding a core
>> stdlib + tests that can be shared by all implementations, potentially
>> living apart from CPython.
>
[...]
>
> if the
> stdlib was actually a set of separate python packages with their own version
> metadata so that packaging tools could manage them, and upgrade them
> independently of python packages when there are bug fixes. If that were the
> case, then pure python packages in the stdlib, of which there are many,
> *really* could be used across python implementations with no changes
> whatsoever...
>

nice ! That's something I really liked about Python.NET

:)

BTW ... is there something like that for Java ? I mean to use J2[SE]E
classes using CPython ?

This could also be useful to have personalized distributions. I mean,
if I want to implement a Py app that will run in devices with limited
capabilities, and let's say that it only imports `sockets` module (or
a few more ;o), then it will be easier to prepare a subset of stdlib
in order to deploy just what is needed in such devices, and save some
space ;o).

Projects like py2exe or others, could use something like that in order
to extract relevant stdlib (modules | packages) and make them
available to Windows apps distributed as exe files (e.g. Hg )

CMIIW anyway, perhaps I'm just dreaming.

however ...

> The big changes I can see from here would be moving the tests to the
> packages from the central tests directory, and adding a setup.py file or
> some other form of metadata providion for each package. Not that big now
> that I've written it ;-)
>

In this case I envision the following issues if one setup.py file is
generated for every module or top-level package (... which is
-considering the previous message- how u plan to do it, isn't it ? )

  - the maintenance effort might increase
  - what about dependencies between stdlib modules ?
  - there are many attributes which will take the same values for each
and every
    packages (e.g. version info, issue tracker, ...) and some that
will be specific
    (e,g, maintainer, author, contact info, dependencies ...)
  - Overhead when a new package is included in stdlib (need to create and
    maintain `setup.py` script, and so on ...)

So my $0.02 here are :

  - to have a single `setup.py` file (in the end it's a script, isn't it ? )
  - provide an argument in order to select module(s) to be included
    (full stdlib if missing) in source distribution. In general any other
    parameterization of the `setup.py` may be just ok, the goal is
    to have only one
  - use a mechanism in order to specify config options for specific pkgs
    modules, and make it available to the global `setup.py`. For example :
    * Specify metadata using top-level fields in modules (e.g. __author__,
       __maintainer__, ...)
    * Specify metadata using separate INI files for each target

What d'u think ?

There may be some issues with sdist anyway
:-/

PS: Will those packages be submitted to PyPI too ? I mean if not
sdists, at least meta-data ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Sobrepasa las 300 descargas el m?dulo dutest  -
http://feedproxy.google.com/~r/simelo-es/~3/U5rff5iTQPI/sobrepasa-las-300-descargas-el-modulo.html

From steven.bethard at gmail.com  Wed Sep 30 17:22:33 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 30 Sep 2009 08:22:33 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
	<DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
Message-ID: <d11dcfba0909300822x34a05e8en942571230a543561@mail.gmail.com>

On Wed, Sep 30, 2009 at 5:21 AM, Barry Warsaw <barry at python.org> wrote:
> On Sep 29, 2009, at 11:15 PM, Martin v. L?wis wrote:
>
>> I would propose that the format argument gets an argument name,
>> according to the syntax it is written in. For PEP 3101 format,
>> I would call the argument "format" (like the method name of the
>> string type), i.e.
>>
>> logging.Formatter(
>> ?format="{asctime} - {name} - {levelname} - {message}")
>>
>> For the % formatting, I suggest "dicttemplate" (assuming that
>> you *have* to use dictionary %(key)s style currently).
>>
>> The positional parameter would also mean dicttemplate, and
>> would be deprecated (eventually requiring a keyword-only
>> parameter).
>
> Although I hate the name 'dicttemplate', this seems like the best solution
> to me. ?Maybe it's good that 'dicttemplate' is so ugly though so that people
> will naturally prefer 'format' :). ?But I like this because there's not
> really any magic, it's explicit, and the decision is made by the coder at
> the call site. ?The implementation does not need to guess at all.

Could you comment on what you think we should do when the parameter is
not positional? As I mentioned upthread, in the case of
logging.Formatter, it's already documented as taking the keyword
parameter "fmt", so we'd have to use the name "fmt" for % formatting.


Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From barry at python.org  Wed Sep 30 17:31:23 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 30 Sep 2009 11:31:23 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0909300822x34a05e8en942571230a543561@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
	<DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
	<d11dcfba0909300822x34a05e8en942571230a543561@mail.gmail.com>
Message-ID: <FA4C011E-2A18-4F8F-B555-BAB8AD78A306@python.org>

On Sep 30, 2009, at 11:22 AM, Steven Bethard wrote:

> On Wed, Sep 30, 2009 at 5:21 AM, Barry Warsaw <barry at python.org>  
> wrote:
>> On Sep 29, 2009, at 11:15 PM, Martin v. L?wis wrote:
>>
>>> I would propose that the format argument gets an argument name,
>>> according to the syntax it is written in. For PEP 3101 format,
>>> I would call the argument "format" (like the method name of the
>>> string type), i.e.
>>>
>>> logging.Formatter(
>>>  format="{asctime} - {name} - {levelname} - {message}")
>>>
>>> For the % formatting, I suggest "dicttemplate" (assuming that
>>> you *have* to use dictionary %(key)s style currently).
>>>
>>> The positional parameter would also mean dicttemplate, and
>>> would be deprecated (eventually requiring a keyword-only
>>> parameter).
>>
>> Although I hate the name 'dicttemplate', this seems like the best  
>> solution
>> to me.  Maybe it's good that 'dicttemplate' is so ugly though so  
>> that people
>> will naturally prefer 'format' :).  But I like this because there's  
>> not
>> really any magic, it's explicit, and the decision is made by the  
>> coder at
>> the call site.  The implementation does not need to guess at all.
>
> Could you comment on what you think we should do when the parameter is
> not positional? As I mentioned upthread, in the case of
> logging.Formatter, it's already documented as taking the keyword
> parameter "fmt", so we'd have to use the name "fmt" for % formatting.

I'm okay with fmt==%-formatting and format=={}-formatting, but I'd  
also be okay with transitioning 'fmt' to 'dicttemplate' or whatever.   
I think the important thing is to be explicit in the method signature  
which one you want (secondary would be trying to standardize this  
across the stdlib).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090930/7a5ccc75/attachment.pgp>

From olemis at gmail.com  Wed Sep 30 17:37:02 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 30 Sep 2009 10:37:02 -0500
Subject: [Python-Dev] sharing stdlib across python implementations
In-Reply-To: <loom.20090930T163756-153@post.gmane.org>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<B2C25766-1783-4CF5-A15F-3FDE904CAE0F@gmail.com>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
	<4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com>
	<4AC36B2A.6020406@simplistix.co.uk>
	<loom.20090930T163756-153@post.gmane.org>
Message-ID: <24ea26600909300837m577947b4s7e537b048740adbd@mail.gmail.com>

On Wed, Sep 30, 2009 at 9:43 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Chris Withers <chris <at> simplistix.co.uk> writes:
>>
>> I'm on on stdlib-sig and I'm afraid I don't have the bandwidth to start
>> on it, but I'd just like to throw in (yet again) that it would be great
>> if the stdlib was actually a set of separate python packages with their
>> own version metadata so that packaging tools could manage them, and
>> upgrade them independently of python packages when there are bug fixes.
>
> This sounds like a bad idea to me. Each Python release is tested and debugged as
> a whole. If you have a lot of possible combinations (module A version 1.1 with
> module B version 1.2, etc.), it becomes impossible for us to ensure proper QA
> for the whole and as a result the quality might become lower, rather than higher.
>

You are right here !

-1 for splitting the test code and test suite

but I still think it could be a good idea ...

> (of course, if we rigorously enforce APIs and preserve compatibility,

-1

> this might
> not be a real issue; but our compatibility story is a bit irregular, IMHO)
>

mainly because of this ;o)

But it's still possible to use a parameterized `setup.py` and leave
things just like they are right now.

For instance, I have started something like that has been dome by the
FLiOOPS  project [1]_ (and possibly others, it's just an example ;o).
In the SVN repository there's a single trunk containing the whole
PyOOP package (which is not complete BTW). Inside of it there are
separate `setup.py` files for other distributions too :

  - `PyOOP` (which should contain them all) employs default `setup.py`
  - `dutest` is a single file (there are more in there) under
`oop.utils` package and
     is distributed separately too, so you can find in there `setup_dutest.py`
     script. All other packages rely on it to build test suites ;o)
  - CASPy (Comprehensive Assertion Support for Python) should be distributed
    separately too, so it should have its own `setup_dbc.py` script
once it's ready.

When I create (SVN) tags for each package, I have to rename it
(extra-overhead), and I once global metadata changes, then I have to
change them all

PS: Another alternative for stdlib could be to have common data in
`setup.cfg` and separate `setup.py` scripts, but I don't really like
this one ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Sobrepasa las 300 descargas el m?dulo dutest  -
http://feedproxy.google.com/~r/simelo-es/~3/U5rff5iTQPI/sobrepasa-las-300-descargas-el-modulo.html

From steven.bethard at gmail.com  Wed Sep 30 17:39:14 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 30 Sep 2009 08:39:14 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <FA4C011E-2A18-4F8F-B555-BAB8AD78A306@python.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
	<DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
	<d11dcfba0909300822x34a05e8en942571230a543561@mail.gmail.com>
	<FA4C011E-2A18-4F8F-B555-BAB8AD78A306@python.org>
Message-ID: <d11dcfba0909300839y5d6e01cew916fae1187a1a2a6@mail.gmail.com>

On Wed, Sep 30, 2009 at 8:31 AM, Barry Warsaw <barry at python.org> wrote:
> On Sep 30, 2009, at 11:22 AM, Steven Bethard wrote:
>> On Wed, Sep 30, 2009 at 5:21 AM, Barry Warsaw <barry at python.org> wrote:
>>> On Sep 29, 2009, at 11:15 PM, Martin v. L?wis wrote:
>>>> I would propose that the format argument gets an argument name,
>>>> according to the syntax it is written in. For PEP 3101 format,
>>>> I would call the argument "format" (like the method name of the
>>>> string type), i.e.
>>>>
>>>> logging.Formatter(
>>>> ?format="{asctime} - {name} - {levelname} - {message}")
>>>>
>>>> For the % formatting, I suggest "dicttemplate" (assuming that
>>>> you *have* to use dictionary %(key)s style currently).
>>
>> Could you comment on what you think we should do when the parameter is
>> not positional? As I mentioned upthread, in the case of
>> logging.Formatter, it's already documented as taking the keyword
>> parameter "fmt", so we'd have to use the name "fmt" for % formatting.
>
> I'm okay with fmt==%-formatting and format=={}-formatting, but I'd also be
> okay with transitioning 'fmt' to 'dicttemplate' or whatever. ?I think the
> important thing is to be explicit in the method signature which one you want
> (secondary would be trying to standardize this across the stdlib).

Thanks for the clarification. I generally like this approach, though
it's not so convenient for argparse which already takes format strings
like this::

    parser = ArgumentParser(usage='%(prog)s [--foo]')
    parser.add_argument(
        '--foo', type=int, default=42,
        help='A foo of type %(type)s, defaulting to %(42)s)

That is, existing keyword arguments that already have good names (and
are pretty much always used as keyword arguments) take format strings.
I'm not sure that changing the name of usage= or help= here is really
an option.

I guess in this case I'm stuck with something like Benjamin's
suggestion of adding an additional flag to control which type of
formatting, and the corresponding 4 versions of cleanup. Ew.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From barry at python.org  Wed Sep 30 17:50:28 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 30 Sep 2009 11:50:28 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0909300839y5d6e01cew916fae1187a1a2a6@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
	<DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
	<d11dcfba0909300822x34a05e8en942571230a543561@mail.gmail.com>
	<FA4C011E-2A18-4F8F-B555-BAB8AD78A306@python.org>
	<d11dcfba0909300839y5d6e01cew916fae1187a1a2a6@mail.gmail.com>
Message-ID: <FA7D27DA-36F5-49E2-85B8-789E2A89647A@python.org>

On Sep 30, 2009, at 11:39 AM, Steven Bethard wrote:

> Thanks for the clarification. I generally like this approach, though
> it's not so convenient for argparse which already takes format strings
> like this::
>
>    parser = ArgumentParser(usage='%(prog)s [--foo]')
>    parser.add_argument(
>        '--foo', type=int, default=42,
>        help='A foo of type %(type)s, defaulting to %(42)s)
>
> That is, existing keyword arguments that already have good names (and
> are pretty much always used as keyword arguments) take format strings.
> I'm not sure that changing the name of usage= or help= here is really
> an option.

Ah right.

> I guess in this case I'm stuck with something like Benjamin's
> suggestion of adding an additional flag to control which type of
> formatting, and the corresponding 4 versions of cleanup. Ew.

I missed Benjamin's suggestion, but in this case I would say add a  
flag to ArgumentParser.  I'm either going to want {} formatting all or  
nothing.  E.g.

import argparse
parser = ArgumentParser(usage='{prog} [--foo]', format=argparse.BRACES)
parser.add_argument(
     '--foo', type=int, default=42,
     help='A foo of type {type}, defaulting to {42}')

(although that last looks weird ;).

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090930/7caccaf5/attachment.pgp>

From olemis at gmail.com  Wed Sep 30 17:51:27 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 30 Sep 2009 10:51:27 -0500
Subject: [Python-Dev] sharing stdlib across python implementations
In-Reply-To: <24ea26600909300837m577947b4s7e537b048740adbd@mail.gmail.com>
References: <4AB8D9F2.6050807@simplistix.co.uk>
	<4AB8E136.8040706@simplistix.co.uk>
	<5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com>
	<f6bc9b490909220823j79bdd1a3j4c905939ef007b3e@mail.gmail.com>
	<1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com>
	<19129.39295.970536.916767@montanaro.dyndns.org>
	<4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com>
	<4AC36B2A.6020406@simplistix.co.uk>
	<loom.20090930T163756-153@post.gmane.org>
	<24ea26600909300837m577947b4s7e537b048740adbd@mail.gmail.com>
Message-ID: <24ea26600909300851n1f56e1fcs9d648329a1b81349@mail.gmail.com>

On Wed, Sep 30, 2009 at 10:37 AM, Olemis Lang <olemis at gmail.com> wrote:
> On Wed, Sep 30, 2009 at 9:43 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Chris Withers <chris <at> simplistix.co.uk> writes:
>>>
[...]
>
> For instance, I have started something like that has been dome by the
> FLiOOPS ?project [1]_

Sorry for the missing reference and typos, should be

For instance, something like that has been done by the FLiOOPS ?project [1]_

.. [1] py trunk @ FLiOOPS @ sf.net
        (http://sourceforge.net/apps/trac/flioops/browser/py/trunk)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Sobrepasa las 300 descargas el m?dulo dutest  -
http://feedproxy.google.com/~r/simelo-es/~3/U5rff5iTQPI/sobrepasa-las-300-descargas-el-modulo.html

From rdmurray at bitdance.com  Wed Sep 30 18:14:37 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 30 Sep 2009 12:14:37 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de>
	<loom.20090928T084811-560@post.gmane.org>
	<4AC11855.7080406@v.loewis.de>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909301200240.18193@kimball.webabinitio.net>

On Wed, 30 Sep 2009 at 10:52, Paul Moore wrote:
> 2009/9/30 Mark Dickinson <dickinsm at gmail.com>:
>> Please could someone who understands the uses of IPNetwork better than
>> I do explain why the following wouldn't be a significant problem, if __eq__
>> and __hash__ were modified to disregard the .ip attribute as suggested:
>>
>>>>> linus = IPv4Network('172.16.200.1/24')
>>>>> snoopy = IPv4Network('172.16.200.3/24')
>>>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'}
>>>>> fqdn[linus] ?# expecting 'linus.peanuts.net'
>> 'snoopy.peanuts.net'
>
> I certainly don't understand IPv4Network better than you :-) But that
> just looks wrong to me - linus and snoopy are hosts not networks, so
> making them IPv4Network classes seems wrong. I'd instinctively make
> them IPv4Address objects (which, I believe, would work).

I didn't reply to Guido's post about not wanting a third class
(IPAddressWithMask or somesuch) because it made it harder for people to
choose the "right" class to use.  He went on to say that if there was
only one class, they'd have a 100% chance of getting it right.

Well, the original ipaddr implementation had one class(*), but it wasn't
deemed acceptable by the community, and at least one real user
of ipaddr has expressed appreciation of the 2.0 addition of the
explicit IPAddress data type, so it seems like that was the right
direction to go.

Now the new implementation has two classes, and the above example shows
that people _will_ make the wrong choice.  I'd say that was _because of_
the loose parsing and the ip attribute.  If IPv4Network only accepted
valid network addresses and masks, and there was a separate parsing
function that took an arbitrary IP+netmask and returned an (IPAddress,
IPNetwork) tuple, then the user would choose the right class, IMO,
because otherwise they couldn't even get their code to parse the input.
That seems like good design to me.

But I think I'm descending to beating a dead horse here....

--David (RDM)

(*) yes, I'm ignoring the IPv4/IPv6 split throughout this message.

From vinay_sajip at yahoo.co.uk  Wed Sep 30 18:16:51 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 30 Sep 2009 16:16:51 +0000 (UTC)
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com>
	<d11dcfba0909272018y20bddaf2u6e3378520b0ba012@mail.gmail.com>
	<1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com>
	<d11dcfba0909272045y414d0b4td3d8c9e88eee18f7@mail.gmail.com>
	<d11dcfba0909272046p5fd6ca42l632e448633ce7d73@mail.gmail.com>
	<9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com>
	<20090928083820.GA488@laurie.devork>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<9d153b7c0909280818v30760a44p11ab24ba9251d4ef@mail.gmail.com>
Message-ID: <loom.20090930T181127-959@post.gmane.org>

Yuvgoog Greenle <ubershmekel <at> gmail.com> writes:

>
+1 for adding argparse and eventually deprecating optparse, -0 for deprecating
getopt.

> 2. Big modules - 1 uber-module for each subject that does everything. Maybe
logging is an example of this.

I'm not sure that it is, if you mean code size. In Python 2.5, logging is 1300
SLOC, less than say tarfile, pickletools, pydoc and decimal.

Regards,

Vinay Sajip


From foom at fuhm.net  Wed Sep 30 18:19:31 2009
From: foom at fuhm.net (James Y Knight)
Date: Wed, 30 Sep 2009 12:19:31 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <200910010034.27427.steve@pearwood.info>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
Message-ID: <ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>


On Sep 30, 2009, at 10:34 AM, Steven D'Aprano wrote:
>> E.g.
>> x = newstyle_formatstr("{} {} {}")
>> x % (1,2,3) == x.format(1,2,3) == "1 2 3"
>
> Moving along, let's suppose the newstyle_formatstr is introduced.  
> What's
> the intention then? Do we go through the std lib and replace every  
> call
> to (say)
>    somestring % args
> with
>    newstyle_formatstr(somestring) % args
> instead? That seems terribly pointless to me

Indeed, that *would* be terribly pointless! Actually, more than  
pointless, it would be broken, as you've changed the API from taking  
oldstyle format strings to newstyle format strings.

That is not the suggestion. The intention is to change /nearly  
nothing/ in the std lib, and yet allow users to use newstyle string  
substitution with every API.

Many Python APIs (e.g. logging) currently take a %-type formatting  
string. It cannot simply be changed to take a {}-type format string,  
because of backwards compatibility concerns. Either a new API can be  
added to every one of those functions/classes, or, a single API can be  
added to inform those places to use newstyle format strings.

>> This could allow for a controlled switch towards the new format
>> string format, with a long deprecation period for users to migrate:
>>
>> 1) introduce the above feature, and recommend in docs that people
>> only ever use new-style format strings, wrapping the string in
>> newstyle_formatstr() when necessary for passing to an API which uses
>> % internally.
>
> And how are people supposed to know what the API uses internally?

It's documented, (as it already must be, today!).

> Personally, I think your chances of getting people to write:
> logging.Formatter(newstyle_formatstr("%(asctime)s - %(name)s - % 
> (level)s - %(msg)s"))
> instead of
> logging.Formatter("%(asctime)s - %(name)s - %(level)s - %(msg)s")

That's not my proposal.

The user could write either:
logging.Formatter("%(asctime)s - %(name)s - %(level)s - %(msg)s")
(as always -- that can't be changed without a long deprecation  
period), or:
logging.Formatter(newstyle_formatstr("{asctime} - {name} - {level} -  
{msg}")

This despite the fact that logging has not been changed to use {}- 
style formatting internally. It should continue to call "self._fmt %  
record.__dict__" for backward compatibility.

That's not to say that this proposal would allow no work to be done to  
check the stdlib for issues. The Logging module presents one: it  
checks if the format string contains "%{asctime}" to see if it should  
bother to calculate the time. That of course would need to be changed.  
Best would be to stick an instance which lazily generates its string  
representation into the dict. The other APIs mentioned on this thread  
(BaseHTTPServer, email.generator) will work immediately without  
changes, however.

James

From vinay_sajip at yahoo.co.uk  Wed Sep 30 18:21:52 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 30 Sep 2009 16:21:52 +0000 (UTC)
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
Message-ID: <loom.20090930T181855-247@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:

> Obviously if one of the getopt supporters has a better way of doing
> this then please speak up.

I'm not a getopt supporter per se - I'm +1 for argparse - but Opster provides
some nice syntax sugar on top of getopt, and can be seen here:

http://blogg.ingspree.net/blog/2009/09/14/opster/

I've contacted Steven about a similar approach for argparse, and we're waiting
for Yuvgoog's work on argparse(func) to be done to see if there's a feasible
similar approach.




From python at rcn.com  Wed Sep 30 18:36:53 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 30 Sep 2009 09:36:53 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com><4AC2CD65.30905@v.loewis.de><DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
	<d11dcfba0909300822x34a05e8en942571230a543561@mail.gmail.com>
Message-ID: <8A34E62222BC4795B959E26E5F313AA0@RaymondLaptop1>


>>> The positional parameter would also mean dicttemplate, and
>>> would be deprecated (eventually requiring a keyword-only
>>> parameter).
>>
>> Although I hate the name 'dicttemplate', this seems like the best solution
>> to me. Maybe it's good that 'dicttemplate' is so ugly though so that people
>> will naturally prefer 'format' :). But I like this because there's not
>> really any magic, it's explicit, and the decision is made by the coder at
>> the call site. The implementation does not need to guess at all.
> 
> Could you comment on what you think we should do when the parameter is
> not positional? As I mentioned upthread, in the case of
> logging.Formatter, it's already documented as taking the keyword
> parameter "fmt", so we'd have to use the name "fmt" for % formatting.

Unless there is a firm decision to switch to kill %-formatting across the board,
I don't think anything should be done at all.   

Creating Py3.x was all about removing cruft and clutter.  I don't think it 
would be improved by adding two ways to do it for everything in the 
standard library.  That is a lot of additional code, API expansion, and 
new testing, fatter docs, and extra maintenance, but giving us no new 
functionality.

Anytime we start hearing about newstyle/oldstyle combinations, I think
a flag should go up.  Anytime there is a proposal to make sweeping
additions that do not  add new capabilities, a flag should go up.

I understand the desire to have all formatting support both ways,
but I don't think it is worth the costs.  People *never* need both ways
though they may have differing preferences about which *one* to use.

my-two-cents,


Raymond 





From tjreedy at udel.edu  Wed Sep 30 18:38:44 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 30 Sep 2009 12:38:44 -0400
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	<8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com>	<200909280330.30643.steve@pearwood.info>	<4AC03410.4010104@v.loewis.de>	<loom.20090928T084811-560@post.gmane.org>	<4AC11855.7080406@v.loewis.de>	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>	<4AC12AC6.80304@v.loewis.de>
	<4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
Message-ID: <ha01ik$cv5$1@ger.gmane.org>

Mark Dickinson wrote:
> On Wed, Sep 30, 2009 at 1:44 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Please could someone who understands the uses of IPNetwork better than
> I do explain why the following wouldn't be a significant problem, if __eq__
> and __hash__ were modified to disregard the .ip attribute as suggested:
> 
>>>> linus = IPv4Network('172.16.200.1/24')
>>>> snoopy = IPv4Network('172.16.200.3/24')
>>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'}
>>>> fqdn[linus]  # expecting 'linus.peanuts.net'
> 'snoopy.peanuts.net'
> 
> Is this just a problem of education, teaching the users not to abuse
> IPv4Network this way?  Or is this just an unlikely use of IPv4Network?
> Or have I misunderstood the proposal altogether?

This gets at why I suggested the docs be organized as I suggested, with 
'Network is range of addresses defined by address within range ...'
and 'Definition address is retained' *immediately* followed by 
'Definition address is mostly ignored, including for comparisons'. The 
latter caveat should mention the implication for hash() and set/dict 
membership. The point is to emphasize from the beginning that a network 
is a network (with an address, that many will ignore), rather than an 
address (with a network).

Terry Jan Reedy


From guido at python.org  Wed Sep 30 18:38:50 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Sep 2009 09:38:50 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <loom.20090930T181855-247@post.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com> 
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com> 
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com> 
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com> 
	<loom.20090930T181855-247@post.gmane.org>
Message-ID: <ca471dc20909300938h64ace968l2621499ebc4c3a6c@mail.gmail.com>

On a tangent -- a use case that I see happening frequently but which
is pretty poorly supported by optparse is a command-line that has a
bunch of general flags, then a 'subcommand name', and then more flags,
specific to the subcommand. Most here are probably familiar with this
pattern from SVN, Hg, and other revision control systems (P4 anyone?)
with a rich command-line interface. There are some variants, e.g.
whether global and subcommand-specific flags may overlap, and whether
flags may follow positional args (Hg and SVN seem to differ here a
bit).

I've helped write at least two tools at Google that have this
structure; both used different approaches, and neither was
particularly easy to get right. Getting all the different --help
output to make sense was mostly a manual process. (E.g. "foo --help"
should print the general flags and the list of known subcommands,
whereas "foo --help subcommand" should print flags and other usage
info about the specific subcommand.) Also switching out to different
calls based on the subcommand should/might be part of this.

I would be willing to live with a third option parser in the stdlib if
it solved this problem well.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From robert.kern at gmail.com  Wed Sep 30 18:44:23 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 30 Sep 2009 11:44:23 -0500
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
Message-ID: <ha01t7$e2j$1@ger.gmane.org>

On 2009-09-29 18:38 PM, Steven Bethard wrote:

> I don't really use GUI libraries, so I wouldn't be able to come up
> with such an example. I'd also rather not make API changes based on
> speculative use cases, so before I spend time documenting these
> things, I'd really like to hear from someone who has already, say,
> used getopt or optparse in conjunction with a GUI library, and what
> feedback they have about that.

I use argparse (and previously optparse) frequently to handle the command line 
arguments of GUI apps. I tend to use them in the same way as CLI programs, 
though, since I usually only use command line arguments when starting the GUIs 
from the terminal. I am blissfully unaware of the problems Paul mentioned about 
Windows GUI-mode programs. I'm not sure what would make a program "GUI-mode" or 
not. Certainly, I have written Python programs that use wxPython and PyQt on 
Windows that print to stdout/stderr, and they appear to work fine.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From martin at v.loewis.de  Wed Sep 30 18:48:16 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 30 Sep 2009 18:48:16 +0200
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC33DE1.3040304@trueblade.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de> <4AC33DE1.3040304@trueblade.com>
Message-ID: <4AC38BD0.9070608@v.loewis.de>

>>> There's a lot of code already out there (in the standard library and
>>> other places) that uses %-style formatting, when in Python 3.0 we
>>> should be encouraging {}-style formatting. 
>>
>> I don't agree that we should do that. I see nothing wrong with using
>> % substitution.
> 
> It's a maintenance burden.

Well - that's the cost of keeping it in the language. It's not a problem
with using it while it *is* in the language.

So if a decision was made to eventually remove % formatting, it would
be reasonable to start migrating code to PEP 3101. However, no such
decision has been made (and hopefully won't be throughout 3.x), so
as the mechanism *is* available, there is no need to start changing
existing code (except the for actual issue Steven discusses, namely
libraries that expect strings in % template form).

> I'd note that PEP 3101 calls str.format() a replacement for
> %-formatting, not an alternate mechanism to achieve the same end.

I think this is a mis-wording; the intent of the PEP apparently is
to propose this mechanism as an option, not as an actual replacement.
This becomes clear when reading the "Backwards Compatibility" section:

# Backwards compatibility can be maintained by leaving the existing
# mechanisms in place.  The new system does not collide with any of
# the method names of the existing string formatting techniques, so
# both systems can co-exist until it comes time to deprecate the
# older system.

Regards,
Martin

From tjreedy at udel.edu  Wed Sep 30 18:49:34 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 30 Sep 2009 12:49:34 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
Message-ID: <ha026u$f5h$1@ger.gmane.org>

James Y Knight wrote:
 > allow users to use newstyle string substitution with every API.

However it is done, I think someone (like new Python programmers) should 
be able to program in Python3, and use everything in the stdlib, without 
ever learning % formatting -- and that I should be able to forget about 
it ;-).

+10 on the goal.

Terry Jan Reedy


From martin at v.loewis.de  Wed Sep 30 18:50:25 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 30 Sep 2009 18:50:25 +0200
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
	<DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
Message-ID: <4AC38C51.5050807@v.loewis.de>

> Although I hate the name 'dicttemplate', this seems like the best
> solution to me.  Maybe it's good that 'dicttemplate' is so ugly though
> so that people will naturally prefer 'format' :).  But I like this
> because there's not really any magic, it's explicit, and the decision is
> made by the coder at the call site.  The implementation does not need to
> guess at all.

Unfortunately, as Steven pointed out, the parameter is *already*
documented with the name "fmt". So one option would be to call it
"fmt" and "format"; the other option would be to not only deprecate
the positional passing, but also the passing under the name fmt=.

As for calling it "dicttemplate" - I'm sure people can and will propose
alternative spellings :-)

Regards,
Martin

From robert.kern at gmail.com  Wed Sep 30 18:58:20 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 30 Sep 2009 11:58:20 -0500
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <ca471dc20909300938h64ace968l2621499ebc4c3a6c@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<loom.20090930T181855-247@post.gmane.org>
	<ca471dc20909300938h64ace968l2621499ebc4c3a6c@mail.gmail.com>
Message-ID: <ha02nc$gtt$1@ger.gmane.org>

On 2009-09-30 11:38 AM, Guido van Rossum wrote:
> On a tangent -- a use case that I see happening frequently but which
> is pretty poorly supported by optparse is a command-line that has a
> bunch of general flags, then a 'subcommand name', and then more flags,
> specific to the subcommand. Most here are probably familiar with this
> pattern from SVN, Hg, and other revision control systems (P4 anyone?)
> with a rich command-line interface. There are some variants, e.g.
> whether global and subcommand-specific flags may overlap, and whether
> flags may follow positional args (Hg and SVN seem to differ here a
> bit).
>
> I've helped write at least two tools at Google that have this
> structure; both used different approaches, and neither was
> particularly easy to get right. Getting all the different --help
> output to make sense was mostly a manual process. (E.g. "foo --help"
> should print the general flags and the list of known subcommands,
> whereas "foo --help subcommand" should print flags and other usage
> info about the specific subcommand.) Also switching out to different
> calls based on the subcommand should/might be part of this.
>
> I would be willing to live with a third option parser in the stdlib if
> it solved this problem well.

I don't think argparse supports the "foo --help subcommand" OOB. I think it 
would be simple to modify argparse to make it do so. It does support general 
options followed by a subcommand with options, though.

http://argparse.googlecode.com/svn/tags/r101/doc/other-methods.html#sub-commands

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From solipsis at pitrou.net  Wed Sep 30 19:01:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 30 Sep 2009 17:01:35 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?=
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
Message-ID: <loom.20090930T185811-479@post.gmane.org>

James Y Knight <foom <at> fuhm.net> writes:
> 
> The user could write either:
> logging.Formatter("%(asctime)s - %(name)s - %(level)s - %(msg)s")
> (as always -- that can't be changed without a long deprecation  
> period), or:
> logging.Formatter(newstyle_formatstr("{asctime} - {name} - {level} -  
> {msg}")

Why not allow logging.Formatter to take a callable, which would in turn call the
callable with keyword arguments?

Therefore, you could write:
   logging.Formatter("{asctime} - {name} - {level} - {msg}".format)

and then:
   logging.critical(name="Python", msg="Buildbots are down")

All this without having to learn about a separate "compatibility wrapper object".

Regards

Antoine.



From python at rcn.com  Wed Sep 30 19:05:05 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 30 Sep 2009 10:05:05 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>	<200910010034.27427.steve@pearwood.info><ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<ha026u$f5h$1@ger.gmane.org>
Message-ID: <0AC8D406FC894629AA95257777B7B70C@RaymondLaptop1>


[Terry Reedy]
> However it is done, I think someone (like new Python programmers) should 
> be able to program in Python3, and use everything in the stdlib, without 
> ever learning % formatting -- and that I should be able to forget about 
> it ;-).
> 
> +10 on the goal.

If that were possible, it would be nice.  But as long as the language supports
%-formatting, it is going to be around in one form or another.   Any non-casual 
user will bump into %-formatting in books, in third-party modules, in ASPN recipes, 
on the newsgroup, and in our own source code.  If they maintain any exising
software, they will likely encounter too.  It doesn't seem to be a subject that
can be ignored.

Also, I think it premature to say that {}-formatting has been proven in battle.
AFAICT, there has been very little uptake.  I've personally made an effort
to use {}-formatting more often but find that I frequently have to lookup
the syntax and need to experiment with the interactive interpreter to get it right.
I haven't found it easy to teach or to get other people to convert.  This is
especially true if the person has encountered %-formatting in other languages
(it is a popular approach).


Raymond

From steven.bethard at gmail.com  Wed Sep 30 19:14:17 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 30 Sep 2009 10:14:17 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <FA7D27DA-36F5-49E2-85B8-789E2A89647A@python.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<4AC2CD65.30905@v.loewis.de>
	<DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org>
	<d11dcfba0909300822x34a05e8en942571230a543561@mail.gmail.com>
	<FA4C011E-2A18-4F8F-B555-BAB8AD78A306@python.org>
	<d11dcfba0909300839y5d6e01cew916fae1187a1a2a6@mail.gmail.com>
	<FA7D27DA-36F5-49E2-85B8-789E2A89647A@python.org>
Message-ID: <d11dcfba0909301014l4239644fu5e77e85eda3a4a95@mail.gmail.com>

On Wed, Sep 30, 2009 at 8:50 AM, Barry Warsaw <barry at python.org> wrote:
> On Sep 30, 2009, at 11:39 AM, Steven Bethard wrote:
>
>> Thanks for the clarification. I generally like this approach, though
>> it's not so convenient for argparse which already takes format strings
>> like this::
>>
>> ? parser = ArgumentParser(usage='%(prog)s [--foo]')
>> ? parser.add_argument(
>> ? ? ? '--foo', type=int, default=42,
>> ? ? ? help='A foo of type %(type)s, defaulting to %(42)s)

Yep, sorry, typo, that should have been %(default)s, not %(42)s.

>> I guess in this case I'm stuck with something like Benjamin's
>> suggestion of adding an additional flag to control which type of
>> formatting, and the corresponding 4 versions of cleanup. Ew.
>
> I missed Benjamin's suggestion, but in this case I would say add a flag to
> ArgumentParser. ?I'm either going to want {} formatting all or nothing.
> ?E.g.
>
> import argparse
> parser = ArgumentParser(usage='{prog} [--foo]', format=argparse.BRACES)

Yeah, that's basically Benjamin's suggestion, with the transition path being:

(1) Introduce format= keyword argument, defaulting to PERCENTS
(2) Deprecate format=PERCENTS
(3) Error on format=PERCENTS (Benjamin suggested just changing the
default here, but that would give a release where errors would pass
silently)
(4) Deprecate format= keyword argument.
(5) Remove format= keyword argument.

It's a little sad that it takes 5 versions to do this, but I guess if
a user is on top of things, at version (1) they add format=BRACES to
all their code, and then remove those at version (4). So even though
there are 5 versions, there are only two code changes required.

At least in the case of argparse, this can be a constructor argument
as you suggest, and we only have to introduce this flag in one place.

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From guido at python.org  Wed Sep 30 19:37:20 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Sep 2009 10:37:20 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <4AC38BD0.9070608@v.loewis.de>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com> 
	<4AC2CD65.30905@v.loewis.de> <4AC33DE1.3040304@trueblade.com> 
	<4AC38BD0.9070608@v.loewis.de>
Message-ID: <ca471dc20909301037g239d6c10l43643d77d2eef48f@mail.gmail.com>

On Wed, Sep 30, 2009 at 9:48 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I'd note that PEP 3101 calls str.format() a replacement for
>> %-formatting, not an alternate mechanism to achieve the same end.
>
> I think this is a mis-wording; the intent of the PEP apparently is
> to propose this mechanism as an option, not as an actual replacement.
> This becomes clear when reading the "Backwards Compatibility" section:

The problem is, PEP 3101 and our interpretation of it evolved. The
original proposal for {}-formatting was certainly put forward with the
aim to completely *replace* %-formatting, and care was taken in the
design to cover all use cases, avoid known problems, etc.

Then we started looking seriously at conversion from Python 2 to
Python 3 and we discovered that converting %-formatting to
{}-formatting was a huge can of worms, and decided it wasn't worth to
try and do *at the time* given the Python 3 schedule. We considered
some kind of gentle deprecation warning, but decided that even that
would be too noisy. So now we have two competing mechanisms.

In the long run, say Python 4, I think we don't need both, and we
should get rid of one. My preference is still getting rid of
%-formatting, due to the problems with it that prompted the design of
{}-formatting (no need to reiterate the list here).

So how do we get there? My proposal would be to let this be a gradual
take-over of a new, superior species in the same niche as an older
species. (Say, modern man over Neanderthal man.) Thus, as new code is
written (especially example code, which will be copied widely), we
should start using {}-formatting, and when new APIs are designed that
tie in to some kind of formatting, they should use {}-formatting.
Adding support for {}-formatting, in addition to %-formatting, to
existing APIs like the logging package also strikes me as a good idea,
as long as backwards compatibility can be preserved. (I have no strong
ideas on how to do this right now.)

If we do this right, by the time Python 4 comes around, {}-formatting
will have won the race, and there won't be a question about removing
%-formatting at the time. I wouldn't be surprised if by then static
analysis techniques will have improved so that we *can* consider
automatic conversion by then.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Wed Sep 30 19:51:12 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 30 Sep 2009 19:51:12 +0200
Subject: [Python-Dev] Python logging and 1.5.2 compatibility
In-Reply-To: <loom.20090930T160811-948@post.gmane.org>
References: <loom.20090930T152706-630@post.gmane.org>	<h9vn2a$3lg$1@ger.gmane.org>
	<loom.20090930T160811-948@post.gmane.org>
Message-ID: <4AC39A90.2050007@v.loewis.de>

> PEP 291 (Backward Compatibility for Standard Library) mentions logging as having
> 1.5.2 compatibility, and I'm thinking that it can be removed from that PEP
> altogether.

Done!

Martin

From barry at python.org  Wed Sep 30 20:22:53 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 30 Sep 2009 14:22:53 -0400
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <ha02nc$gtt$1@ger.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<loom.20090930T181855-247@post.gmane.org>
	<ca471dc20909300938h64ace968l2621499ebc4c3a6c@mail.gmail.com>
	<ha02nc$gtt$1@ger.gmane.org>
Message-ID: <212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org>

On Sep 30, 2009, at 12:58 PM, Robert Kern wrote:

> I don't think argparse supports the "foo --help subcommand" OOB. I  
> think it would be simple to modify argparse to make it do so. It  
> does support general options followed by a subcommand with options,  
> though.

Right.  I've made it kind of work in Mailman 3, but it would be nice  
for argparse to support this out of the box.  Note that I think you  
want two forms:

foo help subcommand
foo subcommand --help

to basically print the same help text.  This is the way bzr does it  
for example and it works great.

Other than that, I think argparse supports Guido's use case very nicely.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090930/2161f5a6/attachment.pgp>

From barry at python.org  Wed Sep 30 20:27:59 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 30 Sep 2009 14:27:59 -0400
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <loom.20090930T185811-479@post.gmane.org>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>
	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>
	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<200910010034.27427.steve@pearwood.info>
	<ABAA2878-0444-42E5-B0EC-FB35BDDAA1D6@fuhm.net>
	<loom.20090930T185811-479@post.gmane.org>
Message-ID: <5353D5BD-1862-42C2-913F-C75AD73002DC@python.org>

On Sep 30, 2009, at 1:01 PM, Antoine Pitrou wrote:

> Why not allow logging.Formatter to take a callable, which would in  
> turn call the
> callable with keyword arguments?
>
> Therefore, you could write:
>   logging.Formatter("{asctime} - {name} - {level} - {msg}".format)

This is a very interesting idea.

Note that one of the reasons to /at least/ support {}-strings also is  
that %-strings are simply too error prone in many situations.  For  
example, if I decide to support internationalization of log format  
strings, and all I can use is %-strings, it's almost guaranteed that I  
will have bugs because a translator forgot the trailing 's'.  This  
exactly the motivation that led to PEP 292 $-strings.

In fact, while we're at it, it would be kind of cool if I could use $- 
strings in log templates.  Antoine's idea of accepting a callable  
might fit that bill nicely.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090930/c8c788f5/attachment.pgp>

From brett at python.org  Wed Sep 30 19:46:28 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 30 Sep 2009 10:46:28 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <loom.20090930T181855-247@post.gmane.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com> 
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com> 
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com> 
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com> 
	<loom.20090930T181855-247@post.gmane.org>
Message-ID: <bbaeab100909301046r68f0d7bcsca8b720c12adb3f2@mail.gmail.com>

On Wed, Sep 30, 2009 at 09:21, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Brett Cannon <brett <at> python.org> writes:
>
>> Obviously if one of the getopt supporters has a better way of doing
>> this then please speak up.
>
> I'm not a getopt supporter per se - I'm +1 for argparse - but Opster provides
> some nice syntax sugar on top of getopt, and can be seen here:
>
> http://blogg.ingspree.net/blog/2009/09/14/opster/
>
> I've contacted Steven about a similar approach for argparse, and we're waiting
> for Yuvgoog's work on argparse(func) to be done to see if there's a feasible
> similar approach.

I am reluctant to jump on these argument parsing decorators until they
have existed outside the standard library for a while and have settled
down as I suspect some people want to use annotations to do type
conversion/checking, others don't, etc.

-Brett

From sridharr at activestate.com  Wed Sep 30 21:36:01 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Wed, 30 Sep 2009 12:36:01 -0700
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-FC243A.21293229092009@news.gmane.org>
	<21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org>
Message-ID: <op.u024qbnl1q4jlt@whymac.activestate.com>

2.6.3rc1 builds fine on Linux x86/x86_64, MacOSX 10.4 ppc/x86, Windows  
32bit/64bit, HP-UX, AIX and Solaris just like 2.6.2 did.

-srid

On Wed, 30 Sep 2009 05:34:02 -0700, Barry Warsaw <barry at python.org> wrote:

> On Sep 30, 2009, at 12:29 AM, Ned Deily wrote:
>
>> In my opinion, the standard python.org OS X installer for 2.6.3 now
>> "works well" on 10.4, 10.5, and 10.6 and is ready for release.
>
> Fantastic, thanks.  Martin's uploaded the Windows binaries so I'll
> make the announcement now.  No commits to the 2.6 tree are allowed
> without checking with me first (via IRC, bug tracker or email).  I'll
> make an exception for svnmerge bookkeeping.
>
> -Barry
>


From barry at python.org  Wed Sep 30 21:44:14 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 30 Sep 2009 15:44:14 -0400
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <op.u024qbnl1q4jlt@whymac.activestate.com>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-FC243A.21293229092009@news.gmane.org>
	<21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org>
	<op.u024qbnl1q4jlt@whymac.activestate.com>
Message-ID: <E67D3FEF-CF1A-48F1-8E4B-A614794C57CF@python.org>

On Sep 30, 2009, at 3:36 PM, Sridhar Ratnakumar wrote:

> 2.6.3rc1 builds fine on Linux x86/x86_64, MacOSX 10.4 ppc/x86,  
> Windows 32bit/64bit, HP-UX, AIX and Solaris just like 2.6.2 did.

Thanks for the feedback!  Did you run the test suite on any of these?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090930/ce21720c/attachment.pgp>

From python at rcn.com  Wed Sep 30 22:03:06 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 30 Sep 2009 13:03:06 -0700
Subject: [Python-Dev] Python 2.6.3
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org><4AA7ADCB.6080101@gmail.com><11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org><nad-2A73EB.10290309092009@ger.gmane.org><9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org><nad-FC243A.21293229092009@news.gmane.org><21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org>
	<op.u024qbnl1q4jlt@whymac.activestate.com>
Message-ID: <446A32AEF715427CB9D76CB23A368E96@RaymondLaptop1>


[Sridhar Ratnakumar]
> 2.6.3rc1 builds fine on Linux x86/x86_64, MacOSX 10.4 ppc/x86, Windows  
> 32bit/64bit, HP-UX, AIX and Solaris just like 2.6.2 did.

Did the test suite pass too?


Raymond

From sridharr at activestate.com  Wed Sep 30 22:06:47 2009
From: sridharr at activestate.com (Sridhar Ratnakumar)
Date: Wed, 30 Sep 2009 13:06:47 -0700
Subject: [Python-Dev] Python 2.6.3
In-Reply-To: <E67D3FEF-CF1A-48F1-8E4B-A614794C57CF@python.org>
References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org>
	<4AA7ADCB.6080101@gmail.com>
	<11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org>
	<nad-2A73EB.10290309092009@ger.gmane.org>
	<9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org>
	<nad-FC243A.21293229092009@news.gmane.org>
	<21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org>
	<op.u024qbnl1q4jlt@whymac.activestate.com>
	<E67D3FEF-CF1A-48F1-8E4B-A614794C57CF@python.org>
Message-ID: <op.u0255laz1q4jlt@whymac.activestate.com>

On Wed, 30 Sep 2009 12:44:14 -0700, Barry Warsaw <barry at python.org> wrote:

> 2.6.3rc1 builds fine on Linux x86/x86_64, MacOSX 10.4 ppc/x86, Windows  
> 32bit/64bit, HP-UX, AIX and Solaris just like 2.6.2 did.
> Thanks for the feedback!  Did you run the test suite on any of these?

I will run the tests sometime tonight and let you know.

-srid


From guido at python.org  Wed Sep 30 22:11:12 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 30 Sep 2009 13:11:12 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net> 
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com> 
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> 
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> 
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> 
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> 
	<4AC33BA1.8050203@gmail.com>
	<5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com>
Message-ID: <ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>

On Wed, Sep 30, 2009 at 4:33 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Wed, Sep 30, 2009 at 12:06 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Mark Dickinson wrote:
>>> Okay, so maybe this is an abuse of IPv4Network. ?But I'd (mis?)understood
>>> that the retention of the .ip attribute was precisely a convenience to allow
>>> this sort of use. ?If not, then what's it for? ?I've read the PEP and almost
>>> all of this thread, but I can't help feeling I'm still missing something. ?If
>>> someone could point out the obvious to me I'd be grateful.
>>
>> You're not missing anything that I'm aware of - unlike the use case for
>> accepting a denormalised network definition in the IPNetwork constructor
>> (which has been made quite clear in the list discussion, even if it is
>> still a bit vague in the PEP), the use case for *retaining* the host
>> information on the network object hasn't been well articulated at all.
>>
>> The closest anyone has come to describing a use case is an indirect
>> comment via Guido that leaving out the attribute would involve real code
>> having to find somewhere else to stash the original address details
>> (e.g. by passing around an IPAddres/IPNetwork tuple rather than just an
>> IPNetwork object).
>
> Ah, thanks---I'd missed that bit. ?So the .ip attribute is mainly for
> backwards compatibility with existing uses/users of ipaddr. ?I guess
> that makes sense, then. ?In particular, if it's suggested that new code
> shouldn't make use of the .ip attribute, then the list/dict membership
> problems described above can't arise.

I don't know -- this is (from what Peter told me) a common use case so
he (and presumably  others) would from time to time have to write
extra code to keep track of that IP address in a new app. Since this
is just one extra attribute on an IPNetwork object I don't think that
adding extra classes which only differ from the IPvXNetwork classes in
having this one extra attribute is worthy.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rdmurray at bitdance.com  Wed Sep 30 22:33:27 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 30 Sep 2009 16:33:27 -0400 (EDT)
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> 
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net> 
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com> 
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> 
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> 
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> 
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> 
	<4AC33BA1.8050203@gmail.com>
	<5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com>
	<ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0909301618550.18193@kimball.webabinitio.net>

On Wed, 30 Sep 2009 at 13:11, Guido van Rossum wrote:
> On Wed, Sep 30, 2009 at 4:33 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>> On Wed, Sep 30, 2009 at 12:06 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> Mark Dickinson wrote:
>>>> Okay, so maybe this is an abuse of IPv4Network. ?But I'd (mis?)understood
>>>> that the retention of the .ip attribute was precisely a convenience to allow
>>>> this sort of use. ?If not, then what's it for? ?I've read the PEP and almost
>>>> all of this thread, but I can't help feeling I'm still missing something. ?If
>>>> someone could point out the obvious to me I'd be grateful.
>>>
>>> You're not missing anything that I'm aware of - unlike the use case for
>>> accepting a denormalised network definition in the IPNetwork constructor
>>> (which has been made quite clear in the list discussion, even if it is
>>> still a bit vague in the PEP), the use case for *retaining* the host
>>> information on the network object hasn't been well articulated at all.
>>>
>>> The closest anyone has come to describing a use case is an indirect
>>> comment via Guido that leaving out the attribute would involve real code
>>> having to find somewhere else to stash the original address details
>>> (e.g. by passing around an IPAddres/IPNetwork tuple rather than just an
>>> IPNetwork object).
>>
>> Ah, thanks---I'd missed that bit. ?So the .ip attribute is mainly for
>> backwards compatibility with existing uses/users of ipaddr. ?I guess
>> that makes sense, then. ?In particular, if it's suggested that new code
>> shouldn't make use of the .ip attribute, then the list/dict membership
>> problems described above can't arise.
>
> I don't know -- this is (from what Peter told me) a common use case so
> he (and presumably  others) would from time to time have to write
> extra code to keep track of that IP address in a new app. Since this
> is just one extra attribute on an IPNetwork object I don't think that
> adding extra classes which only differ from the IPvXNetwork classes in
> having this one extra attribute is worthy.

I don't believe anyone ever suggested adding a class that differed from
IPvXNetwork by the presence or absence of an 'ip' attribute.  The two
approaches(*) suggested were:

     1) do not add another class, just pass around (IPvXAddress,
     IPVXNetwork) tuples when the address needs to be retained (or write
     your own tailored trivial class, like I did in my example).

     2) add a class that differs from IPvXAddress by having a 'network'
     attribute that points (possibly lazily) to an IPvXNetwork object,
     and perhaps 'netmask' and 'hostmask' attributes.

Especially after my experience with writing a real example program, I
prefer (1).

--David

(*) For completeness there was also a third approach: add a 'network'
attribute to IPvXAddress that would be None if there were no associated
netmask.  Conceptual problems with this were raised which I won't
summarize; IMO it would only be slightly better than IPvXNetwork having an
'.ip' attribute.

From amcnabb at mcnabbs.org  Wed Sep 30 22:17:35 2009
From: amcnabb at mcnabbs.org (Andrew McNabb)
Date: Wed, 30 Sep 2009 14:17:35 -0600
Subject: [Python-Dev] PEP 389: argparse - new command line parsing	module
In-Reply-To: <212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<loom.20090930T181855-247@post.gmane.org>
	<ca471dc20909300938h64ace968l2621499ebc4c3a6c@mail.gmail.com>
	<ha02nc$gtt$1@ger.gmane.org>
	<212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org>
Message-ID: <20090930201735.GD27796@mcnabbs.org>

On Wed, Sep 30, 2009 at 02:22:53PM -0400, Barry Warsaw wrote:
>
> Right.  I've made it kind of work in Mailman 3, but it would be nice for 
> argparse to support this out of the box.  Note that I think you want two 
> forms:
>
> foo help subcommand
> foo subcommand --help
>
> to basically print the same help text.  This is the way bzr does it for 
> example and it works great.

In some commands, options as well as subcommands can change subsequent
parsing.  The iptables command is a good example of a command-line
program that follows this practice.  From the man page:

  after [a module name is specified], various extra command line options
  become available, depending on the specific module.  You can specify
  multiple extended match modules in one line, and you can use the -h or
  --help options after the module has been specified to receive help
  specific to that module.

In the case of iptables, module names are specified as options, not as
subcommands.

>From my cursory reading of the documentation, it looks like argparse can
only add subparsers for subcommands.  Is there any way to add subparsers
based on options instead (as iptables does)?

Also, is it possible to add these subparsers dynamically?  For example,
you would want to be able to load a module immediately after parsing the
name instead of having to keep a predetermined list of all module names.
I'm pretty sure that bzr dynamically loads modules this way.  Can
argparse help with this?

Sorry for all of the questions.  I ask them because I have some
experience with adding the above features to optparse, and it was a lot
of work to get it right.  It also seems like there are a lot of programs
that need to load modules dynamically.  I would be really excited if
argparse made this easier than optparse did.


-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868

From robert.kern at gmail.com  Wed Sep 30 23:01:32 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 30 Sep 2009 16:01:32 -0500
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <20090930201735.GD27796@mcnabbs.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<20090928130317.GR23367@snowy.squish.net>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<loom.20090930T181855-247@post.gmane.org>	<ca471dc20909300938h64ace968l2621499ebc4c3a6c@mail.gmail.com>	<ha02nc$gtt$1@ger.gmane.org>	<212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org>
	<20090930201735.GD27796@mcnabbs.org>
Message-ID: <ha0gvd$4k8$1@ger.gmane.org>

On 2009-09-30 15:17 PM, Andrew McNabb wrote:
> On Wed, Sep 30, 2009 at 02:22:53PM -0400, Barry Warsaw wrote:
>>
>> Right.  I've made it kind of work in Mailman 3, but it would be nice for
>> argparse to support this out of the box.  Note that I think you want two
>> forms:
>>
>> foo help subcommand
>> foo subcommand --help
>>
>> to basically print the same help text.  This is the way bzr does it for
>> example and it works great.
>
> In some commands, options as well as subcommands can change subsequent
> parsing.  The iptables command is a good example of a command-line
> program that follows this practice.  From the man page:
>
>    after [a module name is specified], various extra command line options
>    become available, depending on the specific module.  You can specify
>    multiple extended match modules in one line, and you can use the -h or
>    --help options after the module has been specified to receive help
>    specific to that module.
>
> In the case of iptables, module names are specified as options, not as
> subcommands.
>
>> From my cursory reading of the documentation, it looks like argparse can
> only add subparsers for subcommands.  Is there any way to add subparsers
> based on options instead (as iptables does)?

I have not done so, but I suspect so. The implementation of .add_subparsers() 
adds to the positional argument list, but one could be written to append to the 
option list.

> Also, is it possible to add these subparsers dynamically?  For example,
> you would want to be able to load a module immediately after parsing the
> name instead of having to keep a predetermined list of all module names.
> I'm pretty sure that bzr dynamically loads modules this way.  Can
> argparse help with this?

Not out-of-box, but it looks fairly straightforward to plug in. The subparser 
logic is mostly encapsulated in the _SubparsersAction class. You can register a 
new class for it:

     parser.register('action', 'parsers', MySubParsersAction)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From ubershmekel at gmail.com  Wed Sep 30 23:15:14 2009
From: ubershmekel at gmail.com (Yuvgoog Greenle)
Date: Thu, 1 Oct 2009 00:15:14 +0300
Subject: [Python-Dev] Announcing PEP 3136
In-Reply-To: <ca471dc20909291347j7ffb202dv6f7a5b8fd0f2a9bd@mail.gmail.com>
References: <20090929202035.GA5492@gmail.com>
	<ca471dc20909291347j7ffb202dv6f7a5b8fd0f2a9bd@mail.gmail.com>
Message-ID: <9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>

I like how python has a minimalistic and powerful syntax (-1 for the break
___ PEP).
Also, I really dislike the for/else ambiguity "butterflies".
When is the else after a loop executed?1. When the loop isn't entered at
all.
2. When the loop terminates through exhaustion of the list (does this
include when the list was empty?)
3. When the loop didn't exit because of a break statement.

HINTS: The way django does it is opposite the way python does it and there
may be more than one correct answer.

Any chances of cleaning this one up for python 4? I'm not sure how though I
have a few ideas.

On Tue, Sep 29, 2009 at 11:47 PM, Guido van Rossum <guido at python.org> wrote:

> On Tue, Sep 29, 2009 at 1:20 PM, Hatem Nassrat <hnassrat at gmail.com> wrote:
> > Tue Jul 3 10:14:17 CEST 2007, Guido van Rossum wrote:
> >> On 6/30/07, Matt Chisholm <matt-python at theory.org> wrote:
> >> > I've created and submitted a new PEP proposing support for labels in
> >> > Python's break and continue statements.  Georg Brandl has graciously
> >> > added it to the PEP list as PEP 3136:
> >> >
> >> > http://www.python.org/dev/peps/pep-3136/
> >>
> >> I think this is a good summary of various proposals that have been
> >> floated in the past, plus some new ones. As a PEP, it falls short
> >> because it doesn't pick a solution but merely offers a large menu of
> >> possible options. Also, there is nothing about implementation yet.
> >>
> >> However, I'm rejecting it on the basis that code so complicated to
> >> require this feature is very rare. In most cases there are existing
> >> work-arounds that produce clean code, for example using 'return'.
> >
> > I agree that this feature will only serve as a quick hack and in many
> > cases it would be misused and ugly code will be the result. However,
> > it seems that there are some shortcuts that have sneaked into python
> > (I am a fairly new Python programmer, only since late 2.4, so don't
> > shoot me). The specific one of which I speak about is:
> >
> > while_stmt ::=  "while" expression ":" suite
> >                ["else" ":" suite]
> >
> > for_stmt ::=  "for" target_list "in" expression_list ":" suite
> >              ["else" ":" suite]
> >
> > try1_stmt ::=  "try" ":" suite
> >               ("except" [expression ["as" target]] ":" suite)+
> >               ["else" ":" suite]
> >               ["finally" ":" suite]
> >
> > All these else's seem like shortcuts to me. I did find a use for them,
> > once I found out they existed, but I get butterflies whenever I do.
>
> In English, butterflies are usually a good thing (they mean you'e in love).
>
> These else clauses (assuming you're talking about that) have been in
> the language pretty much forever. The combined except/finally clause
> is newer, it was added because Java allows it and it was actually a
> pretty common usage.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091001/40d5bfa2/attachment.htm>

From fuzzyman at voidspace.org.uk  Wed Sep 30 23:21:47 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 30 Sep 2009 22:21:47 +0100
Subject: [Python-Dev] Announcing PEP 3136
In-Reply-To: <9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>
References: <20090929202035.GA5492@gmail.com>	<ca471dc20909291347j7ffb202dv6f7a5b8fd0f2a9bd@mail.gmail.com>
	<9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com>
Message-ID: <4AC3CBEB.1000303@voidspace.org.uk>

Yuvgoog Greenle wrote:
> I like how python has a minimalistic and powerful syntax (-1 for the 
> break ___ PEP).
>
> Also, I really dislike the for/else ambiguity "butterflies".
>
> When is the else after a loop executed?
> 1. When the loop isn't entered at all.
If you mean because the list / iterator is empty then yes. If you mean 
something else then explain.

> 2. When the loop terminates through exhaustion of the list (does this 
> include when the list was empty?)
Yes.
> 3. When the loop didn't exit because of a break statement.
>

No.

This is pretty much the definition of the else statement semantics and 
are clearly defined.

> HINTS: The way django does it is opposite the way python does it and 
> there may be more than one correct answer.
>
If you mean the Django template language does it differently from Python 
then you'll have to speak to the Django guys. The Django template 
language may be inspired by Python and written in Python but it isn't 
Python and has no obligation to use the same semantics. Python is very 
clear.

> Any chances of cleaning this one up for python 4? I'm not sure how 
> though I have a few ideas.
>
See above. The else statement semantics for both loops and exception 
handling are extremely useful and don't need cleaning up. If you have 
further ideas then the Python-ideas list is the right place to discuss 
potential language changes or additions.

All the best,


Michael Foord

> On Tue, Sep 29, 2009 at 11:47 PM, Guido van Rossum <guido at python.org 
> <mailto:guido at python.org>> wrote:
>
>     On Tue, Sep 29, 2009 at 1:20 PM, Hatem Nassrat <hnassrat at gmail.com
>     <mailto:hnassrat at gmail.com>> wrote:
>     > Tue Jul 3 10:14:17 CEST 2007, Guido van Rossum wrote:
>     >> On 6/30/07, Matt Chisholm <matt-python at theory.org
>     <http://theory.org>> wrote:
>     >> > I've created and submitted a new PEP proposing support for
>     labels in
>     >> > Python's break and continue statements.  Georg Brandl has
>     graciously
>     >> > added it to the PEP list as PEP 3136:
>     >> >
>     >> > http://www.python.org/dev/peps/pep-3136/
>     >>
>     >> I think this is a good summary of various proposals that have been
>     >> floated in the past, plus some new ones. As a PEP, it falls short
>     >> because it doesn't pick a solution but merely offers a large
>     menu of
>     >> possible options. Also, there is nothing about implementation yet.
>     >>
>     >> However, I'm rejecting it on the basis that code so complicated to
>     >> require this feature is very rare. In most cases there are existing
>     >> work-arounds that produce clean code, for example using 'return'.
>     >
>     > I agree that this feature will only serve as a quick hack and in
>     many
>     > cases it would be misused and ugly code will be the result. However,
>     > it seems that there are some shortcuts that have sneaked into python
>     > (I am a fairly new Python programmer, only since late 2.4, so don't
>     > shoot me). The specific one of which I speak about is:
>     >
>     > while_stmt ::=  "while" expression ":" suite
>     >                ["else" ":" suite]
>     >
>     > for_stmt ::=  "for" target_list "in" expression_list ":" suite
>     >              ["else" ":" suite]
>     >
>     > try1_stmt ::=  "try" ":" suite
>     >               ("except" [expression ["as" target]] ":" suite)+
>     >               ["else" ":" suite]
>     >               ["finally" ":" suite]
>     >
>     > All these else's seem like shortcuts to me. I did find a use for
>     them,
>     > once I found out they existed, but I get butterflies whenever I do.
>
>     In English, butterflies are usually a good thing (they mean you'e
>     in love).
>
>     These else clauses (assuming you're talking about that) have been in
>     the language pretty much forever. The combined except/finally clause
>     is newer, it was added because Java allows it and it was actually a
>     pretty common usage.
>
>     --
>     --Guido van Rossum (home page: http://www.python.org/~guido/
>     <http://www.python.org/%7Eguido/>)
>     _______________________________________________
>     Python-Dev mailing list
>     Python-Dev at python.org <mailto:Python-Dev at python.org>
>     http://mail.python.org/mailman/listinfo/python-dev
>     Unsubscribe:
>     http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From vinay_sajip at yahoo.co.uk  Wed Sep 30 23:23:12 2009
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 30 Sep 2009 21:23:12 +0000 (UTC)
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<20090928130317.GR23367@snowy.squish.net>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<loom.20090930T181855-247@post.gmane.org>
	<bbaeab100909301046r68f0d7bcsca8b720c12adb3f2@mail.gmail.com>
Message-ID: <loom.20090930T232124-53@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:

> I am reluctant to jump on these argument parsing decorators until they
> have existed outside the standard library for a while and have settled
> down as I suspect some people want to use annotations to do type
> conversion/checking, others don't, etc.
> 

I agree - it's just an approach which shows some promise and is worth exploring.
I'm not suggesting jumping in with both feet. But it does show about getopt that
perhaps there's life in the old dog yet ;-)

Regards,

Vinay Sajip


From martin at v.loewis.de  Wed Sep 30 23:34:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 30 Sep 2009 23:34:11 +0200
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
	<4AC33BA1.8050203@gmail.com>	<5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com>
	<ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>
Message-ID: <4AC3CED3.80104@v.loewis.de>

> I don't know -- this is (from what Peter told me) a common use case so
> he (and presumably  others) would from time to time have to write
> extra code to keep track of that IP address in a new app.

I, and probably others, would really, REALLY like to see one such
"common use case". I do find it puzzling that non of the readers here
can think of any, and the continued assertion that use cases do exist
without them being presented is somewhat disturbing.

Regards,
Martin

From steven.bethard at gmail.com  Wed Sep 30 23:40:20 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 30 Sep 2009 14:40:20 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <20090930201735.GD27796@mcnabbs.org>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>
	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>
	<200909290127.03215.steve@pearwood.info>
	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>
	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>
	<loom.20090930T181855-247@post.gmane.org>
	<ca471dc20909300938h64ace968l2621499ebc4c3a6c@mail.gmail.com>
	<ha02nc$gtt$1@ger.gmane.org>
	<212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org>
	<20090930201735.GD27796@mcnabbs.org>
Message-ID: <d11dcfba0909301440r35d30edbk584c8ee6958c7df0@mail.gmail.com>

On Wed, Sep 30, 2009 at 1:17 PM, Andrew McNabb <amcnabb at mcnabbs.org> wrote:
> >From my cursory reading of the documentation, it looks like argparse can
> only add subparsers for subcommands. ?Is there any way to add subparsers
> based on options instead (as iptables does)?

Currently this is not supported, but it would be a nice feature.

> Also, is it possible to add these subparsers dynamically? ?For example,
> you would want to be able to load a module immediately after parsing the
> name instead of having to keep a predetermined list of all module names.
> I'm pretty sure that bzr dynamically loads modules this way. ?Can
> argparse help with this?

You can probably already do this. I'm not 100% sure what you want to
do, but it's certainly possible to define an argparse.Action that
loads a module when it's invoked. It might look something like::

    class MyAction(argparse.Action):
        def __call__(self, parser, namespace, value, option_string=None):
            mod = __import__(value) # or whatever

Steve
-- 
Where did you get that preposterous hypothesis?
Did Steve tell you that?
        --- The Hiphopopotamus

From ncoghlan at gmail.com  Wed Sep 30 12:43:50 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Sep 2009 20:43:50 +1000
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <d11dcfba0909300035s7e67264l925b3b8c5388da3e@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com>	<4AC2CD65.30905@v.loewis.de>	<d11dcfba0909292135u2f27967pfb8cada6becc32fb@mail.gmail.com>	<A84B24EE-0F56-44A5-911A-10610D98444C@fuhm.net>
	<d11dcfba0909300035s7e67264l925b3b8c5388da3e@mail.gmail.com>
Message-ID: <4AC33666.9080709@gmail.com>

Steven Bethard wrote:
> So I understand how this might help a user to move from %-style
> formatting to {}-style formatting, but it's still not clear to me how
> to use this to evolve an API. In particular, if the goal is for the
> API to move from accepting %-style format strings to {}-style format
> strings, how should that API use newstyle_formatstr or
> oldstyle_formatstr to make this transition?

The problem is that many (most?) of the problematic APIs (such as
logging) will have multiple users in a given application, so getting the
granularity of any behavioural switches right is going to be difficult.

Providing a formatstr() type that makes .__mod__() work based on a
.format() style string (and a formatmod() type that does the opposite)
would allow for extremely fine-grained decision making, since every
format string will either be an ordinary str instance or else an
instance of the formatting subclass.

(Note that the primary use case for both proposed types is an
application developer adapting between two otherwise incompatible third
party libraries - the choice of class just changes based on whether the
old code being adapted is the code invoking mod on a format string or
the code providing a format string that expects to be used with the mod
operator).

I don't see any way for delayed formatting of "normal" strings in any
existing API to move away from %-formatting except via a long and
painful deprecation process (i.e. first warning when bare strings are
used for a release or two, then switching entirely to the new formatting
method) or by duplicating the API and maintaining the two interfaces in
parallel for the foreseeable future.

As Paul noted, the two proposed classes may also be useful to the
library developer during such a transition process - they could accept
strings in the "wrong" format just by wrapping them appropriately rather
than having to maintain the parallel APIs all the way through the
software stack.

Probably worth letting these concepts bake for a while longer, but it
definitely be nice to do *something* to help enable this transition in
2.7/3.2.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From peter at hda3.com  Wed Sep 30 23:48:25 2009
From: peter at hda3.com (Peter Moody)
Date: Wed, 30 Sep 2009 14:48:25 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <4AC3CED3.80104@v.loewis.de>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
	<4AC33BA1.8050203@gmail.com>
	<5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com>
	<ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>
	<4AC3CED3.80104@v.loewis.de>
Message-ID: <8517e9350909301448v4a7f891q3e12cec01ebde44a@mail.gmail.com>

[- at google.com]

this is what I wrote:

it's not so much a need as it is useful.  it's useful to take an
address like 192.168.1.100/24 and derive a bunch of information from
it (like the network address, broadcast address, containing supernets,
etc), but still remember that the original address was 192.168.1.100.
having a separate class or two for this is overly burdensome in my
mind. FWIW, netaddr does the same thing.

Cheers,
/peter

On Wed, Sep 30, 2009 at 2:34 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I don't know -- this is (from what Peter told me) a common use case so
>> he (and presumably ?others) would from time to time have to write
>> extra code to keep track of that IP address in a new app.
>
> I, and probably others, would really, REALLY like to see one such
> "common use case". I do find it puzzling that non of the readers here
> can think of any, and the continued assertion that use cases do exist
> without them being presented is somewhat disturbing.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From python at rcn.com  Wed Sep 30 23:51:19 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 30 Sep 2009 14:51:19 -0700
Subject: [Python-Dev] PEP 3144 review.
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com><Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net><ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com><4AC12AC6.80304@v.loewis.de>
	<4AC2A9D7.4080603@gmail.com><5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com><79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com><5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com><4AC33BA1.8050203@gmail.com>	<5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com><ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>
	<4AC3CED3.80104@v.loewis.de>
Message-ID: <8656B2EE4F6648CCA5A717C5E99D93A8@RaymondLaptop1>


>> I don't know -- this is (from what Peter told me) a common use case so
>> he (and presumably  others) would from time to time have to write
>> extra code to keep track of that IP address in a new app.

[MvL]
> I, and probably others, would really, REALLY like to see one such
> "common use case". I do find it puzzling that non of the readers here
> can think of any, and the continued assertion that use cases do exist
> without them being presented is somewhat disturbing.

A concrete use case would help put the discussion on a better technical footing.

FWIW, it doesn't look like any of the competing packages ever
found a need to add support for this yet-to-be-presented use case:  
http://code.google.com/p/netaddr/wiki/YetAnotherPythonIPModule



Raymond


From peter at hda3.com  Wed Sep 30 23:56:07 2009
From: peter at hda3.com (Peter Moody)
Date: Wed, 30 Sep 2009 14:56:07 -0700
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <8656B2EE4F6648CCA5A717C5E99D93A8@RaymondLaptop1>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
	<4AC33BA1.8050203@gmail.com>
	<5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com>
	<ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>
	<4AC3CED3.80104@v.loewis.de>
	<8656B2EE4F6648CCA5A717C5E99D93A8@RaymondLaptop1>
Message-ID: <8517e9350909301456t2051fd48yab3948c0b903bc43@mail.gmail.com>

[- at google.com]

On Wed, Sep 30, 2009 at 2:51 PM, Raymond Hettinger <python at rcn.com> wrote:
>
>>> I don't know -- this is (from what Peter told me) a common use case so
>>> he (and presumably ?others) would from time to time have to write
>>> extra code to keep track of that IP address in a new app.
>
> [MvL]
>>
>> I, and probably others, would really, REALLY like to see one such
>> "common use case". I do find it puzzling that non of the readers here
>> can think of any, and the continued assertion that use cases do exist
>> without them being presented is somewhat disturbing.
>
> A concrete use case would help put the discussion on a better technical
> footing.
>
> FWIW, it doesn't look like any of the competing packages ever
> found a need to add support for this yet-to-be-presented use case:
> ?http://code.google.com/p/netaddr/wiki/YetAnotherPythonIPModule

netaddr actually does have support for this.

>>> import netaddr
>>> netaddr.IPNetwork('1.1.1.1/24').ip
IPAddress('1.1.1.1')
>>>

cheers,
/peter

> Raymond
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com
>

From daniel at stutzbachenterprises.com  Wed Sep 30 23:59:56 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 30 Sep 2009 16:59:56 -0500
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <Pine.LNX.4.64.0909301618550.18193@kimball.webabinitio.net>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
	<4AC33BA1.8050203@gmail.com>
	<5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com>
	<ca471dc20909301311p440b89f0u7e882f00f750a5d@mail.gmail.com>
	<Pine.LNX.4.64.0909301618550.18193@kimball.webabinitio.net>
Message-ID: <eae285400909301459y445d021fh254a129eb34fc578@mail.gmail.com>

.On Wed, Sep 30, 2009 at 3:33 PM, R. David Murray <rdmurray at bitdance.com>wrote:

>    1) do not add another class, just pass around (IPvXAddress,
>    IPVXNetwork) tuples when the address needs to be retained (or write
>    your own tailored trivial class, like I did in my example).
>

I've been puzzled by objections to forcing the user to decide what metadata
to store.  Every time I've ever needed to store an IP address or a network,
I've always need to store a bunch of other affiliated data with it.  E.g.:

class GnutellaPeer:
    # IP address, port, timestamp, and dozen of other fields

class NetworkInterface:
   # IP address, subnet mask, name, MAC address, etc.

class Route:
   # interface, IP address, subnet mask, gateway address, route priority

Is it such a burden for the programmer to store one extra field in the class
they will inevitably write?

For that matter, I do not see the advantage to an IPNetwork class that saves
the programmer from having to separately store the IP address and subnet
mask.  Maybe I am biased by my background: I learned network programming in
C, and I think of an IP address as little more than an integer type with a
special string representation.

People have voiced support for the IPNetwork class and for the occasional
utility of an .ip field.  I assume they have good use cases.  It would be
nice if the use cases were collected into the PEP, in a clear and articulate
way.  Preferably by someone other than ipaddr author, for whom the use cases
are understandably a bit too obvious to explain at length with
exasperation.  It aught to be easy to justify the functionality of the
library, if the use cases are clearly articulated.

The current PEP begins by noting that many other IP address libraries are
available, but doesn't describe what makes ipaddr unique or superior other
than a claim to being lightweight.  After downloading several of the other
IP address libraries (http://bit.ly/483Yw4), ipaddr appears to be the second
largest, after the huge netaddr package.<http://packages.python.org/netaddr/>
I worry that this discussion has focused too much on the details of ipaddr
(and the false dichotomy of "ipaddr versus nothing"), without properly
tackling the question of "What use-cases for IP addresses are sufficiently
universal* that they belong in the standard library?"

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090930/f316be3e/attachment.htm>

From a.badger at gmail.com  Wed Sep 30 01:59:45 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Tue, 29 Sep 2009 16:59:45 -0700
Subject: [Python-Dev] PEP 389: argparse - new command line parsing module
In-Reply-To: <d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
References: <d11dcfba0909271359i743dbbc9rf79da11564f49fd5@mail.gmail.com>	<d11dcfba0909280728l144ef629x6d35e70c76c19e6d@mail.gmail.com>	<200909290127.03215.steve@pearwood.info>	<d11dcfba0909280849o21d1fa6wa870e76a1c7d307@mail.gmail.com>	<bbaeab100909281222n3ea552d5rdae0a4c48cd061e@mail.gmail.com>	<d11dcfba0909281257h29c84eb7g9aa6a5eea8fbce72@mail.gmail.com>	<9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com>	<79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com>	<d11dcfba0909291357o3a3ac24fpfb856a6367462e99@mail.gmail.com>	<4AC2846E.8060304@g.nevcal.com>
	<d11dcfba0909291638p3a630166m9b02f073daacd8ed@mail.gmail.com>
Message-ID: <4AC29F71.3000307@gmail.com>

On 09/29/2009 04:38 PM, Steven Bethard wrote:
> On Tue, Sep 29, 2009 at 3:04 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
>> On approximately 9/29/2009 1:57 PM, came the following characters from the
>> keyboard of Steven Bethard:
>>> If you're not using argparse to write command line applications, then
>>> I don't feel bad if you have to do a tiny bit of extra work to take
>>> care of that use case. In this particular situation, all you have to
>>> do is subclass ArgumentParser and override exit() to do whatever you
>>> think it should do.
> [snip]
>>> There is only a single method in argparse that prints things,
>>> _print_message(). So if you want it to do something else, you can
>>> simply override it in a subclass. I can make that method public if
>>> this is a common use case.
>>
>> Documenting both of these options would forestall people from thinking it is
>> only useful for console applications.
> 
> I'm totally fine with people thinking it is only useful for console
> applications. That's what it's intended for. That said, if there are
> people out there who want to use it for other applications, I'm happy
> to make things easier for them if I know concretely what they want.
> 
Note: on Unix systems, --help should still print to the terminal, not
pop up a GUI text box with the help information.  So being able to
override the behaviour might be good but it is more than a simple, GUI
vs console distinction.  Are we talking about anything else than --help
output (for the printing question)?

About exit(), I agree with others about wanting to catch the exception
myself and then choosing to exit from the code.  I'm not sure that it's
actually useful in practice, though...it might just feel cleaner but not
actually be that helpful.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090929/eb4c94f4/attachment.pgp>

From brett at python.org  Wed Sep 30 20:12:02 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 30 Sep 2009 11:12:02 -0700
Subject: [Python-Dev] transitioning from % to {} formatting
In-Reply-To: <79990c6b0909300529u4383e02esa21b464b957fb015@mail.gmail.com>
References: <d11dcfba0909291727q23376384kfb521e991626e5be@mail.gmail.com> 
	<4AC2CD65.30905@v.loewis.de>
	<DA31CA13-AA53-4DCA-9E0A-AE0030DED7B8@python.org> 
	<79990c6b0909300529u4383e02esa21b464b957fb015@mail.gmail.com>
Message-ID: <bbaeab100909301112l18b9f67p27725fb34bb1c67b@mail.gmail.com>

On Wed, Sep 30, 2009 at 05:29, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/9/30 Barry Warsaw <barry at python.org>:
>> Although I hate the name 'dicttemplate', this seems like the best solution
>> to me. ?Maybe it's good that 'dicttemplate' is so ugly though so that people
>> will naturally prefer 'format' :). ?But I like this because there's not
>> really any magic, it's explicit, and the decision is made by the coder at
>> the call site. ?The implementation does not need to guess at all.
>
> Ignoring the "make it ugly to persuade people not to use it" aspect,
> why not just use 'template'?

Might confuse people with string.Template.

-Brett

From ncoghlan at gmail.com  Wed Sep 30 13:06:09 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Sep 2009 21:06:09 +1000
Subject: [Python-Dev] PEP 3144 review.
In-Reply-To: <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com>	
	<4AC03410.4010104@v.loewis.de>	
	<loom.20090928T084811-560@post.gmane.org>	
	<4AC11855.7080406@v.loewis.de>	
	<Pine.LNX.4.64.0909281623180.18193@kimball.webabinitio.net>	
	<ca471dc20909281343hb6c953fg979624798eed5d23@mail.gmail.com>	
	<4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com>	
	<5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com>	
	<79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com>
	<5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com>
Message-ID: <4AC33BA1.8050203@gmail.com>

Mark Dickinson wrote:
> On Wed, Sep 30, 2009 at 10:52 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>> 2009/9/30 Mark Dickinson <dickinsm at gmail.com>:
>>> Please could someone who understands the uses of IPNetwork better than
>>> I do explain why the following wouldn't be a significant problem, if __eq__
>>> and __hash__ were modified to disregard the .ip attribute as suggested:
>>>
>>>>>> linus = IPv4Network('172.16.200.1/24')
>>>>>> snoopy = IPv4Network('172.16.200.3/24')
>>>>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'}
>>>>>> fqdn[linus]  # expecting 'linus.peanuts.net'
>>> 'snoopy.peanuts.net'
>> I certainly don't understand IPv4Network better than you :-) But that
>> just looks wrong to me - linus and snoopy are hosts not networks, so
>> making them IPv4Network classes seems wrong. I'd instinctively make
>> them IPv4Address objects (which, I believe, would work).
> 
> Okay, so maybe this is an abuse of IPv4Network.  But I'd (mis?)understood
> that the retention of the .ip attribute was precisely a convenience to allow
> this sort of use.  If not, then what's it for?  I've read the PEP and almost
> all of this thread, but I can't help feeling I'm still missing something.  If
> someone could point out the obvious to me I'd be grateful.

You're not missing anything that I'm aware of - unlike the use case for
accepting a denormalised network definition in the IPNetwork constructor
(which has been made quite clear in the list discussion, even if it is
still a bit vague in the PEP), the use case for *retaining* the host
information on the network object hasn't been well articulated at all.

The closest anyone has come to describing a use case is an indirect
comment via Guido that leaving out the attribute would involve real code
having to find somewhere else to stash the original address details
(e.g. by passing around an IPAddres/IPNetwork tuple rather than just an
IPNetwork object).

However, while I'd still be a little happier if the .ip attribute went
away all together and another means was found to conveniently associate
an IPAddress and an IPNetwork, keeping it doesn't bother me anywhere
near as much as having network equivalence defined in terms of something
other than the network ID and the netmask.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------