From guido at python.org  Sun Jun  1 00:29:18 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 31 May 2008 15:29:18 -0700
Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String
	Redux (aka String ABC)
In-Reply-To: <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1>
References: <loom.20080527T192243-415@post.gmane.org>
	<483FB8F6.4060504@canterbury.ac.nz>
	<loom.20080530T144341-819@post.gmane.org>
	<200805311313.11501.steve@pearwood.info> <g1rd87$461$1@ger.gmane.org>
	<1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1>
Message-ID: <ca471dc20805311529y5a30a618u5c603336bd4117b5@mail.gmail.com>

I'm willing to meet you halfway. I really don't want isinstance(x,
str) to return True for something that doesn't inherit from the
concrete str type; this is bound to lead to too  much confusion and
breakage. But I'm fine with a String ABC (or any other ABC, e.g.
Atomic?) that doesn't define any methods but can be used for type
testing. How about that?

--Guido

On Sat, May 31, 2008 at 5:25 AM, Raymond Hettinger <python at rcn.com> wrote:
> ISTM, the whole reason people are asking for a String ABC is so you can
> write isinstance(obj, String) and allow registered string-like objects to be
> accepted.
>
> The downside is that everytime you want this for a concrete class or type,
> it is necessary to write a whole new ABC listing all of the required
> methods.  Also, you have to change all of the client code's isinstance tests
> from concrete to abstract.
>
> I propose a simpler approach.  Provide an alternative registration function
> that overrides isinstance() so that objects can explicitly fake any concrete
> type:
>
>  s = UserString('whiffleball')
>  print isinstance(s, str)            --> False
>  register_lookalike(UserString, str)
>  print isinstance(s, str)            --> True
>
> Besides saving us from writing tons of new ABCs, the approach works with
> existing code that already uses isinstance() with concrete types.
>
> The ABCs that would remain are ones that are truly abstract, that define a
> generic interface (like mappings and sequences) and ones that offer some
> useful mixin behavior.  The remaining ABCs are ones where you have a
> fighting chance of actually being able to implement the interface (unlike
> String where it would be darned tricky to fully emulate encode(), split(),
> etc.)
>
> This would completely eliminate the need for numbers.Integral for example.
>  AFAICT, its sole use case is to provide a way for numeric's integral types
> (like int8, int32) to pass themselves off as having the same API as regular
> ints.  Unfortunately, the current approach requires all consumer code to
> switch from isinstance(x,int) to isinstance(x,Integral).  It would be more
> useful if we could simply write register_lookalike(x,int) and be done with
> it (no need for numbers.py and its attendant abc machinery).
>
> If we don't do this, then String won't be the last request.  People will
> want Datetime for example.  Pretty much any concrete type could have a
> look-a-like that wanted its own ABC and for all client code to switch from
> testing concrete types.
>
>
> Raymond
>
>
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From r.m.oudkerk at googlemail.com  Sun Jun  1 00:31:27 2008
From: r.m.oudkerk at googlemail.com (r.m.oudkerk)
Date: Sat, 31 May 2008 23:31:27 +0100
Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module)
In-Reply-To: <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>
References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com>
	<978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com>
	<4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com>
	<g1osss$5nq$1@ger.gmane.org> <48402B50.6040809@gmail.com>
	<978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com>
	<79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>
Message-ID: <ac4216f0805311531h624ad56ei62dc60141783b3e@mail.gmail.com>

On 31/05/2008, Paul Moore <p.f.moore at gmail.com> wrote:
> 2008/5/30 Farshid Lashkari <flashk at gmail.com>:
>> I'm not sure if there will be any side affects to modifying
>> sys.executable though. Should this be the official way of supporting
>> embedded interpreters or should there be a
>> multiprocessing.setExecutable() method?
>
> +1 for setExecutable (I'd prefer set_executable, to be PEP 8
> compliant). Make it explicit, rather than fiddling with stuff in sys
> manually.

That is easy to do.

An issue not mentioned in the PEP is naming conventions.  In recent
versions I have tried to consistently use mixedCase for functions and
methods (other than factory functions) because that is what threading
does (give or take settrace(), setprofile(), stack_size().)

I am certainly open to using lowercase/lower_case_with_underscores for
all functions/methods except for Process's methods and possibly
currentProcess(), but I would like some feed back on that.

From solipsis at pitrou.net  Sun Jun  1 00:45:24 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 31 May 2008 22:45:24 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?optimization_required=3A_=2Eformat=28=29_i?=
	=?utf-8?q?s_much_slower=09than_=25?=
References: <g1fee5$f8b$1@ger.gmane.org>
	<fb73205e0805261619r4a789590g46dcadf6676c2092@mail.gmail.com>
Message-ID: <loom.20080531T224307-298@post.gmane.org>

Simon Cross <hodgestar <at> gmail.com> writes:
> My tests show that the old-style % formatting is much faster when the
> final string is 20 characters or less:
> 
> $ ./python -m timeit "'....|....|....|...%s' % '12'"
> 10000000 loops, best of 3: 0.0764 usec per loop


You are the victim of a constant-folding optimization:

$ ./python -m timeit             "'....|....|....|...%s' % '12'"
10000000 loops, best of 3: 0.0926 usec per loop
$ ./python -m timeit -s "s='12'" "'....|....|....|...%s' % s"
1000000 loops, best of 3: 0.525 usec per loop


>>> def f(): return '....|....|....|...%s' % '12'
... 
>>> dis.dis(f)
  1           0 LOAD_CONST               3 ('....|....|....|...12') 
              3 RETURN_VALUE         


cheers

Antoine.



From ncoghlan at gmail.com  Sun Jun  1 03:26:52 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 01 Jun 2008 11:26:52 +1000
Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String
 Redux	(aka String ABC)
In-Reply-To: <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1>
References: <loom.20080527T192243-415@post.gmane.org>	<483FB8F6.4060504@canterbury.ac.nz>	<loom.20080530T144341-819@post.gmane.org><200805311313.11501.steve@pearwood.info>	<g1rd87$461$1@ger.gmane.org>
	<1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1>
Message-ID: <4841FADC.7090005@gmail.com>

Raymond Hettinger wrote:
> If we don't do this, then String won't be the last request.  People will 
> want Datetime for example.  Pretty much any concrete type could have a 
> look-a-like that wanted its own ABC and for all client code to switch 
> from testing concrete types.

If I remember rightly, the machinery in the ABC's to support 
registration slows down some other operations with the types - do we 
want to pay that price all the time?

If we do, then it may be a matter of moving some of the registration 
machinery from ABCMeta up into type itself.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From python at rcn.com  Sun Jun  1 03:41:40 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 31 May 2008 18:41:40 -0700
Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String
	Redux (aka String ABC)
References: <loom.20080527T192243-415@post.gmane.org>
	<483FB8F6.4060504@canterbury.ac.nz>
	<loom.20080530T144341-819@post.gmane.org>
	<200805311313.11501.steve@pearwood.info>
	<g1rd87$461$1@ger.gmane.org>
	<1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1>
	<ca471dc20805311529y5a30a618u5c603336bd4117b5@mail.gmail.com>
Message-ID: <34093CA46419445EBD6232F96268DFE0@RaymondLaptop1>

From: "Guido van Rossum" <guido at python.org>
> I'm willing to meet you halfway. I really don't want isinstance(x,
> str) to return True for something that doesn't inherit from the
> concrete str type; this is bound to lead to too  much confusion and
> breakage.  

Probably true.  It was an attractive idea though.  Unless all client
code converts all its isinstance() tests from concrete to abstract,
life is going to be tough for people writing look-alike classes
which will have limited applicability.


> But I'm fine with a String ABC (or any other ABC, e.g.
> Atomic?) that doesn't define any methods but can be used for type
> testing. How about that?

That's progress!  It makes abstract substitution possible while
still saving us a lot of code and avoiding over-specification.  

I propose the following empty abstract classes:  
    String, Datetime, Deque, and Socket.

-1 on Atomic though.  Earlier in the thread it was made clear that
that atomicity is not an intrinsic property of a type; instead it varies
across applications (when flattening email folders, a multi-part mime 
message is atomic, but when flattening individual messages, a
multi-part mime message is just another nested container, part
of the tree, not one of the leaves).

Are you open to considering numbers.Integral to be one of the
new empty abstract classes?  That would make it much easier
for objects wanting to pass themselves as integers.  As it stands
now, an aspiring Integral class is required to implement a number
of  arcana including:  __rxor__, __rrshift__, __pow__, __invert__,
__index__, and __long__.  


Raymond

From mhammond at skippinet.com.au  Sun Jun  1 03:28:43 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Sun, 1 Jun 2008 11:28:43 +1000
Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module)
In-Reply-To: <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>
References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com>	<978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com>	<4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com>	<g1osss$5nq$1@ger.gmane.org>
	<48402B50.6040809@gmail.com>	<978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com>
	<79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>
Message-ID: <00ac01c8c386$e10c68e0$a3253aa0$@com.au>

> 2008/5/30 Farshid Lashkari <flashk at gmail.com>:
> > I'm not sure if there will be any side affects to modifying
> > sys.executable though. Should this be the official way of supporting
> > embedded interpreters or should there be a
> > multiprocessing.setExecutable() method?
> 
> +1 for setExecutable (I'd prefer set_executable, to be PEP 8
> compliant). Make it explicit, rather than fiddling with stuff in sys
> manually.

sys.executable typically means the "current" executable, and py2exe etc
already fiddles with that.  The question in such a context seems to be "what
executable should I use for this processing functionality?".  In a py2exe
like environment, it might not be unreasonable to assume that if a custom
executable is to be used, that custom executable may have a different
command-line or other special requirements.  Further, I could imagine a
system that uses an alternative way of starting processes (eg, 'distributed
COM') where the concept of 'executable' (or even 'command-line') don't make
much sense.

So it seems that maybe simply "setExecutable()" isn't the correct
abstraction here, but maybe a "factory" approach, so the entire process
creation mechanism can be replaced rather than just the name of the
executable to spawn?

Cheers,

Mark


From guido at python.org  Sun Jun  1 04:10:20 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 31 May 2008 19:10:20 -0700
Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String
	Redux (aka String ABC)
In-Reply-To: <34093CA46419445EBD6232F96268DFE0@RaymondLaptop1>
References: <loom.20080527T192243-415@post.gmane.org>
	<483FB8F6.4060504@canterbury.ac.nz>
	<loom.20080530T144341-819@post.gmane.org>
	<200805311313.11501.steve@pearwood.info> <g1rd87$461$1@ger.gmane.org>
	<1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1>
	<ca471dc20805311529y5a30a618u5c603336bd4117b5@mail.gmail.com>
	<34093CA46419445EBD6232F96268DFE0@RaymondLaptop1>
Message-ID: <ca471dc20805311910v7c8531f0sc0bfed6ec6ac8f53@mail.gmail.com>

On Sat, May 31, 2008 at 6:41 PM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Guido van Rossum" <guido at python.org>
>> I'm willing to meet you halfway. I really don't want isinstance(x,
>> str) to return True for something that doesn't inherit from the
>> concrete str type; this is bound to lead to too  much confusion and
>> breakage.
>
> Probably true.  It was an attractive idea though.  Unless all client
> code converts all its isinstance() tests from concrete to abstract,
> life is going to be tough for people writing look-alike classes
> which will have limited applicability.

I'd rather require that people rewrite their code to benefit from some
new piece of functionality than foisting it upon them regardless,
breaking some perfectly fine working in the process. This is how we've
always done it.

>> But I'm fine with a String ABC (or any other ABC, e.g.
>> Atomic?) that doesn't define any methods but can be used for type
>> testing. How about that?
>
> That's progress!  It makes abstract substitution possible while
> still saving us a lot of code and avoiding over-specification.
> I propose the following empty abstract classes:     String, Datetime, Deque,
> and Socket.

Sounds like a mini-PEP is in place. It should focus on the code to
actually define these and the intended ways to use them.

> -1 on Atomic though.  Earlier in the thread it was made clear that
> that atomicity is not an intrinsic property of a type; instead it varies
> across applications (when flattening email folders, a multi-part mime
> message is atomic, but when flattening individual messages, a
> multi-part mime message is just another nested container, part
> of the tree, not one of the leaves).

Fine, it was just an idle thought.

> Are you open to considering numbers.Integral to be one of the
> new empty abstract classes?  That would make it much easier
> for objects wanting to pass themselves as integers.  As it stands
> now, an aspiring Integral class is required to implement a number
> of  arcana including:  __rxor__, __rrshift__, __pow__, __invert__,
> __index__, and __long__.

I don't think Integer should be completely abstract (what good is an
int you can't add 1 to?) but I could be amenable to reducing the set
of required operations (which could then resurface as a separate ABC).
Please write another mini-PEP. Where did you see __long__? That seems
a mistake (at least in 3.0).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Sun Jun  1 05:09:10 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 31 May 2008 20:09:10 -0700
Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String
	Redux (aka String ABC)
References: <loom.20080527T192243-415@post.gmane.org>
	<483FB8F6.4060504@canterbury.ac.nz>
	<loom.20080530T144341-819@post.gmane.org>
	<200805311313.11501.steve@pearwood.info>
	<g1rd87$461$1@ger.gmane.org>
	<1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1>
	<ca471dc20805311529y5a30a618u5c603336bd4117b5@mail.gmail.com>
	<34093CA46419445EBD6232F96268DFE0@RaymondLaptop1>
	<ca471dc20805311910v7c8531f0sc0bfed6ec6ac8f53@mail.gmail.com>
Message-ID: <EF1A2D2B79654AB3A88EB3F7F31A1784@RaymondLaptop1>

[Raymond]
>> I propose the following empty abstract classes:     String, Datetime, Deque,
>> and Socket.

[GvR]
> Sounds like a mini-PEP is in place. It should focus on the code to
> actually define these and the intended ways to use them.

Okay, will run a Google code search to see if real code exists that
runs isinstance tests on the concrete types.   Since the new classes 
are very lightweight (completely empty), these probably only need
minimal justification.  

The case for String has already been made.  And the concept of a 
Socket is already fully abstract. Not sure I really care about Deque.  

The Datetime.class is tricky.  The existence of many implementations
of date/time routines indicates that there is a need; however, they
don't share the API so they likely won't fit under a common umbrella.



>> Are you open to considering numbers.Integral to be one of the
>> new empty abstract classes?  That would make it much easier
>> for objects wanting to pass themselves as integers.  As it stands
>> now, an aspiring Integral class is required to implement a number
>> of  arcana including:  __rxor__, __rrshift__, __pow__, __invert__,
>> __index__, and __long__.
> 
> I don't think Integer should be completely abstract (what good is an
> int you can't add 1 to?) but I could be amenable to reducing the set
> of required operations (which could then resurface as a separate ABC).
> Please write another mini-PEP. 

Okay.  Will propose to remove the bit flipping methods and anything
else that doesn't seem essential to integeriness.  Will take a look at
the integral types in numeric to see what that actually implement.



> Where did you see __long__? That seems
> a mistake (at least in 3.0).

It's the first listed abstract method in the Py2.6 code.


Raymond

From guido at python.org  Sun Jun  1 05:46:15 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 31 May 2008 20:46:15 -0700
Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String
	Redux (aka String ABC)
In-Reply-To: <EF1A2D2B79654AB3A88EB3F7F31A1784@RaymondLaptop1>
References: <loom.20080527T192243-415@post.gmane.org>
	<483FB8F6.4060504@canterbury.ac.nz>
	<loom.20080530T144341-819@post.gmane.org>
	<200805311313.11501.steve@pearwood.info> <g1rd87$461$1@ger.gmane.org>
	<1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1>
	<ca471dc20805311529y5a30a618u5c603336bd4117b5@mail.gmail.com>
	<34093CA46419445EBD6232F96268DFE0@RaymondLaptop1>
	<ca471dc20805311910v7c8531f0sc0bfed6ec6ac8f53@mail.gmail.com>
	<EF1A2D2B79654AB3A88EB3F7F31A1784@RaymondLaptop1>
Message-ID: <ca471dc20805312046g2a09e937s70f1001bf87f0b21@mail.gmail.com>

On Sat, May 31, 2008 at 8:09 PM, Raymond Hettinger <python at rcn.com> wrote:
> [Raymond]
>>>
>>> I propose the following empty abstract classes:     String, Datetime,
>>> Deque,
>>> and Socket.
>
> [GvR]
>>
>> Sounds like a mini-PEP is in place. It should focus on the code to
>> actually define these and the intended ways to use them.
>
> Okay, will run a Google code search to see if real code exists that
> runs isinstance tests on the concrete types.

I wasn't asking for existing code -- I was asking for the code you
propose to add to abc.py (or wherever).

> Since the new classes are
> very lightweight (completely empty), these probably only need
> minimal justification.

Again, in a mini-PEP I'm not so much looking for justification but for
a precise spec.

> The case for String has already been made.

Actually I'm not sure. One you know that isinstance(x, String) is
true, what can you assume you can do with x?

> And the concept of a Socket is already fully abstract.

Can you elaborate? There's a very specific API that is assumed of
sockets. The code that creates sockets is usually pretty close to the
code that consumes it. There are some major classes that cut right
through the APIs though: connection or listening (the latter being
something on which you call accept()), stream or datagram, and as a
special case of stream SSL and the like.

> Not sure I really care about Deque.
> The Datetime.class is tricky.  The existence of many implementations
> of date/time routines indicates that there is a need; however, they
> don't share the API so they likely won't fit under a common umbrella.

Right. I'm now beginning to wonder what exactly you're after here --
saying that something is an "X" without saying anything about an API
isn't very useful. You need to have at least *some* API to be able to
do anything with that knowledge.

>>> Are you open to considering numbers.Integral to be one of the
>>> new empty abstract classes?  That would make it much easier
>>> for objects wanting to pass themselves as integers.  As it stands
>>> now, an aspiring Integral class is required to implement a number
>>> of  arcana including:  __rxor__, __rrshift__, __pow__, __invert__,
>>> __index__, and __long__.
>>
>> I don't think Integer should be completely abstract (what good is an
>> int you can't add 1 to?) but I could be amenable to reducing the set
>> of required operations (which could then resurface as a separate ABC).
>> Please write another mini-PEP.
>
> Okay.  Will propose to remove the bit flipping methods and anything
> else that doesn't seem essential to integeriness.  Will take a look at
> the integral types in numeric to see what that actually implement.
>
>
>
>> Where did you see __long__? That seems
>> a mistake (at least in 3.0).
>
> It's the first listed abstract method in the Py2.6 code.

That actually makes sense -- correct interoperability with longs
probably requires that.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From alexandre at peadrop.com  Sun Jun  1 06:11:19 2008
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Sun, 1 Jun 2008 00:11:19 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
Message-ID: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>

Would anyone mind if I did add a public C API for gc.disable() and
gc.enable()? I would like to use it as an optimization for the pickle
module (I found out that I get a good 2x speedup just by disabling the
GC while loading large pickles). Of course, I could simply import the
gc module and call the functions there, but that seems overkill to me.
I included the patch below for review.

-- Alexandre



Index: Include/objimpl.h
===================================================================
--- Include/objimpl.h   (revision 63766)
+++ Include/objimpl.h   (working copy)
@@ -221,8 +221,10 @@
  * ==========================
  */

-/* C equivalent of gc.collect(). */
+/* C equivalent of gc.collect(), gc.enable() and gc.disable(). */
 PyAPI_FUNC(Py_ssize_t) PyGC_Collect(void);
+PyAPI_FUNC(void) PyGC_Enable(void);
+PyAPI_FUNC(void) PyGC_Disable(void);

 /* Test if a type has a GC head */
 #define PyType_IS_GC(t) PyType_HasFeature((t), Py_TPFLAGS_HAVE_GC)
Index: Modules/gcmodule.c
===================================================================
--- Modules/gcmodule.c  (revision 63766)
+++ Modules/gcmodule.c  (working copy)
@@ -1252,6 +1252,18 @@
        return n;
 }

+void
+PyGC_Disable(void)
+{
+    enabled = 0;
+}
+
+void
+PyGC_Enable(void)
+{
+    enabled = 1;
+}
+
 /* for debugging */
 void
 _PyGC_Dump(PyGC_Head *g)

From rhamph at gmail.com  Sun Jun  1 06:28:35 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 31 May 2008 22:28:35 -0600
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
Message-ID: <aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>

On Sat, May 31, 2008 at 10:11 PM, Alexandre Vassalotti
<alexandre at peadrop.com> wrote:
> Would anyone mind if I did add a public C API for gc.disable() and
> gc.enable()? I would like to use it as an optimization for the pickle
> module (I found out that I get a good 2x speedup just by disabling the
> GC while loading large pickles). Of course, I could simply import the
> gc module and call the functions there, but that seems overkill to me.
> I included the patch below for review.

I'd rather see it fixed.  It behaves quadratically if you load enough
to trigger full collection a few times.


-- 
Adam Olsen, aka Rhamphoryncus

From python at rcn.com  Sun Jun  1 08:15:22 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 31 May 2008 23:15:22 -0700
Subject: [Python-Dev] Mini-Pep:  Simplifying the Integral ABC
Message-ID: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>

Target:  Py2.6 and Py3.0
Author: Raymond Hettinger
Date: May 31, 2008

Motivation
----------
The principal purpose of an abstract base class is to support multiple
implementations of an API; thereby allowing one concrete class to be
substitutable for another. This purpose is defeated when useful substitutions
are precluded because the ABC is too aggressive in its behavioral requirements
-- mandating too many methods, mandating methods that are difficult to
implement, or too closely following the full API of a single concrete type.

The Integral ABC is somewhat extensive and requires essentially every behavior
exhibited by the int concrete class.  Usefully, it provides for basic integer
behaviors like simple arithmetic and ordering relations.  However, the current
ABC goes beyond that and requires bit-flipping and other operations that are
not fundamental to the notion of being an integer.  That makes it challenging
to define a new Integral class that isn't already an int.

Proposal
--------
Remove non-essential abstract methods like __index__, three argument __pow__,
__lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__, __xor__,
__rxor__, __or__, __ror__, and __invert__, numerator, and denominator.

Discussion
----------
The only known use cases for variants of int are types that limit the range of
values to those that fit in a fixed storage width.

One existing implementation is in numpy which has types like int0, int8,
int16, int32, and int16.  The numpy integral types implement all the methods
present in Integral except for the new methods like __trunc__, __index__,
real, imag, conjugate, numerator, and denominator.  For the most part, they
are fully substitutable into code that expects regular ints; however, they do
have wrap-around behaviors such as int8(200)+int8(100) --> int8(44). The
wrap-around behaviors make the numpy types totally unsuitable for some
applications of Integral such as the fractions module which accepts any
integral numerator and denominator.


From python at rcn.com  Sun Jun  1 08:59:00 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 31 May 2008 23:59:00 -0700
Subject: [Python-Dev] Mini-Pep:  An Empty String ABC
Message-ID: <A8708637239F4FEEA25DE934DBFE639C@RaymondLaptop1>

Mini-Pep:  An Empty String ABC
Target:  Py2.6 and Py3.0
Author:  Raymond Hettinger

Proposal
--------

Add a new collections ABC specified as:

    class String(Sequence):
        pass

Motivation
----------
Having an ABC for strings allows string look-alike classes to declare
themselves as sequences that contain text.  Client code (such as a flatten
operation or tree searching tool) may use that ABC to usefully differentiate
strings from other sequences (i.e. containers vs containees).  And in code
that only relies on sequence behavior, isinstance(x,str) may be usefully
replaced by isinstance(x,String) so that look-alikes can be substituted in
calling code.

A natural temptation is add other methods to the String ABC, but strings are a
tough case.  Beyond simple sequence manipulation, the string methods get very
complex.  An ABC that included those methods would make it tough to write a
compliant class that could be registered as a String.  The split(), rsplit(),
partition(), and rpartition() methods are examples of methods that would be
difficult to emulate correctly.  Also, starting with Py3.0, strings are
essentially abstract sequences of code points, meaning that an encode() method
is essential to being able to usefully transform them back into concrete data.
Unfortunately, the encode method is so complex that it cannot be readily
emulated by an aspiring string look-alike.

Besides complexity, another problem with the concrete str API is the extensive
number of methods.  If string look-alikes were required to emulate the likes
of zfill(), ljust(), title(), translate(), join(), etc., it would
significantly add to the burden of writing a class complying with the String
ABC.

The fundamental problem is that of balancing a client function's desire to
rely on a broad number of behaviors against the difficulty of writing a
compliant look-alike class.  For other ABCs, the balance is more easily struck
because the behaviors are fewer in number, because they are easier to
implement correctly, and because some methods can be provided as mixins.  For
a String ABC, the balance should lean toward minimalism due to the large
number of methods and how difficult it is to implement some of the correctly.

A last reason to avoid expanding the String API is that almost none of the
candidate methods characterize the notion of "stringiness".  With something
calling itself an integer, an __add__() method would be expected as it is
fundamental to the notion of "integeriness".  In contrast, methods like
startswith() and title() are non-essential extras -- we would not discount
something as being not stringlike if those methods were not present.





From fredrik.johansson at gmail.com  Sun Jun  1 11:53:05 2008
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Sun, 1 Jun 2008 11:53:05 +0200
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
Message-ID: <3d0cebfb0806010253g77ce1f05i9111d8ad7b11c4ff@mail.gmail.com>

On Sun, Jun 1, 2008 at 8:15 AM, Raymond Hettinger <python at rcn.com> wrote:
> Discussion
> ----------
> The only known use cases for variants of int are types that limit the range
> of
> values to those that fit in a fixed storage width.

Add:
* Faster big integers (gmpy)
* Integers with exact division to rationals (e.g. sympy)

Fredrik

From greg.ewing at canterbury.ac.nz  Sun Jun  1 12:54:15 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 01 Jun 2008 22:54:15 +1200
Subject: [Python-Dev] Iterable String Redux (aka String ABC)
In-Reply-To: <loom.20080530T144341-819@post.gmane.org>
References: <loom.20080527T192243-415@post.gmane.org>
	<ca471dc20805271242q63e80104mfad81d1b1014df60@mail.gmail.com>
	<loom.20080527T200901-899@post.gmane.org> <g1hrsc$6a5$1@ger.gmane.org>
	<5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1>
	<08May28.143320pdt.58698@synergy1.parc.xerox.com>
	<483DFB8B.5050909@canterbury.ac.nz> <g1lkg5$g4l$1@ger.gmane.org>
	<483F312E.1020302@canterbury.ac.nz> <g1o6dr$psg$1@ger.gmane.org>
	<483FB8F6.4060504@canterbury.ac.nz>
	<loom.20080530T144341-819@post.gmane.org>
Message-ID: <48427FD7.4090005@canterbury.ac.nz>

Armin Ronacher wrote:

> basestring is not subclassable for example.  Also it requires subclassing
> which ABCs do not.

The use case that was cited was recognising subclasses of
UserString, and that's what I was responding to. If
basestring were made subclassable and UserString inherited
from it, that use case would be covered.

Recognising string-like objects *without* requiring
subclassing is a hopeless morass to get into, in my
opinion. You'll just have endless arguments about which
of the zillion methods of str should be in the blessed
set which confers string-ness.

I also think that the ABC idea in general suffers from
that problem, to one degree or another depending on
the class involved. Strings are just an extreme case.

-- 
Greg


From solipsis at pitrou.net  Sun Jun  1 13:42:58 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 1 Jun 2008 11:42:58 +0000 (UTC)
Subject: [Python-Dev] Mini-Pep:  An Empty String ABC
References: <A8708637239F4FEEA25DE934DBFE639C@RaymondLaptop1>
Message-ID: <loom.20080601T112342-101@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> Also, starting with Py3.0, strings are
> essentially abstract sequences of code points, meaning that an encode() method
> is essential to being able to usefully transform them back into concrete data.

Well, that depends:
- is a String the specification of a generic range of types which one might want
to special-case in some algorithms, e.g. flatten()
- or is a String the specification of something which is meant to be used as a
replacement of str (or, perhaps, bytes)?

If you answer the former, the String API should be very minimal and there is no
reason for it to support "encoding" or "decoding". Such a String doesn't have to
be a string of characters, it can contain arbitrary objects, e.g. DNA elements.

If you answer the latter, what use is a String subclass which isn't a drop-in
replacement for either str or bytes? Saying "hello, I'm a String" is not very
useful if you can't be used anywhere in existing code. I think most Python
coders wouldn't go out of their way to allow arbitrary String instances as
parameters for their functions, rather than objects conforming to the full str
(or, perhaps, bytes) API.


I'd like to know the use cases of a String ABC representing replacements of the
str class, though. I must admit I've never used UserString and the like, and
don't know how useful they can be. However, the docs have the following to say: 

? This UserString class from this module is available for backward compatibility
only. If you are writing code that does not need to work with versions of Python
earlier than Python 2.2, please consider subclassing directly from the built-in
str type instead of using UserString ?.

So, apart from compatibility purposes, what is the point currently of *not*
directly subclassing str?

Regards

Antoine.



From p.f.moore at gmail.com  Sun Jun  1 15:44:59 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 1 Jun 2008 14:44:59 +0100
Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module)
In-Reply-To: <ac4216f0805311531h624ad56ei62dc60141783b3e@mail.gmail.com>
References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com>
	<978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com>
	<4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com>
	<g1osss$5nq$1@ger.gmane.org> <48402B50.6040809@gmail.com>
	<978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com>
	<79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>
	<ac4216f0805311531h624ad56ei62dc60141783b3e@mail.gmail.com>
Message-ID: <79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com>

2008/5/31 r. m. oudkerk <r.m.oudkerk at googlemail.com>:
> I am certainly open to using lowercase/lower_case_with_underscores for
> all functions/methods except for Process's methods and possibly
> currentProcess(), but I would like some feed back on that.

I dislike mixedCase, but consistency with the rest of the library is
more important - and as processing is matching the API of threading,
which used mixedCase, it should follow that convention.

Wasn't there some talk of changing modules to use PEP 8 conventions
(lowercase_with_underscore) as part of the Python 3.0 conversion? Did
that ever happen?

Paul.

From p.f.moore at gmail.com  Sun Jun  1 15:57:55 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 1 Jun 2008 14:57:55 +0100
Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String
	Redux (aka String ABC)
In-Reply-To: <ca471dc20805312046g2a09e937s70f1001bf87f0b21@mail.gmail.com>
References: <loom.20080527T192243-415@post.gmane.org>
	<loom.20080530T144341-819@post.gmane.org>
	<200805311313.11501.steve@pearwood.info> <g1rd87$461$1@ger.gmane.org>
	<1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1>
	<ca471dc20805311529y5a30a618u5c603336bd4117b5@mail.gmail.com>
	<34093CA46419445EBD6232F96268DFE0@RaymondLaptop1>
	<ca471dc20805311910v7c8531f0sc0bfed6ec6ac8f53@mail.gmail.com>
	<EF1A2D2B79654AB3A88EB3F7F31A1784@RaymondLaptop1>
	<ca471dc20805312046g2a09e937s70f1001bf87f0b21@mail.gmail.com>
Message-ID: <79990c6b0806010657i75fedd33q8a220dfb339848ae@mail.gmail.com>

2008/6/1 Guido van Rossum <guido at python.org>:
>> The case for String has already been made.
>
> Actually I'm not sure. One you know that isinstance(x, String) is
> true, what can you assume you can do with x?
[...]
> Right. I'm now beginning to wonder what exactly you're after here --
> saying that something is an "X" without saying anything about an API
> isn't very useful. You need to have at least *some* API to be able to
> do anything with that knowledge.

Apologies to Raymond if I'm putting words into his mouth, but I think
it's more about *not* doing things with the type - a String is a
Sequence that we don't wish to iterate through (in the flatten case),
so the code winds up looking like

    for elem in seq:
        if isinstance(elem, Sequence) and not isinstance(elem, String):
            recurse into the element
        else:
            deal with the element as atomic

This implies that other "empty" abstract types aren't useful, though,
as they are not subclasses of anything else...

Paul.

From ncoghlan at gmail.com  Sun Jun  1 16:28:02 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 02 Jun 2008 00:28:02 +1000
Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module)
In-Reply-To: <79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com>
References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com>	
	<978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com>	
	<4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com>	
	<g1osss$5nq$1@ger.gmane.org> <48402B50.6040809@gmail.com>	
	<978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com>	
	<79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>	
	<ac4216f0805311531h624ad56ei62dc60141783b3e@mail.gmail.com>
	<79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com>
Message-ID: <4842B1F2.6010901@gmail.com>

Paul Moore wrote:
> Wasn't there some talk of changing modules to use PEP 8 conventions
> (lowercase_with_underscore) as part of the Python 3.0 conversion? Did
> that ever happen?

We fixed the module names that used mixed case - the amount of work that 
turned out to be involved in just doing that much for PEP 3108 makes me 
shudder at the thought of trying to fix all of the standard library APIs 
that currently don't follow the style guide...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From aahz at pythoncraft.com  Sun Jun  1 21:53:00 2008
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 1 Jun 2008 12:53:00 -0700
Subject: [Python-Dev] Mini-Pep:  Simplifying the Integral ABC
In-Reply-To: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
Message-ID: <20080601195300.GA2945@panix.com>

On Sat, May 31, 2008, Raymond Hettinger wrote:
>
> Proposal
> --------
> Remove non-essential abstract methods like __index__, three argument
> __pow__, __lshift__, __rlshift__, __rshift__, __rrshift__, __and__,
> __rand__, __xor__, __rxor__, __or__, __ror__, and __invert__,
> numerator, and denominator.

The only thing I object to is removing __index__ -- the whole point of an
integral class is that it is substitutable as an index for sequences in
a way that other numeric types are not.  Having an __index__ special
method is a key indicator for duck-typing purposes not covered by the
ABC.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Need a book?  Use your library!

From g.brandl at gmx.net  Sun Jun  1 22:37:07 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 01 Jun 2008 22:37:07 +0200
Subject: [Python-Dev] [Python-3000]  Finishing up PEP 3108
In-Reply-To: <5c6f2a5d0805311452v15a87d06g899fa8182dbd9d2a@mail.gmail.com>
References: <bbaeab100805282138x5ccac2d8q100fa0e969828bb4@mail.gmail.com>	<g1ll0v$i0p$1@ger.gmane.org>	<bbaeab100805291032v7749ba6n1752a0ef5be9cbca@mail.gmail.com>	<g1rr4o$956$1@ger.gmane.org>
	<5c6f2a5d0805311452v15a87d06g899fa8182dbd9d2a@mail.gmail.com>
Message-ID: <g1v19t$640$1@ger.gmane.org>

Mark Dickinson schrieb:
> On Sat, May 31, 2008 at 11:33 AM, Georg Brandl <g.brandl at gmx.net 
> <mailto:g.brandl at gmx.net>> wrote:
> 
> 
>     Now that the docs are reST, the source is almost pretty enough to
>     display
>     it raw, but I could also imagine a "text" writer that removes the more
>     obscure markup to present a casual-reader-friendly text version.
> 
>     The needed sources could then be distributed with Python -- it shouldn't
>     be more than about 200 kb.
> 
> 
> +1 from me.  Would this mean that htmllib and sgmllib could be
> removed without further ado.

OK, I've now implemented this in the trunk (will merge to 3k soon --
htmllib and sgmllib can go then).

The topic help is contained in a new module, pydoc_topics.py, which pydoc
imports to provide this help. The module can be generated with Sphinx
by running "make pydoc-topics" in the Doc/ directory. (This is one more
step for the release process, but it is an easy one.)

The module is currently ~ 400 kb in size. If this is deemed to be a problem,
we could use zlib to compress the contents -- which of course is bad for
systems without the zlib module (are there any?).

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From martin at v.loewis.de  Sun Jun  1 23:28:30 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 01 Jun 2008 23:28:30 +0200
Subject: [Python-Dev] [Python-3000]  Finishing up PEP 3108
In-Reply-To: <g1v19t$640$1@ger.gmane.org>
References: <bbaeab100805282138x5ccac2d8q100fa0e969828bb4@mail.gmail.com>	<g1ll0v$i0p$1@ger.gmane.org>	<bbaeab100805291032v7749ba6n1752a0ef5be9cbca@mail.gmail.com>	<g1rr4o$956$1@ger.gmane.org>	<5c6f2a5d0805311452v15a87d06g899fa8182dbd9d2a@mail.gmail.com>
	<g1v19t$640$1@ger.gmane.org>
Message-ID: <4843147E.8050907@v.loewis.de>

> The module is currently ~ 400 kb in size. If this is deemed to be a
> problem,
> we could use zlib to compress the contents -- which of course is bad for
> systems without the zlib module (are there any?).

In the distribution, the file gets compressed, anyway. In the
installation, I don't think it is a problem.

Regards,
Martin

From guido at python.org  Mon Jun  2 00:43:38 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 1 Jun 2008 15:43:38 -0700
Subject: [Python-Dev] Iterable String Redux (aka String ABC)
In-Reply-To: <48427FD7.4090005@canterbury.ac.nz>
References: <loom.20080527T192243-415@post.gmane.org>
	<5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1>
	<08May28.143320pdt.58698@synergy1.parc.xerox.com>
	<483DFB8B.5050909@canterbury.ac.nz> <g1lkg5$g4l$1@ger.gmane.org>
	<483F312E.1020302@canterbury.ac.nz> <g1o6dr$psg$1@ger.gmane.org>
	<483FB8F6.4060504@canterbury.ac.nz>
	<loom.20080530T144341-819@post.gmane.org>
	<48427FD7.4090005@canterbury.ac.nz>
Message-ID: <ca471dc20806011543q2cb533a0pf3770cb8cbd9844d@mail.gmail.com>

On Sun, Jun 1, 2008 at 3:54 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> The use case that was cited was recognising subclasses of
> UserString, and that's what I was responding to. If
> basestring were made subclassable and UserString inherited
> from it, that use case would be covered.

UserString intentionally doesn't subclass basestring. When basestring
was introduced, it was specifically meant to be the base class of
*only* str and unicode. There are quite a few core APIs that accept no
substitutes, and being an instance of basestring was intended to
guarantee that a value is accepted by such APIs.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From qgallet at gmail.com  Mon Jun  2 00:45:53 2008
From: qgallet at gmail.com (Quentin Gallet-Gilles)
Date: Mon, 2 Jun 2008 00:45:53 +0200
Subject: [Python-Dev] [Python-3000]  Finishing up PEP 3108
In-Reply-To: <8b943f2b0805290839s7a1f3238g9e21407a56c34159@mail.gmail.com>
References: <bbaeab100805282138x5ccac2d8q100fa0e969828bb4@mail.gmail.com>
	<g1ll0v$i0p$1@ger.gmane.org>
	<8b943f2b0805290625w19ab1fd3l48f00f40e630c39d@mail.gmail.com>
	<483EC414.7080603@ibp.de>
	<8b943f2b0805290839s7a1f3238g9e21407a56c34159@mail.gmail.com>
Message-ID: <8b943f2b0806011545x6a11f019r3c412bc3ebc1a3ab@mail.gmail.com>

I've uploaded a patch for the aifc module (http://bugs.python.org/issue2847).
I'm still working on the testsuite.
Comments are welcome!

Quentin

On Thu, May 29, 2008 at 5:39 PM, Quentin Gallet-Gilles <qgallet at gmail.com>
wrote:

>
> On Thu, May 29, 2008 at 4:56 PM, Lars Immisch <lars at ibp.de> wrote:
>
>> <snip>
>>
>>>        Issue 2847 - the aifc module still imports the cl module in 3.0.
>>>        Problem is that the cl module is gone. =) So it seems silly to
>>> have
>>>        the imports lying about. This can probably be changed to critical.
>>>
>>>
>>>    It shouldn't be a problem to rip everything cl-related out of aifc.
>>>    The question is how useful aifc will be after that ...
>>>
>>>
>>> Has someone already used that module ? I took a look into it, but I'm a
>>> bit confused about the various compression types, case-sensitivity and
>>> compatibility issues [1]. Are Apple's "alaw" and SGI's "ALAW" really the
>>> same encoding ? Can we use the audioop module for ALAW, just like it's
>>> already done for ULAW ?
>>>
>>
>> There is just one alaw I've ever come across (G.711), and the audioop
>> implementation could be used (audioop's alaw support is younger than the
>> aifc module, BTW)
>>
>> The capitalisation is confusing, but your document [1] says: "Apple
>> Computer's QuickTime player recognize only the Apple compression types.
>> Although "ALAW" and "ULAW" contain identical sound samples to the "alaw" and
>> "ulaw" formats and were in use long before Apple introduced the new codes,
>>  QuickTime does not recognize them."
>>
>> So this seems just a matter of naming in the AIFC, but not a matter of two
>> different alaw implementations.
>>
>> - Lars
>>
>
> Ok, I'll handle this issue. I'll be using the audioop implementation as a
> replacement of the SGI compression library. I'll also create a test suite,
> as Brett mentioned in the bug tracker the module was missing one.
>
> Quentin
>
>
>>
>> [1] http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/AIFF/AIFF.html
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080602/b5b4be23/attachment.htm>

From guido at python.org  Mon Jun  2 00:57:09 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 1 Jun 2008 15:57:09 -0700
Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String
	Redux (aka String ABC)
In-Reply-To: <79990c6b0806010657i75fedd33q8a220dfb339848ae@mail.gmail.com>
References: <loom.20080527T192243-415@post.gmane.org>
	<200805311313.11501.steve@pearwood.info> <g1rd87$461$1@ger.gmane.org>
	<1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1>
	<ca471dc20805311529y5a30a618u5c603336bd4117b5@mail.gmail.com>
	<34093CA46419445EBD6232F96268DFE0@RaymondLaptop1>
	<ca471dc20805311910v7c8531f0sc0bfed6ec6ac8f53@mail.gmail.com>
	<EF1A2D2B79654AB3A88EB3F7F31A1784@RaymondLaptop1>
	<ca471dc20805312046g2a09e937s70f1001bf87f0b21@mail.gmail.com>
	<79990c6b0806010657i75fedd33q8a220dfb339848ae@mail.gmail.com>
Message-ID: <ca471dc20806011557m48f829c1q9c9844315dd382b1@mail.gmail.com>

On Sun, Jun 1, 2008 at 6:57 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2008/6/1 Guido van Rossum <guido at python.org>:
>>> The case for String has already been made.
>>
>> Actually I'm not sure. One you know that isinstance(x, String) is
>> true, what can you assume you can do with x?
> [...]
>> Right. I'm now beginning to wonder what exactly you're after here --
>> saying that something is an "X" without saying anything about an API
>> isn't very useful. You need to have at least *some* API to be able to
>> do anything with that knowledge.
>
> Apologies to Raymond if I'm putting words into his mouth, but I think
> it's more about *not* doing things with the type - a String is a
> Sequence that we don't wish to iterate through (in the flatten case),
> so the code winds up looking like
>
>    for elem in seq:
>        if isinstance(elem, Sequence) and not isinstance(elem, String):
>            recurse into the element
>        else:
>            deal with the element as atomic

I thought that was he meant too, until he said he rejected my offhand
suggestion of Atomic with these words: "Earlier in the thread it was
made clear that that atomicity is not an intrinsic property of a type;
instead it varies across applications [...]"

> This implies that other "empty" abstract types aren't useful, though,
> as they are not subclasses of anything else...

There's a thread on this out now I believe.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Jun  2 01:06:10 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 1 Jun 2008 16:06:10 -0700
Subject: [Python-Dev] Mini-Pep: An Empty String ABC
In-Reply-To: <A8708637239F4FEEA25DE934DBFE639C@RaymondLaptop1>
References: <A8708637239F4FEEA25DE934DBFE639C@RaymondLaptop1>
Message-ID: <ca471dc20806011606vece88b8g679c38f089040440@mail.gmail.com>

This PEP is incomplete without specifying exactly which built-in and
stdlib types should be registered as String instances.

I'm also confused -- the motivation seems mostly "so that you can skip
iterating over it when flattening a nested sequence" but earlier you
rejected my "Atomic" proposal, saying "Earlier in the thread it was
made clear that that atomicity is not an intrinsic property of a type;
instead it varies across applications [...]". Isn't this String
proposal just that by another name?

Finally, I fully expect lots of code writing isinstance(x, String) and
making many more assumptions than promised by the String ABC. For
example that s[0] has the same type as s (not true for bytes). Or that
it is hashable (the Sequence class doesn't define __hash__). Or that
s1+s2 will work (not in the Sequence class either). And many more.

All this makes me lean towards a rejection of this proposal -- it
seems worse than no proposal at all. It could perhaps be rescued by
adding some small set of defined operations.

--Guido

On Sat, May 31, 2008 at 11:59 PM, Raymond Hettinger <python at rcn.com> wrote:
> Mini-Pep:  An Empty String ABC
> Target:  Py2.6 and Py3.0
> Author:  Raymond Hettinger
>
> Proposal
> --------
>
> Add a new collections ABC specified as:
>
>   class String(Sequence):
>       pass
>
> Motivation
> ----------
> Having an ABC for strings allows string look-alike classes to declare
> themselves as sequences that contain text.  Client code (such as a flatten
> operation or tree searching tool) may use that ABC to usefully differentiate
> strings from other sequences (i.e. containers vs containees).  And in code
> that only relies on sequence behavior, isinstance(x,str) may be usefully
> replaced by isinstance(x,String) so that look-alikes can be substituted in
> calling code.
>
> A natural temptation is add other methods to the String ABC, but strings are
> a
> tough case.  Beyond simple sequence manipulation, the string methods get
> very
> complex.  An ABC that included those methods would make it tough to write a
> compliant class that could be registered as a String.  The split(),
> rsplit(),
> partition(), and rpartition() methods are examples of methods that would be
> difficult to emulate correctly.  Also, starting with Py3.0, strings are
> essentially abstract sequences of code points, meaning that an encode()
> method
> is essential to being able to usefully transform them back into concrete
> data.
> Unfortunately, the encode method is so complex that it cannot be readily
> emulated by an aspiring string look-alike.
>
> Besides complexity, another problem with the concrete str API is the
> extensive
> number of methods.  If string look-alikes were required to emulate the likes
> of zfill(), ljust(), title(), translate(), join(), etc., it would
> significantly add to the burden of writing a class complying with the String
> ABC.
>
> The fundamental problem is that of balancing a client function's desire to
> rely on a broad number of behaviors against the difficulty of writing a
> compliant look-alike class.  For other ABCs, the balance is more easily
> struck
> because the behaviors are fewer in number, because they are easier to
> implement correctly, and because some methods can be provided as mixins.
>  For
> a String ABC, the balance should lean toward minimalism due to the large
> number of methods and how difficult it is to implement some of the
> correctly.
>
> A last reason to avoid expanding the String API is that almost none of the
> candidate methods characterize the notion of "stringiness".  With something
> calling itself an integer, an __add__() method would be expected as it is
> fundamental to the notion of "integeriness".  In contrast, methods like
> startswith() and title() are non-essential extras -- we would not discount
> something as being not stringlike if those methods were not present.
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at krypto.org  Mon Jun  2 01:30:54 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 1 Jun 2008 16:30:54 -0700
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <483FBCB4.5020007@egenix.com>
References: <48397ECC.9070805@cheimes.de> <483B2D02.8040400@cheimes.de>
	<483BDE11.509@egenix.com> <483D300B.5090309@egenix.com>
	<52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com>
	<483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de>
	<483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com>
	<483FBCB4.5020007@egenix.com>
Message-ID: <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>

On Fri, May 30, 2008 at 1:37 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 2008-05-30 00:57, Nick Coghlan wrote:
>>
>> M.-A. Lemburg wrote:
>>>
>>> * Why can't we have both PyString *and* PyBytes exposed in 2.x,
>>> with one redirecting to the other ?
>>
>> We do have that - the PyString_* names still work perfectly fine in 2.x.
>> They just won't be used in the Python core codebase anymore - everything in
>> the Python core will use either PyBytes_* or PyUnicode_* regardless of which
>> branch (2.x or 3.x) you're working on. I think that's a good thing for ease
>> of maintenance in the future, even if it takes people a while to get their
>> heads around it right now.
>
> Sorry, I probably wasn't clear enough:
>
> Why can't we have both PyString *and* PyBytes exposed as C
> APIs (ie. visible in code and in the linker) in 2.x, with one redirecting
> to the other ?
>
>>> * Why should the 2.x code base turn to hacks, just because 3.x wants
>>> to restructure itself ?
>>
>> With the better explanation from Greg of what the checked in approach
>> achieves (i.e. preserving exact ABI compatibility for PyString_*, while
>> allowing PyBytes_* to be used at the source code level), I don't see what
>> has been done as being any more of a hack than the possibly more common
>> "#define <oldname> <newname>" (which *would* break binary compatibility).
>>
>> The only things that I think would tidy it up further would be to:
>> - include an explanation of the approach and its effects on API and ABI
>> backward and forward compatibility within 2.x and between 2.x and 3.x in
>> stringobject.h
>> - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0
>
> Which is what I was suggesting all along; sorry if I wasn't
> clear enough on that.
>
> The standard approach is that you provide #define redirects from the
> old APIs to the new ones (which are then picked up by the compiler)
> *and* add function wrappers to the same affect (to make linkers,
> dynamic load APIs such ctypes and debuggers happy).
>
>
> Example from pythonrun.h|c:
> ---------------------------
>
> /* Use macros for a bunch of old variants */
> #define PyRun_String(str, s, g, l) PyRun_StringFlags(str, s, g, l, NULL)
>
> /* Deprecated C API functions still provided for binary compatiblity */
>
> #undef PyRun_String
> PyAPI_FUNC(PyObject *)
> PyRun_String(const char *str, int s, PyObject *g, PyObject *l)
> {
>        return PyRun_StringFlags(str, s, g, l, NULL);
> }
>

Okay, how about this?  http://codereview.appspot.com/1521

Using that patch, both PyString_ and PyBytes_ APIs are available using
function stubs similar to the above.  I opted to define the stub
functions right next to the ones they were stubbing rather than
putting them all at the end of the file or in another file but they
could be moved if someone doesn't like them that way.

> I still believe that we should *not* make "easy of merging" the
> primary motivation for backporting changes in 3.x to 2.x. Software
> design should not be guided by restrictions in the tool chain,
> if not absolutely necessary.
>
> The main argument for a backport needs to be general usefulness
> to the 2.x users, IMHO... just like any other feature that
> makes it into 2.x.
>
> If merging is difficult then this needs to be addressed, but
> there are more options to that than always going back to the
> original 2.x trunk code. I've given a few suggestions on how
> this could be approached in other emails on this thread.

I am not the one doing the merging or working on merge tools so I'll
leave this up to those that are.

-gps

From greg.ewing at canterbury.ac.nz  Mon Jun  2 03:16:03 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 02 Jun 2008 13:16:03 +1200
Subject: [Python-Dev] Iterable String Redux (aka String ABC)
In-Reply-To: <ca471dc20806011543q2cb533a0pf3770cb8cbd9844d@mail.gmail.com>
References: <loom.20080527T192243-415@post.gmane.org>
	<5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1>
	<08May28.143320pdt.58698@synergy1.parc.xerox.com>
	<483DFB8B.5050909@canterbury.ac.nz> <g1lkg5$g4l$1@ger.gmane.org>
	<483F312E.1020302@canterbury.ac.nz> <g1o6dr$psg$1@ger.gmane.org>
	<483FB8F6.4060504@canterbury.ac.nz>
	<loom.20080530T144341-819@post.gmane.org>
	<48427FD7.4090005@canterbury.ac.nz>
	<ca471dc20806011543q2cb533a0pf3770cb8cbd9844d@mail.gmail.com>
Message-ID: <484349D3.2050103@canterbury.ac.nz>

Guido van Rossum wrote:
> There are quite a few core APIs that accept no
> substitutes, and being an instance of basestring was intended to
> guarantee that a value is accepted by such APIs.

In that case, the idea of a user-defined string class
that doesn't inherit from str or unicode seems to be
a lost cause, since it will never be acceptable in
those places, whatever is done with ABCs.

-- 
Greg

From ncoghlan at gmail.com  Mon Jun  2 12:47:08 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 02 Jun 2008 20:47:08 +1000
Subject: [Python-Dev] Mini-Pep: An Empty String ABC
In-Reply-To: <ca471dc20806011606vece88b8g679c38f089040440@mail.gmail.com>
References: <A8708637239F4FEEA25DE934DBFE639C@RaymondLaptop1>
	<ca471dc20806011606vece88b8g679c38f089040440@mail.gmail.com>
Message-ID: <4843CFAC.8070503@gmail.com>

Guido van Rossum wrote:
> This PEP is incomplete without specifying exactly which built-in and
> stdlib types should be registered as String instances.
> 
> I'm also confused -- the motivation seems mostly "so that you can skip
> iterating over it when flattening a nested sequence" but earlier you
> rejected my "Atomic" proposal, saying "Earlier in the thread it was
> made clear that that atomicity is not an intrinsic property of a type;
> instead it varies across applications [...]". Isn't this String
> proposal just that by another name?
> 
> Finally, I fully expect lots of code writing isinstance(x, String) and
> making many more assumptions than promised by the String ABC. For
> example that s[0] has the same type as s (not true for bytes). Or that
> it is hashable (the Sequence class doesn't define __hash__). Or that
> s1+s2 will work (not in the Sequence class either). And many more.

I think the PEP also needs to explain why having multiple small one-off 
string ABCs is a bad thing. The whole point of providing a standard ABC 
mechanism is to enable exactly that: allowing a library to say "Here is 
my concept of what a string class needs to provide - register with this 
ABC to tell me that I can use your class without blowing up 
unexpectedly". The library can then preregister a bunch of other classes 
it knows about that do the right thing (such as the builtin str type)

That is, to write a flatten operation with ABC's you might do something 
along the lines of:

from abc import ABCMeta

class Atomic(metaclass=ABCMeta):
   """ABC for iterables that the flatten function will not expand"""

Atomic.register(str) # Consider builtin strings to be atomic

def flatten(obj, atomic=Atomic):
   itr = None
   if not isinstance(obj, atomic):
     try:
       itr = iter(obj)
     except (TypeError, AttributeError):
       pass
   if itr is not None:
     for item in itr:
       for x in flatten(item):
         yield x
   else:
     yield obj

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From regebro at gmail.com  Mon Jun  2 16:52:58 2008
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 2 Jun 2008 16:52:58 +0200
Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?
In-Reply-To: <4838EA2D.7060402@jcea.es>
References: <4838EA2D.7060402@jcea.es>
Message-ID: <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com>

On Sun, May 25, 2008 at 6:25 AM, Jesus Cea <jcea at jcea.es> wrote:
> Since I need to port bsddb3 to py3k, what I need to know?. Is any
> *updated* document out there?.

No, but there is a not yet complete, but quite updated set of examples.

http://code.google.com/p/python-incompatibility/

This is a collection of code snippets that will show code that does
work under 2.x but not under 3.x, and the 3.x way of doing it (as well
as a way that works under both 2.6 and 3.0, in applicable cases).

There is no tests for changes in the C-API, if you (or somebody else)
would like to add that (or any other tests) that would be very
appreciated!

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64

From skip at pobox.com  Mon Jun  2 14:09:14 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 2 Jun 2008 07:09:14 -0500
Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module)
In-Reply-To: <4843B971.6080707@gmail.com>
References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com>
	<978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com>
	<4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com>
	<g1osss$5nq$1@ger.gmane.org> <48402B50.6040809@gmail.com>
	<978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com>
	<79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>
	<ac4216f0805311531h624ad56ei62dc60141783b3e@mail.gmail.com>
	<79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com>
	<4842B1F2.6010901@gmail.com>
	<18499.18335.638383.979060@montanaro-dyndns-org.local>
	<4843B971.6080707@gmail.com>
Message-ID: <18499.58090.240170.352835@montanaro-dyndns-org.local>


    >> If the 3.0 API of a module is going to involve breakage which
    >> requires authors to update their applications wouldn't this be a good
    >> time to PEP-8-ify the module?  (Not suggesting that threading would
    >> fall into this category.)

    Nick> Updating application code to deal with a module name change is
    Nick> easy - just use an "import x as y" statement. Catching the
    Nick> ImportError for the 3.0 name and falling back to the 2.6 name (or
    Nick> vice-versa) even makes it possible to support both names fairly
    Nick> easily.

    Nick> Changing the names of actual objects within the modules is tougher
    Nick> though - there are many more ways to access those.

I think you misunderstood what I wrote.  Suppose you decided that the API
for the threading modules needs significant rework, changes which will break
much, if not all current usage.  I'm only suggesting that if you decide the
API change is worthwhile that you take the opportunity to make the class,
method and data names PEP8-compliant.  I'm not talking about gratuitous
change to the identifier names, but situations where the user is going to
have to rework their code anyway.

Skip


From jnoller at gmail.com  Mon Jun  2 17:11:35 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 2 Jun 2008 11:11:35 -0400
Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module)
In-Reply-To: <ac4216f0805311531h624ad56ei62dc60141783b3e@mail.gmail.com>
References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com>
	<978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com>
	<4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com>
	<g1osss$5nq$1@ger.gmane.org> <48402B50.6040809@gmail.com>
	<978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com>
	<79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>
	<ac4216f0805311531h624ad56ei62dc60141783b3e@mail.gmail.com>
Message-ID: <4222a8490806020811u7ea7d7d5ye66b39bee54b1737@mail.gmail.com>

On Sat, May 31, 2008 at 6:31 PM, r.m.oudkerk <r.m.oudkerk at googlemail.com> wrote:
> On 31/05/2008, Paul Moore <p.f.moore at gmail.com> wrote:
>> 2008/5/30 Farshid Lashkari <flashk at gmail.com>:
>>> I'm not sure if there will be any side affects to modifying
>>> sys.executable though. Should this be the official way of supporting
>>> embedded interpreters or should there be a
>>> multiprocessing.setExecutable() method?
>>
>> +1 for setExecutable (I'd prefer set_executable, to be PEP 8
>> compliant). Make it explicit, rather than fiddling with stuff in sys
>> manually.
>
> That is easy to do.

Also note - someone just pointed out to me that the executable
manipulation as-is breaks when you execute things within IDLE.

I'll add all of this to the PEP.

-jesse

From ncoghlan at gmail.com  Mon Jun  2 11:12:17 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 02 Jun 2008 19:12:17 +1000
Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module)
In-Reply-To: <18499.18335.638383.979060@montanaro-dyndns-org.local>
References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com>
	<978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com>
	<4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com>
	<g1osss$5nq$1@ger.gmane.org> <48402B50.6040809@gmail.com>
	<978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com>
	<79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>
	<ac4216f0805311531h624ad56ei62dc60141783b3e@mail.gmail.com>
	<79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com>
	<4842B1F2.6010901@gmail.com>
	<18499.18335.638383.979060@montanaro-dyndns-org.local>
Message-ID: <4843B971.6080707@gmail.com>

skip at pobox.com wrote:
>     Nick> We fixed the module names that used mixed case - the amount of
>     Nick> work that turned out to be involved in just doing that much for
>     Nick> PEP 3108 makes me shudder at the thought of trying to fix all of
>     Nick> the standard library APIs that currently don't follow the style
>     Nick> guide...
> 
> If the 3.0 API of a module is going to involve breakage which requires
> authors to update their applications wouldn't this be a good time to
> PEP-8-ify the module?  (Not suggesting that threading would fall into this
> category.)

Updating application code to deal with a module name change is easy - 
just use an "import x as y" statement. Catching the ImportError for the 
3.0 name and falling back to the 2.6 name (or vice-versa) even makes it 
possible to support both names fairly easily.

Changing the names of actual objects within the modules is tougher 
though - there are many more ways to access those.

Cheers,
Nick

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From python at rcn.com  Mon Jun  2 22:54:08 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Jun 2008 13:54:08 -0700
Subject: [Python-Dev] Mini-Pep: An Empty String ABC
References: <A8708637239F4FEEA25DE934DBFE639C@RaymondLaptop1>
	<ca471dc20806011606vece88b8g679c38f089040440@mail.gmail.com>
Message-ID: <07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1>

From: "Guido van Rossum" <guido at python.org>
> All this makes me lean towards a rejection of this proposal -- it
> seems worse than no proposal at all. It could perhaps be rescued by
> adding some small set of defined operations.

By subclassing Sequence, we get index() and count() mixins for free.

We can also add other mixin freebies like __hash__(), __eq__(), __ne__(), endswith(), startswith(), find(), rfind(), and rindex().

It's tempting to add center, lust, rjust, and zfill, but those require some sort of constructor that accepts an iterable argument.

As important as what is included are the methods intentionally left out.   I'm trying to avoid insisting on abstractmethods like 
encode(), split(), join(), and other methods that place an undue burden on a class being registered as a String.


Raymond






From guido at python.org  Mon Jun  2 23:23:15 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Jun 2008 14:23:15 -0700
Subject: [Python-Dev] Mini-Pep: An Empty String ABC
In-Reply-To: <07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1>
References: <A8708637239F4FEEA25DE934DBFE639C@RaymondLaptop1>
	<ca471dc20806011606vece88b8g679c38f089040440@mail.gmail.com>
	<07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1>
Message-ID: <ca471dc20806021423o132a39d2kdb49822f107b5636@mail.gmail.com>

Please try to find the largest set of methods that you're comfortable
with. __add__ comes to mind.

Note that if you add __hash__, this rules out bytearray -- is that
your intention? __hash__ is intentionally not part of the "read-only"
ABCs because read-only doesn't mean immutable.

Also, (again) please list which built-in types you want to register.

On Mon, Jun 2, 2008 at 1:54 PM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Guido van Rossum" <guido at python.org>
>>
>> All this makes me lean towards a rejection of this proposal -- it
>> seems worse than no proposal at all. It could perhaps be rescued by
>> adding some small set of defined operations.
>
> By subclassing Sequence, we get index() and count() mixins for free.
>
> We can also add other mixin freebies like __hash__(), __eq__(), __ne__(),
> endswith(), startswith(), find(), rfind(), and rindex().
>
> It's tempting to add center, lust, rjust, and zfill, but those require some
> sort of constructor that accepts an iterable argument.
>
> As important as what is included are the methods intentionally left out.
> I'm trying to avoid insisting on abstractmethods like encode(), split(),
> join(), and other methods that place an undue burden on a class being
> registered as a String.
>
>
> Raymond
>
>
>
>
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mal at egenix.com  Mon Jun  2 14:33:08 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 02 Jun 2008 14:33:08 +0200
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
References: <48397ECC.9070805@cheimes.de>
	<483B2D02.8040400@cheimes.de>	<483BDE11.509@egenix.com>
	<483D300B.5090309@egenix.com>	<52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com>	<483ECA52.6040000@egenix.com>
	<483ECF94.7060607@cheimes.de>	<483EF139.8000606@egenix.com>
	<483F34C3.3050402@gmail.com>	<483FBCB4.5020007@egenix.com>
	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
Message-ID: <4843E884.1060705@egenix.com>

On 2008-06-02 01:30, Gregory P. Smith wrote:
> On Fri, May 30, 2008 at 1:37 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Sorry, I probably wasn't clear enough:
>>
>> Why can't we have both PyString *and* PyBytes exposed as C
>> APIs (ie. visible in code and in the linker) in 2.x, with one redirecting
>> to the other ?
>>
>>>> * Why should the 2.x code base turn to hacks, just because 3.x wants
>>>> to restructure itself ?
>>> With the better explanation from Greg of what the checked in approach
>>> achieves (i.e. preserving exact ABI compatibility for PyString_*, while
>>> allowing PyBytes_* to be used at the source code level), I don't see what
>>> has been done as being any more of a hack than the possibly more common
>>> "#define <oldname> <newname>" (which *would* break binary compatibility).
>>>
>>> The only things that I think would tidy it up further would be to:
>>> - include an explanation of the approach and its effects on API and ABI
>>> backward and forward compatibility within 2.x and between 2.x and 3.x in
>>> stringobject.h
>>> - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0
>> Which is what I was suggesting all along; sorry if I wasn't
>> clear enough on that.
>>
>> The standard approach is that you provide #define redirects from the
>> old APIs to the new ones (which are then picked up by the compiler)
>> *and* add function wrappers to the same affect (to make linkers,
>> dynamic load APIs such ctypes and debuggers happy).
>>
>>
>> Example from pythonrun.h|c:
>> ---------------------------
>>
>> /* Use macros for a bunch of old variants */
>> #define PyRun_String(str, s, g, l) PyRun_StringFlags(str, s, g, l, NULL)
>>
>> /* Deprecated C API functions still provided for binary compatiblity */
>>
>> #undef PyRun_String
>> PyAPI_FUNC(PyObject *)
>> PyRun_String(const char *str, int s, PyObject *g, PyObject *l)
>> {
>>        return PyRun_StringFlags(str, s, g, l, NULL);
>> }
>>
> 
> Okay, how about this?  http://codereview.appspot.com/1521
> 
> Using that patch, both PyString_ and PyBytes_ APIs are available using
> function stubs similar to the above.  I opted to define the stub
> functions right next to the ones they were stubbing rather than
> putting them all at the end of the file or in another file but they
> could be moved if someone doesn't like them that way.

Thanks. I was working on a similar patch. Looks like you beat
me to it.

The only thing I'm not sure about is having the wrappers in the
same file - this is likely to cause merge conflicts when doing
direct merging and even with an automated renaming approach,
the extra code would be easier to remove if it were e.g. at
the end of the file or even better: in a separate file.

My patch worked slightly differently: it adds wrappers PyString*
that forward calls to the PyBytes* APIs and they all live in
stringobject.c. stringobject.h then also provides aliases
so that recompiled extensions pick up the new API names.

While working on my patch I ran into an issue that I haven't
been able to resolve: the wrapper functions got optimized away
by the linker and even though they appear in the libpython2.6.a,
they don't end up in the python binary itself.

As a result, importing Python 2.5 in the resulting 2.6
binary still fails with a unresolved PyString symbol.

Please check whether that's the case for your patch as well.

>> I still believe that we should *not* make "easy of merging" the
>> primary motivation for backporting changes in 3.x to 2.x. Software
>> design should not be guided by restrictions in the tool chain,
>> if not absolutely necessary.
>>
>> The main argument for a backport needs to be general usefulness
>> to the 2.x users, IMHO... just like any other feature that
>> makes it into 2.x.
>>
>> If merging is difficult then this needs to be addressed, but
>> there are more options to that than always going back to the
>> original 2.x trunk code. I've given a few suggestions on how
>> this could be approached in other emails on this thread.
> 
> I am not the one doing the merging or working on merge tools so I'll
> leave this up to those that are.

I'm not sure whether there are any specific merge tools around -
apart from the 2to3.py script.

There also doesn't seem to be any documentation on the merge
process itself (at least nothing that Google can find in the
PEPs), so it's difficult to make any suggestions.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 02 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            34 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From dalcinl at gmail.com  Mon Jun  2 18:05:07 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 2 Jun 2008 13:05:07 -0300
Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module)
In-Reply-To: <00ac01c8c386$e10c68e0$a3253aa0$@com.au>
References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com>
	<978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com>
	<4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com>
	<g1osss$5nq$1@ger.gmane.org> <48402B50.6040809@gmail.com>
	<978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com>
	<79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>
	<00ac01c8c386$e10c68e0$a3253aa0$@com.au>
Message-ID: <e7ba66e40806020905i9c7a847n34a7527f4628d2e2@mail.gmail.com>

On 5/31/08, Mark Hammond <mhammond at skippinet.com.au> wrote:
>  So it seems that maybe simply "setExecutable()" isn't the correct
>  abstraction here, but maybe a "factory" approach, so the entire process
>  creation mechanism can be replaced rather than just the name of the
>  executable to spawn?

Indeed. If the spawn mechanisms (and even the connection mechanisms)
were fully abstracted, then I believe that extending pyProcessing to
work in clusters with something like MPI would be far easier. But
perhaps I'm just dreaming...


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From skip at pobox.com  Mon Jun  2 03:06:39 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 1 Jun 2008 20:06:39 -0500
Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module)
In-Reply-To: <4842B1F2.6010901@gmail.com>
References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com>
	<978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com>
	<4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com>
	<g1osss$5nq$1@ger.gmane.org> <48402B50.6040809@gmail.com>
	<978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com>
	<79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com>
	<ac4216f0805311531h624ad56ei62dc60141783b3e@mail.gmail.com>
	<79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com>
	<4842B1F2.6010901@gmail.com>
Message-ID: <18499.18335.638383.979060@montanaro-dyndns-org.local>


    Nick> We fixed the module names that used mixed case - the amount of
    Nick> work that turned out to be involved in just doing that much for
    Nick> PEP 3108 makes me shudder at the thought of trying to fix all of
    Nick> the standard library APIs that currently don't follow the style
    Nick> guide...

If the 3.0 API of a module is going to involve breakage which requires
authors to update their applications wouldn't this be a good time to
PEP-8-ify the module?  (Not suggesting that threading would fall into this
category.)

Skip

From dalcinl at gmail.com  Mon Jun  2 18:17:47 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 2 Jun 2008 13:17:47 -0300
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
References: <48397ECC.9070805@cheimes.de> <483BDE11.509@egenix.com>
	<483D300B.5090309@egenix.com>
	<52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com>
	<483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de>
	<483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com>
	<483FBCB4.5020007@egenix.com>
	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
Message-ID: <e7ba66e40806020917i7152bfe6l6cb1a7b48507d3a5@mail.gmail.com>

Are you completelly sure of adding those guys:  PyBytes_InternXXX ???


On 6/1/08, Gregory P. Smith <greg at krypto.org> wrote:
> On Fri, May 30, 2008 at 1:37 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>  > On 2008-05-30 00:57, Nick Coghlan wrote:
>  >>
>  >> M.-A. Lemburg wrote:
>  >>>
>  >>> * Why can't we have both PyString *and* PyBytes exposed in 2.x,
>  >>> with one redirecting to the other ?
>  >>
>  >> We do have that - the PyString_* names still work perfectly fine in 2.x.
>  >> They just won't be used in the Python core codebase anymore - everything in
>  >> the Python core will use either PyBytes_* or PyUnicode_* regardless of which
>  >> branch (2.x or 3.x) you're working on. I think that's a good thing for ease
>  >> of maintenance in the future, even if it takes people a while to get their
>  >> heads around it right now.
>  >
>  > Sorry, I probably wasn't clear enough:
>  >
>  > Why can't we have both PyString *and* PyBytes exposed as C
>  > APIs (ie. visible in code and in the linker) in 2.x, with one redirecting
>  > to the other ?
>  >
>  >>> * Why should the 2.x code base turn to hacks, just because 3.x wants
>  >>> to restructure itself ?
>  >>
>  >> With the better explanation from Greg of what the checked in approach
>  >> achieves (i.e. preserving exact ABI compatibility for PyString_*, while
>  >> allowing PyBytes_* to be used at the source code level), I don't see what
>  >> has been done as being any more of a hack than the possibly more common
>  >> "#define <oldname> <newname>" (which *would* break binary compatibility).
>  >>
>  >> The only things that I think would tidy it up further would be to:
>  >> - include an explanation of the approach and its effects on API and ABI
>  >> backward and forward compatibility within 2.x and between 2.x and 3.x in
>  >> stringobject.h
>  >> - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0
>  >
>  > Which is what I was suggesting all along; sorry if I wasn't
>  > clear enough on that.
>  >
>  > The standard approach is that you provide #define redirects from the
>  > old APIs to the new ones (which are then picked up by the compiler)
>  > *and* add function wrappers to the same affect (to make linkers,
>  > dynamic load APIs such ctypes and debuggers happy).
>  >
>  >
>  > Example from pythonrun.h|c:
>  > ---------------------------
>  >
>  > /* Use macros for a bunch of old variants */
>  > #define PyRun_String(str, s, g, l) PyRun_StringFlags(str, s, g, l, NULL)
>  >
>  > /* Deprecated C API functions still provided for binary compatiblity */
>  >
>  > #undef PyRun_String
>  > PyAPI_FUNC(PyObject *)
>  > PyRun_String(const char *str, int s, PyObject *g, PyObject *l)
>  > {
>  >        return PyRun_StringFlags(str, s, g, l, NULL);
>  > }
>  >
>
>
> Okay, how about this?  http://codereview.appspot.com/1521
>
>  Using that patch, both PyString_ and PyBytes_ APIs are available using
>  function stubs similar to the above.  I opted to define the stub
>  functions right next to the ones they were stubbing rather than
>  putting them all at the end of the file or in another file but they
>  could be moved if someone doesn't like them that way.
>
>
>  > I still believe that we should *not* make "easy of merging" the
>  > primary motivation for backporting changes in 3.x to 2.x. Software
>  > design should not be guided by restrictions in the tool chain,
>  > if not absolutely necessary.
>  >
>  > The main argument for a backport needs to be general usefulness
>  > to the 2.x users, IMHO... just like any other feature that
>  > makes it into 2.x.
>  >
>  > If merging is difficult then this needs to be addressed, but
>  > there are more options to that than always going back to the
>  > original 2.x trunk code. I've given a few suggestions on how
>  > this could be approached in other emails on this thread.
>
>
> I am not the one doing the merging or working on merge tools so I'll
>  leave this up to those that are.
>
>  -gps
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com
>


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From greg at krypto.org  Tue Jun  3 00:21:18 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 2 Jun 2008 15:21:18 -0700
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <4843E884.1060705@egenix.com>
References: <48397ECC.9070805@cheimes.de> <483D300B.5090309@egenix.com>
	<52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com>
	<483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de>
	<483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com>
	<483FBCB4.5020007@egenix.com>
	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
	<4843E884.1060705@egenix.com>
Message-ID: <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>

On Mon, Jun 2, 2008 at 5:33 AM, M.-A. Lemburg <mal at egenix.com> wrote:

>
>> Okay, how about this?  http://codereview.appspot.com/1521
>>
>> Using that patch, both PyString_ and PyBytes_ APIs are available using
>> function stubs similar to the above.  I opted to define the stub
>> functions right next to the ones they were stubbing rather than
>> putting them all at the end of the file or in another file but they
>> could be moved if someone doesn't like them that way.
>>
>
> Thanks. I was working on a similar patch. Looks like you beat
> me to it.
>
> The only thing I'm not sure about is having the wrappers in the
> same file - this is likely to cause merge conflicts when doing
> direct merging and even with an automated renaming approach,
> the extra code would be easier to remove if it were e.g. at
> the end of the file or even better: in a separate file.
>
> My patch worked slightly differently: it adds wrappers PyString*
> that forward calls to the PyBytes* APIs and they all live in
> stringobject.c. stringobject.h then also provides aliases
> so that recompiled extensions pick up the new API names.
>
> While working on my patch I ran into an issue that I haven't
> been able to resolve: the wrapper functions got optimized away
> by the linker and even though they appear in the libpython2.6.a,
> they don't end up in the python binary itself.
>
> As a result, importing Python 2.5 in the resulting 2.6
> binary still fails with a unresolved PyString symbol.
>
> Please check whether that's the case for your patch as well.


I think that is going to happen no matter which approach is used (yours or
mine) unless we force some included code to call each of the stubs
(needlessly inefficient).  One way to do that is to reference them all from
a section of code called conditionally based upon an always false condition
that the compiler and linker can never predetermine is false so that it
cannot be eliminated as dead code.

Given that, should we bother?  I don't think we really need PyBytes_ to show
up in the binary ABI for 2.x even if that is how we write the calls in the
python internals code.  The arguments put forth that debugging is easier if
you can just set a breakpoint on what you read may be true but including
stub functions doesn't help this when most of the time they're compiled
under the alternate name using #defines so a breakpoint set on the stub name
will not actually trigger.

API wise we're really providing the PyBytes* names to make module author's
work of writing code that targets 2.6 and 3.x easier but isn't it reasonable
for authors to just be told that they're just #defined aliases for
PyString*.  There is no goal, nor should there be, of a module binary
compiled against 2.x loading and working in 3.x.

I expect most module authors, code generators and such will want to target
Python 2.x earlier than 2.6 as well so should we provide PyBytes_ names as a
public API in 2.6 at all?  (regardless of if we use the PyBytes names
internally for any reason)

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080602/31760a87/attachment.htm>

From greg at krypto.org  Tue Jun  3 00:22:51 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 2 Jun 2008 15:22:51 -0700
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <e7ba66e40806020917i7152bfe6l6cb1a7b48507d3a5@mail.gmail.com>
References: <48397ECC.9070805@cheimes.de> <483D300B.5090309@egenix.com>
	<52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com>
	<483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de>
	<483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com>
	<483FBCB4.5020007@egenix.com>
	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
	<e7ba66e40806020917i7152bfe6l6cb1a7b48507d3a5@mail.gmail.com>
Message-ID: <52dc1c820806021522u77e94406q7575a34bcaee79c1@mail.gmail.com>

-cc: python-3000

I believe those APIs are already there in the existing interface.  Why does
that concern you?

On Mon, Jun 2, 2008 at 9:17 AM, Lisandro Dalcin <dalcinl at gmail.com> wrote:

> Are you completelly sure of adding those guys:  PyBytes_InternXXX ???
>
>
> On 6/1/08, Gregory P. Smith <greg at krypto.org> wrote:
> > On Fri, May 30, 2008 at 1:37 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> >  > On 2008-05-30 00:57, Nick Coghlan wrote:
> >  >>
> >  >> M.-A. Lemburg wrote:
> >  >>>
> >  >>> * Why can't we have both PyString *and* PyBytes exposed in 2.x,
> >  >>> with one redirecting to the other ?
> >  >>
> >  >> We do have that - the PyString_* names still work perfectly fine in
> 2.x.
> >  >> They just won't be used in the Python core codebase anymore -
> everything in
> >  >> the Python core will use either PyBytes_* or PyUnicode_* regardless
> of which
> >  >> branch (2.x or 3.x) you're working on. I think that's a good thing
> for ease
> >  >> of maintenance in the future, even if it takes people a while to get
> their
> >  >> heads around it right now.
> >  >
> >  > Sorry, I probably wasn't clear enough:
> >  >
> >  > Why can't we have both PyString *and* PyBytes exposed as C
> >  > APIs (ie. visible in code and in the linker) in 2.x, with one
> redirecting
> >  > to the other ?
> >  >
> >  >>> * Why should the 2.x code base turn to hacks, just because 3.x wants
> >  >>> to restructure itself ?
> >  >>
> >  >> With the better explanation from Greg of what the checked in approach
> >  >> achieves (i.e. preserving exact ABI compatibility for PyString_*,
> while
> >  >> allowing PyBytes_* to be used at the source code level), I don't see
> what
> >  >> has been done as being any more of a hack than the possibly more
> common
> >  >> "#define <oldname> <newname>" (which *would* break binary
> compatibility).
> >  >>
> >  >> The only things that I think would tidy it up further would be to:
> >  >> - include an explanation of the approach and its effects on API and
> ABI
> >  >> backward and forward compatibility within 2.x and between 2.x and 3.x
> in
> >  >> stringobject.h
> >  >> - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0
> >  >
> >  > Which is what I was suggesting all along; sorry if I wasn't
> >  > clear enough on that.
> >  >
> >  > The standard approach is that you provide #define redirects from the
> >  > old APIs to the new ones (which are then picked up by the compiler)
> >  > *and* add function wrappers to the same affect (to make linkers,
> >  > dynamic load APIs such ctypes and debuggers happy).
> >  >
> >  >
> >  > Example from pythonrun.h|c:
> >  > ---------------------------
> >  >
> >  > /* Use macros for a bunch of old variants */
> >  > #define PyRun_String(str, s, g, l) PyRun_StringFlags(str, s, g, l,
> NULL)
> >  >
> >  > /* Deprecated C API functions still provided for binary compatiblity
> */
> >  >
> >  > #undef PyRun_String
> >  > PyAPI_FUNC(PyObject *)
> >  > PyRun_String(const char *str, int s, PyObject *g, PyObject *l)
> >  > {
> >  >        return PyRun_StringFlags(str, s, g, l, NULL);
> >  > }
> >  >
> >
> >
> > Okay, how about this?  http://codereview.appspot.com/1521
> >
> >  Using that patch, both PyString_ and PyBytes_ APIs are available using
> >  function stubs similar to the above.  I opted to define the stub
> >  functions right next to the ones they were stubbing rather than
> >  putting them all at the end of the file or in another file but they
> >  could be moved if someone doesn't like them that way.
> >
> >
> >  > I still believe that we should *not* make "easy of merging" the
> >  > primary motivation for backporting changes in 3.x to 2.x. Software
> >  > design should not be guided by restrictions in the tool chain,
> >  > if not absolutely necessary.
> >  >
> >  > The main argument for a backport needs to be general usefulness
> >  > to the 2.x users, IMHO... just like any other feature that
> >  > makes it into 2.x.
> >  >
> >  > If merging is difficult then this needs to be addressed, but
> >  > there are more options to that than always going back to the
> >  > original 2.x trunk code. I've given a few suggestions on how
> >  > this could be approached in other emails on this thread.
> >
> >
> > I am not the one doing the merging or working on merge tools so I'll
> >  leave this up to those that are.
> >
> >  -gps
> >  _______________________________________________
> >  Python-Dev mailing list
> >  Python-Dev at python.org
> >  http://mail.python.org/mailman/listinfo/python-dev
> >  Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com
> >
>
>
> --
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080602/49dc6f56/attachment-0001.htm>

From barry at python.org  Tue Jun  3 00:51:46 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 2 Jun 2008 18:51:46 -0400
Subject: [Python-Dev] Postponing the first betas
Message-ID: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

We are going to postpone the first beta releases by one week.  We had  
some problems with mail.python.org today, which prompted a query to  
Guido from me about the postponement.  mail.python.org should now be  
back up normally now, as evidenced by the emailfloodl but in the  
meantime, Guido said:

"I'd also like to see PEP 3138 (CJK-friendly repr()) and the
pyprocessing PEP implemented by then, and perhaps some other small
stuff."

So we're going to do the first beta releases next Wednesday, June 11.   
Please take this time to stabilize all APIs and features, both in  
Python and C.  Next week, I'll do a gut check on critical and release  
blocker bugs, so please also take a look at those and try to fix what  
you can.

Cheers,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSER5gnEjvBPtnXfVAQJlBAQAgfmRwGQzwNFrwvMusIoDNVRuyIObkKO0
FeDYb26RAL1jLXt0x/7jE0fBc5FvhDzUJnnNj3sydfyKU5MCb0eB0VeBTmjHU05l
yncX6zYSoU14OUW+bkG4y7vf+aLD9zlFsj/ybMEZTQh0RMpZ+HBNhup3NJFEDTBM
97q4SIvltAg=
=NBRW
-----END PGP SIGNATURE-----

From guido at python.org  Tue Jun  3 01:09:18 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Jun 2008 16:09:18 -0700
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>
References: <48397ECC.9070805@cheimes.de>
	<52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com>
	<483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de>
	<483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com>
	<483FBCB4.5020007@egenix.com>
	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
	<4843E884.1060705@egenix.com>
	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>
Message-ID: <ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>

I will freely admit that I haven't followed this thread in any detail,
but if it were up to me, I'd have the 2.6 internal code use PyString
(as both what the linker sees and what the human reads in the source
code) and the 3.0 code use PyBytes for the same thing. Let the merges
be damed -- most changes to 2.6 these days seem to be blocked
explicitly from being merged anyway. I'd prefer the 2.6 code base to
stay true to 2.x, and the 3.0 code base start afresh where it makes
sense. We should reindent more of the 3.0 code base to use
4-space-indents in C code too.

I would also add macros that map the PyBytes_* APIs to PyString_*, but
I would not start using these internally except in code newly written
for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString
for 8-bit strings containing text, and PyBytes for 8-bit strings
containing binary data. For 8-bit strings that could contain either
text or data, I'd use PyString, in the spirit of 2.x.

--Guido

On Mon, Jun 2, 2008 at 3:21 PM, Gregory P. Smith <greg at krypto.org> wrote:
>
>
> On Mon, Jun 2, 2008 at 5:33 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>>
>>> Okay, how about this?  http://codereview.appspot.com/1521
>>>
>>> Using that patch, both PyString_ and PyBytes_ APIs are available using
>>> function stubs similar to the above.  I opted to define the stub
>>> functions right next to the ones they were stubbing rather than
>>> putting them all at the end of the file or in another file but they
>>> could be moved if someone doesn't like them that way.
>>
>> Thanks. I was working on a similar patch. Looks like you beat
>> me to it.
>>
>> The only thing I'm not sure about is having the wrappers in the
>> same file - this is likely to cause merge conflicts when doing
>> direct merging and even with an automated renaming approach,
>> the extra code would be easier to remove if it were e.g. at
>> the end of the file or even better: in a separate file.
>>
>> My patch worked slightly differently: it adds wrappers PyString*
>> that forward calls to the PyBytes* APIs and they all live in
>> stringobject.c. stringobject.h then also provides aliases
>> so that recompiled extensions pick up the new API names.
>>
>> While working on my patch I ran into an issue that I haven't
>> been able to resolve: the wrapper functions got optimized away
>> by the linker and even though they appear in the libpython2.6.a,
>> they don't end up in the python binary itself.
>>
>> As a result, importing Python 2.5 in the resulting 2.6
>> binary still fails with a unresolved PyString symbol.
>>
>> Please check whether that's the case for your patch as well.
>
> I think that is going to happen no matter which approach is used (yours or
> mine) unless we force some included code to call each of the stubs
> (needlessly inefficient).  One way to do that is to reference them all from
> a section of code called conditionally based upon an always false condition
> that the compiler and linker can never predetermine is false so that it
> cannot be eliminated as dead code.
>
> Given that, should we bother?  I don't think we really need PyBytes_ to show
> up in the binary ABI for 2.x even if that is how we write the calls in the
> python internals code.  The arguments put forth that debugging is easier if
> you can just set a breakpoint on what you read may be true but including
> stub functions doesn't help this when most of the time they're compiled
> under the alternate name using #defines so a breakpoint set on the stub name
> will not actually trigger.
>
> API wise we're really providing the PyBytes* names to make module author's
> work of writing code that targets 2.6 and 3.x easier but isn't it reasonable
> for authors to just be told that they're just #defined aliases for
> PyString*.  There is no goal, nor should there be, of a module binary
> compiled against 2.x loading and working in 3.x.
>
> I expect most module authors, code generators and such will want to target
> Python 2.x earlier than 2.6 as well so should we provide PyBytes_ names as a
> public API in 2.6 at all?  (regardless of if we use the PyBytes names
> internally for any reason)
>
> -gps
>
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at krypto.org  Tue Jun  3 01:29:16 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 2 Jun 2008 16:29:16 -0700
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>
References: <48397ECC.9070805@cheimes.de> <483ECA52.6040000@egenix.com>
	<483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com>
	<483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com>
	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
	<4843E884.1060705@egenix.com>
	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>
	<ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>
Message-ID: <52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com>

On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum <guido at python.org> wrote:

> I will freely admit that I haven't followed this thread in any detail,
> but if it were up to me, I'd have the 2.6 internal code use PyString

...

Should we read this as a BDFL pronouncement and make it so?

All that would mean change wise is that trunk r63675 as well as possibly
r63672 and r63677 would need to be rolled back and this whole discussion
over if such a big change should have happened would turn into a moot point.

I would also add macros that map the PyBytes_* APIs to PyString_*, but
> I would not start using these internally except in code newly written
> for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString
> for 8-bit strings containing text, and PyBytes for 8-bit strings
> containing binary data. For 8-bit strings that could contain either
> text or data, I'd use PyString, in the spirit of 2.x.
>
> --Guido
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080602/f9efc82e/attachment.htm>

From techtonik at gmail.com  Mon Jun  2 08:59:11 2008
From: techtonik at gmail.com (techtonik)
Date: Mon, 2 Jun 2008 08:59:11 +0200
Subject: [Python-Dev] Issue2889: Add Curses for Windows as native module
	for 2.6
In-Reply-To: <79990c6b0805240615l65975730i9a8d289e2315eada@mail.gmail.com>
References: <d34314100805240023g49181bbk61b68c9874bc3b3f@mail.gmail.com>
	<4837E463.3030805@v.loewis.de>
	<d34314100805240409q259a9083y3bf4b74e2d052352@mail.gmail.com>
	<79990c6b0805240615l65975730i9a8d289e2315eada@mail.gmail.com>
Message-ID: <d34314100806012359n25d7882ax786d1d6dacd5ec1b@mail.gmail.com>

On Sat, May 24, 2008 at 3:15 PM, Paul Moore <p.f.moore at gmail.com> wrote:

>
> > As for PDCurses library itself there is a Makefile in PDCurses
> distribution
> > for Microsoft Visual C++ 2.0+ named vcwin32.mak  I can't afford buying
> > Visual Studio to test if it works with newer versions, but logically
> Visual
> > Studio should be able to convert Makefile to a newer format.
>
> Visual C++ 9.0 Express Edition builds Python quite happily these days.
> So you can certainly do the integration without buying anything. If
> you get stuck on technical details, there are people here who would
> happily give you advice. Windows developers are always welcome!


Finally, I've downloaded Microsoft Visual Studio Express 2008, but it
requires Windows XP to install and I have only Windows2000 at hand. I will
try to get MS Visual Studio Express 2005 and see if it works.


> If you don't have the knowledge needed, and can't spare the time to
> learn (which is entirely acceptable) then you are indeed relying on
> another Windows developer to pick this up. You may be out of luck
> there - nobody has been interested enough to do this before now (it's
> not as if PDCurses is new) so there's not much reason to expect things
> to have changed.
>
> I'll try to spare the time to learn how to include curses support in
windows, but I will appreciate if anybody could review the patches and
convert project to MSVC 2008 in the end.

-- 
--anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080602/eb55f41e/attachment.htm>

From brett at python.org  Tue Jun  3 02:22:51 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 2 Jun 2008 17:22:51 -0700
Subject: [Python-Dev] [Python-3000] Postponing the first betas
In-Reply-To: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org>
References: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org>
Message-ID: <bbaeab100806021722x7682841du7b10501b41db935@mail.gmail.com>

On Mon, Jun 2, 2008 at 3:51 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> We are going to postpone the first beta releases by one week.  We had some
> problems with mail.python.org today, which prompted a query to Guido from me
> about the postponement.  mail.python.org should now be back up normally now,
> as evidenced by the emailfloodl but in the meantime, Guido said:
>
> "I'd also like to see PEP 3138 (CJK-friendly repr()) and the
> pyprocessing PEP implemented by then, and perhaps some other small
> stuff."
>
> So we're going to do the first beta releases next Wednesday, June 11.
>  Please take this time to stabilize all APIs and features, both in Python
> and C.  Next week, I'll do a gut check on critical and release blocker bugs,
> so please also take a look at those and try to fix what you can.
>

Now is as good a time as any to mention that on Wednesday I am flying
out to help my mother move. I don't know when she is going to have her
Internet connection set up, so I might not be back online until June
16. But thanks to all the help I have been receiving on PEP 3108, I
trust the various people involved to continue to do the right thing in
my absence.

-Brett

From musiccomposition at gmail.com  Tue Jun  3 02:26:22 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 2 Jun 2008 19:26:22 -0500
Subject: [Python-Dev] [Python-3000] Postponing the first betas
In-Reply-To: <bbaeab100806021722x7682841du7b10501b41db935@mail.gmail.com>
References: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org>
	<bbaeab100806021722x7682841du7b10501b41db935@mail.gmail.com>
Message-ID: <1afaf6160806021726w2ed8d453p9467ee006b90aaee@mail.gmail.com>

On Mon, Jun 2, 2008 at 7:22 PM, Brett Cannon <brett at python.org> wrote:
>>
>
> Now is as good a time as any to mention that on Wednesday I am flying
> out to help my mother move. I don't know when she is going to have her
> Internet connection set up, so I might not be back online until June
> 16. But thanks to all the help I have been receiving on PEP 3108, I
> trust the various people involved to continue to do the right thing in
> my absence.

That reminds me of those Dilbert cartoons where his mother ends up
knowing much more about computers than he does. :)

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From musiccomposition at gmail.com  Tue Jun  3 02:49:40 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 2 Jun 2008 19:49:40 -0500
Subject: [Python-Dev] converting the stdlib to str.format
Message-ID: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>

Hi all,
As a newly converted fan of str.format, it gives me pangs to see the
whole stdlib using "%." I realize that str.format is not quite up to
the speed standards we'd like, but I'm sure that will change.

In any case, I'm willing to give the TLC to convert the whole stdlib
to str.format, so I just need your permission! <wink>

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From guido at python.org  Tue Jun  3 04:04:07 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Jun 2008 19:04:07 -0700
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
Message-ID: <ca471dc20806021904x13fe447ao85b54d3c8b594e2c@mail.gmail.com>

On Mon, Jun 2, 2008 at 5:49 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> As a newly converted fan of str.format, it gives me pangs to see the
> whole stdlib using "%." I realize that str.format is not quite up to
> the speed standards we'd like, but I'm sure that will change.
>
> In any case, I'm willing to give the TLC to convert the whole stdlib
> to str.format, so I just need your permission! <wink>

So glad you asked! There's a standard answer whenever someone suggests
a sweeping conversion of the entire stdlib to some new feature. It's
"No." :-)

In the past, we've tried things like this a few times, and it was
nearly always a mistake. Things like this can't be automated 100%, and
doing a manual audit of the changes is so incredibly tedious that
inevitably a few incorrect changes slip by.

In theory these would be caught by unittests, but in practice the
difference between practice and theory is larger than in theory, so
inevitably the honor falls to some poor schmuck whose application
fails in a non-trivial way, often months later. I recall one
particularly ill-fated conversion where we some bogus conversions
weren't fixed until several years (and releases!) later. And these
were pretty badly botched too, something like foo(bar(x, y)) vs.
foo(bar(x), y) -- it was just in rarely executed code and due to the
complexity of the expressions (and the fact that the parentheses were
balanced ;-) nobody had noticed before.

It's much better to do this piecemeal, when you are revising a
particular module for some other reason. Then you (presumably) have
the time to review every change thoroughly -- and hopefully you can
even add unittests for all code you touch.

I realize that 2to3 might make such conversions less error-prone. But
most 2to3 conversions are not 100% correct either, and the problem of
falling asleep while reviewing the changes exists especially where
there are so many correct conversions!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Tue Jun  3 09:41:40 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Jun 2008 09:41:40 +0200
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
Message-ID: <4844F5B4.5090703@v.loewis.de>

> In any case, I'm willing to give the TLC to convert the whole stdlib
> to str.format, so I just need your permission! <wink>

Please don't - not before % is actually deprecated (which I hope won't
happen until Python 4, with removal of % in Python 5, in the year
when I retire, i.e. 2037).

Regards,
Martin

From g.brandl at gmx.net  Tue Jun  3 10:21:09 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 03 Jun 2008 10:21:09 +0200
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <4844F5B4.5090703@v.loewis.de>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<4844F5B4.5090703@v.loewis.de>
Message-ID: <g22unc$hbu$1@ger.gmane.org>

Martin v. L?wis schrieb:
>> In any case, I'm willing to give the TLC to convert the whole stdlib
>> to str.format, so I just need your permission! <wink>
> 
> Please don't - not before % is actually deprecated (which I hope won't
> happen until Python 4, with removal of % in Python 5, in the year
> when I retire, i.e. 2037).

Now this is news to me -- was there a discussion that changed the
lifetime expectancy of str.__mod__? I'd always supposed it being
deprecated at some point in 3.x.

Not that I'm opposed to keeping it... it *will* be a pain to migrate.

Georg


From solipsis at pitrou.net  Tue Jun  3 10:44:29 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Jun 2008 08:44:29 +0000 (UTC)
Subject: [Python-Dev] Mini-Pep: An Empty String ABC
References: <A8708637239F4FEEA25DE934DBFE639C@RaymondLaptop1>
	<ca471dc20806011606vece88b8g679c38f089040440@mail.gmail.com>
	<07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1>
Message-ID: <loom.20080603T084123-121@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> 
> By subclassing Sequence, we get index() and count() mixins for free.
> 

It seems to me that Sequence.index()/count() and String.index()/count() 
shouldn't have the same semantics.
In the former case they search for items in the Sequence, in the latter case 
they search for substrings of the String.




From eric+python-dev at trueblade.com  Tue Jun  3 11:03:56 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Tue, 03 Jun 2008 05:03:56 -0400
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <g22unc$hbu$1@ger.gmane.org>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>
	<g22unc$hbu$1@ger.gmane.org>
Message-ID: <484508FC.6070700@trueblade.com>

Georg Brandl wrote:
> Martin v. L?wis schrieb:
>>> In any case, I'm willing to give the TLC to convert the whole stdlib
>>> to str.format, so I just need your permission! <wink>
>>
>> Please don't - not before % is actually deprecated (which I hope won't
>> happen until Python 4, with removal of % in Python 5, in the year
>> when I retire, i.e. 2037).
> 
> Now this is news to me -- was there a discussion that changed the
> lifetime expectancy of str.__mod__? I'd always supposed it being
> deprecated at some point in 3.x.
> 
> Not that I'm opposed to keeping it... it *will* be a pain to migrate.

Guido has previously said he wouldn't mind adding a 
PendingDeprecationWarning to %-formatting in 3.0.  I've attempted to do 
that in http://bugs.python.org/issue2772.  For a reason I don't 
understand, my change broke test_doctest.py, so I've never applied it.

Eric.

From python at rcn.com  Tue Jun  3 12:09:09 2008
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Jun 2008 03:09:09 -0700
Subject: [Python-Dev] Mini-Pep: An Empty String ABC
References: <A8708637239F4FEEA25DE934DBFE639C@RaymondLaptop1><ca471dc20806011606vece88b8g679c38f089040440@mail.gmail.com><07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1>
	<loom.20080603T084123-121@post.gmane.org>
Message-ID: <0D0F7B10924D4B19BB45DDCC121A7435@RaymondLaptop1>

From: "Antoine Pitrou" <solipsis at pitrou.net>
> It seems to me that Sequence.index()/count() and String.index()/count() 
> shouldn't have the same semantics.
> In the former case they search for items in the Sequence, in the latter case 
> they search for substrings of the String.

And the same applies to __contains__().


Raymond

From solipsis at pitrou.net  Tue Jun  3 12:39:04 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Jun 2008 10:39:04 +0000 (UTC)
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
References: <48397ECC.9070805@cheimes.de>
	<52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com>
	<483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de>
	<483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com>
	<483FBCB4.5020007@egenix.com>
	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
	<4843E884.1060705@egenix.com>
	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>
	<ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>
Message-ID: <loom.20080603T103819-364@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> I'd prefer the 2.6 code base to
> stay true to 2.x, and the 3.0 code base start afresh where it makes
> sense. We should reindent more of the 3.0 code base to use
> 4-space-indents in C code too.

Is there any reason reindenting shouldn't be done for 2.6 too?
(apart from "staying true to 2.x" :-))

Antoine.



From g.brandl at gmx.net  Tue Jun  3 12:48:54 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 03 Jun 2008 12:48:54 +0200
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <loom.20080603T103819-364@post.gmane.org>
References: <48397ECC.9070805@cheimes.de>	<52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com>	<483ECA52.6040000@egenix.com>
	<483ECF94.7060607@cheimes.de>	<483EF139.8000606@egenix.com>
	<483F34C3.3050402@gmail.com>	<483FBCB4.5020007@egenix.com>	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>	<4843E884.1060705@egenix.com>	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>	<ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>
	<loom.20080603T103819-364@post.gmane.org>
Message-ID: <g237cd$fn3$1@ger.gmane.org>

Antoine Pitrou schrieb:
> Guido van Rossum <guido <at> python.org> writes:
>> I'd prefer the 2.6 code base to
>> stay true to 2.x, and the 3.0 code base start afresh where it makes
>> sense. We should reindent more of the 3.0 code base to use
>> 4-space-indents in C code too.
> 
> Is there any reason reindenting shouldn't be done for 2.6 too?
> (apart from "staying true to 2.x" :-))

It would make svn blame useless, for a start.

(SVN could really use a feature to exclude certain revisions from
showing up in svn blame.)

Georg


From eric+python-dev at trueblade.com  Tue Jun  3 12:55:37 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Tue, 03 Jun 2008 06:55:37 -0400
Subject: [Python-Dev] Missing checkin emails
Message-ID: <48452329.6020502@trueblade.com>

Did some checkin emails get lost yesterday?  I didn't see mail for a 
checkin of mine, r63895.  Looking at 
http://mail.python.org/pipermail/python-checkins/2008-June/date.html, it 
looks like r63887-r63898 are missing from the archive, too.

We should probably make an effort to review those checkins.

Eric.

From barry at python.org  Tue Jun  3 13:34:00 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 3 Jun 2008 07:34:00 -0400
Subject: [Python-Dev] Missing checkin emails
In-Reply-To: <48452329.6020502@trueblade.com>
References: <48452329.6020502@trueblade.com>
Message-ID: <4E4C5FD0-5521-48A5-93B6-DA22036D9CBD@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 3, 2008, at 6:55 AM, Eric Smith wrote:

> Did some checkin emails get lost yesterday?  I didn't see mail for a  
> checkin of mine, r63895.  Looking at http://mail.python.org/pipermail/python-checkins/2008-June/date.html 
> , it looks like r63887-r63898 are missing from the archive, too.
>
> We should probably make an effort to review those checkins.

mail.python.org had problems yesterday, so if the mail client doesn't  
resend, it's very possible that messages got lost.  Everything is back  
to normal now.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSEUsKnEjvBPtnXfVAQKPbwP+MQGUbxyydoa0OAwMM58mS1IhzLubwCpy
IUd5AQulh4FPRLn04CdOQQzYlmOJg9gtkQUIlQx4iiap75H4QqM91ubSdEnzPrPH
vUXIXvnujKLwEZ4PN08ZQJffp7o5w8ARLso9U0ttYAtlXjx4QYEmxpyKVVe/V/dB
zemVnvW4qfE=
=eojc
-----END PGP SIGNATURE-----

From eric+python-dev at trueblade.com  Tue Jun  3 14:10:56 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Tue, 03 Jun 2008 08:10:56 -0400
Subject: [Python-Dev] Missing checkin emails
In-Reply-To: <4E4C5FD0-5521-48A5-93B6-DA22036D9CBD@python.org>
References: <48452329.6020502@trueblade.com>
	<4E4C5FD0-5521-48A5-93B6-DA22036D9CBD@python.org>
Message-ID: <484534D0.3060602@trueblade.com>

Barry Warsaw wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Jun 3, 2008, at 6:55 AM, Eric Smith wrote:
> 
>> Did some checkin emails get lost yesterday?  I didn't see mail for a 
>> checkin of mine, r63895.  Looking at 
>> http://mail.python.org/pipermail/python-checkins/2008-June/date.html, 
>> it looks like r63887-r63898 are missing from the archive, too.
>>
>> We should probably make an effort to review those checkins.
> 
> mail.python.org had problems yesterday, so if the mail client doesn't 
> resend, it's very possible that messages got lost.  Everything is back 
> to normal now.

Thanks, Barry.

Does anyone know if we can force svn to resend the checkin notifications 
for that range?  It also looks like r63862 and r63915 are missing.  I 
haven't done an exhaustive analysis of what is and isn't present (I also 
don't know if it's normal for revisions to be missing).

Eric.

From guido at python.org  Tue Jun  3 16:29:32 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Jun 2008 07:29:32 -0700
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <g237cd$fn3$1@ger.gmane.org>
References: <48397ECC.9070805@cheimes.de> <483EF139.8000606@egenix.com>
	<483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com>
	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
	<4843E884.1060705@egenix.com>
	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>
	<ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>
	<loom.20080603T103819-364@post.gmane.org> <g237cd$fn3$1@ger.gmane.org>
Message-ID: <ca471dc20806030729ic6b0d73w3b0d67f648bc712c@mail.gmail.com>

On Tue, Jun 3, 2008 at 3:48 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Antoine Pitrou schrieb:
>>
>> Guido van Rossum <guido <at> python.org> writes:
>>>
>>> I'd prefer the 2.6 code base to
>>> stay true to 2.x, and the 3.0 code base start afresh where it makes
>>> sense. We should reindent more of the 3.0 code base to use
>>> 4-space-indents in C code too.
>>
>> Is there any reason reindenting shouldn't be done for 2.6 too?
>> (apart from "staying true to 2.x" :-))
>
> It would make svn blame useless, for a start.
>
> (SVN could really use a feature to exclude certain revisions from
> showing up in svn blame.)

What he said. And "staying true to 2.x" is not a bad rationale. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Jun  3 16:37:11 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Jun 2008 07:37:11 -0700
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <484508FC.6070700@trueblade.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<4844F5B4.5090703@v.loewis.de> <g22unc$hbu$1@ger.gmane.org>
	<484508FC.6070700@trueblade.com>
Message-ID: <ca471dc20806030737g123f71d3i2440d3b02aaadfda@mail.gmail.com>

On Tue, Jun 3, 2008 at 2:03 AM, Eric Smith
<eric+python-dev at trueblade.com> wrote:
> Georg Brandl wrote:
>>
>> Martin v. L?wis schrieb:
>>>>
>>>> In any case, I'm willing to give the TLC to convert the whole stdlib
>>>> to str.format, so I just need your permission! <wink>
>>>
>>> Please don't - not before % is actually deprecated (which I hope won't
>>> happen until Python 4, with removal of % in Python 5, in the year
>>> when I retire, i.e. 2037).
>>
>> Now this is news to me -- was there a discussion that changed the
>> lifetime expectancy of str.__mod__? I'd always supposed it being
>> deprecated at some point in 3.x.

I think Martin was attempting humor. :-) There's widespread
disagreement on when we should retire %.

>> Not that I'm opposed to keeping it... it *will* be a pain to migrate.
>
> Guido has previously said he wouldn't mind adding a
> PendingDeprecationWarning to %-formatting in 3.0.  I've attempted to do that
> in http://bugs.python.org/issue2772.  For a reason I don't understand, my
> change broke test_doctest.py, so I've never applied it.

For 3.0, it should be at best a SilentDeprecationWarning.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mal at egenix.com  Tue Jun  3 19:43:45 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 03 Jun 2008 19:43:45 +0200
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>
References: <48397ECC.9070805@cheimes.de>	<52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com>	<483ECA52.6040000@egenix.com>
	<483ECF94.7060607@cheimes.de>	<483EF139.8000606@egenix.com>
	<483F34C3.3050402@gmail.com>	<483FBCB4.5020007@egenix.com>	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>	<4843E884.1060705@egenix.com>	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>
	<ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>
Message-ID: <484582D1.7000309@egenix.com>

On 2008-06-03 01:09, Guido van Rossum wrote:
> I will freely admit that I haven't followed this thread in any detail,
> but if it were up to me, I'd have the 2.6 internal code use PyString
> (as both what the linker sees and what the human reads in the source
> code) and the 3.0 code use PyBytes for the same thing. Let the merges
> be damed -- most changes to 2.6 these days seem to be blocked
> explicitly from being merged anyway. I'd prefer the 2.6 code base to
> stay true to 2.x, and the 3.0 code base start afresh where it makes
> sense. We should reindent more of the 3.0 code base to use
> 4-space-indents in C code too.
> 
> I would also add macros that map the PyBytes_* APIs to PyString_*, but
> I would not start using these internally except in code newly written
> for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString
> for 8-bit strings containing text, and PyBytes for 8-bit strings
> containing binary data. For 8-bit strings that could contain either
> text or data, I'd use PyString, in the spirit of 2.x.

+1

Let's work on better merge tools that edit the trunk code base
into shape for a 3.x checkin.

Using automated tools for this
is likely going to lower the probability of bugs introduced
due to unnoticed merge conflicts and in the end is also going
to be a benefit to everyone wanting to maintain a single code
base for both targets.

Perhaps we could revive the old Tools/scripts/fixcid.py that was
used for the 1.4->1.5 renaming ?!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 03 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            33 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From martin at v.loewis.de  Tue Jun  3 20:54:02 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Jun 2008 20:54:02 +0200
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <g22unc$hbu$1@ger.gmane.org>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>
	<g22unc$hbu$1@ger.gmane.org>
Message-ID: <4845934A.6030708@v.loewis.de>

>> Please don't - not before % is actually deprecated (which I hope won't
>> happen until Python 4, with removal of % in Python 5, in the year
>> when I retire, i.e. 2037).
> 
> Now this is news to me -- was there a discussion that changed the
> lifetime expectancy of str.__mod__? I'd always supposed it being
> deprecated at some point in 3.x.

The PEP doesn't specify anything, and I don't recall any discussion,
either - the specific timing suggested above is merely my own hopes.

Regards,
Martin

From martin at v.loewis.de  Tue Jun  3 20:58:24 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Jun 2008 20:58:24 +0200
Subject: [Python-Dev] Missing checkin emails
In-Reply-To: <484534D0.3060602@trueblade.com>
References: <48452329.6020502@trueblade.com>	<4E4C5FD0-5521-48A5-93B6-DA22036D9CBD@python.org>
	<484534D0.3060602@trueblade.com>
Message-ID: <48459450.7020505@v.loewis.de>

> Does anyone know if we can force svn to resend the checkin notifications
> for that range?  It also looks like r63862 and r63915 are missing.  I
> haven't done an exhaustive analysis of what is and isn't present (I also
> don't know if it's normal for revisions to be missing).

I just resent the range that you indicated (r63887-r63898, inclusive).

Regards,
Martin

From r.m.oudkerk at googlemail.com  Tue Jun  3 21:16:31 2008
From: r.m.oudkerk at googlemail.com (r.m.oudkerk)
Date: Tue, 3 Jun 2008 20:16:31 +0100
Subject: [Python-Dev] Postponing the first betas
In-Reply-To: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org>
References: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org>
Message-ID: <ac4216f0806031216t73d8ea3n7b07eb3fb12eea71@mail.gmail.com>

On 02/06/2008, Barry Warsaw <barry at python.org> wrote:
> meantime, Guido said:
>
> "I'd also like to see PEP 3138 (CJK-friendly repr()) and the
> pyprocessing PEP implemented by then, and perhaps some other small
> stuff."

The pyprocessing unit tests crash with a fatal error when run on Linux
with a debug version of the interpreter.  This is because the GILState
stuff is not fork aware.  I submitted a patch some months  ago:

    http://bugs.python.org/issue1683

Could somebody review it please.

Cheers,

Richard.

From jnoller at gmail.com  Tue Jun  3 23:30:53 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 3 Jun 2008 17:30:53 -0400
Subject: [Python-Dev] PEP 371: Additional Discussion
Message-ID: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>

Per feedback from Guido, the python-dev list and others, I have sent
in an updated version of PEP 371 - the inclusion of the pyprocessing
module into the standard library.

URL: http://www.python.org/dev/peps/pep-0371/

New highlights:
 * The module will be renamed to "multiprocessing"
 * The API will become PEP 8 compliant
 * Additional Benchmark data added (thanks Alec!)
 * Additional grammar and terminology changes

Please review the new more final draft of the PEP and send out
suggestions and feedback. It looks like this is on the slate for 2.6b1
and 3.0b1 so the sooner we get feedback, the better.

-Jesse

From p.f.moore at gmail.com  Tue Jun  3 23:38:42 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 3 Jun 2008 22:38:42 +0100
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
Message-ID: <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>

2008/6/3 Jesse Noller <jnoller at gmail.com>:
> Per feedback from Guido, the python-dev list and others, I have sent
> in an updated version of PEP 371 - the inclusion of the pyprocessing
> module into the standard library.
>
> URL: http://www.python.org/dev/peps/pep-0371/
>
> New highlights:
>  * The module will be renamed to "multiprocessing"
>  * The API will become PEP 8 compliant

Doesn't that kill the intent that it's a drop-in replacement for
threading? I'd say don't do this unless threading is made PEP 8
compliant as well.

Paul.

From python at rcn.com  Tue Jun  3 23:43:27 2008
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Jun 2008 14:43:27 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
Message-ID: <F58430C304134512908AE6768974761A@RaymondLaptop1>

>>  * The API will become PEP 8 compliant
> 
> Doesn't that kill the intent that it's a drop-in replacement for
> threading?

IMO, it is essential that the API match the theading module, PEP 8 be damned.


Raymond



From jnoller at gmail.com  Wed Jun  4 00:02:58 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 3 Jun 2008 18:02:58 -0400
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <F58430C304134512908AE6768974761A@RaymondLaptop1>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
Message-ID: <465497BF-CB4A-46F3-B438-3B60C1FF7E3A@gmail.com>

What about also including a patch to fix the threading api to be pep 8  
compliant?

On Jun 3, 2008, at 5:43 PM, "Raymond Hettinger" <python at rcn.com> wrote:

>>> * The API will become PEP 8 compliant
>> Doesn't that kill the intent that it's a drop-in replacement for
>> threading?
>
> IMO, it is essential that the API match the theading module, PEP 8  
> be damned.
>
>
> Raymond
>
>

From musiccomposition at gmail.com  Wed Jun  4 00:05:08 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 3 Jun 2008 17:05:08 -0500
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <465497BF-CB4A-46F3-B438-3B60C1FF7E3A@gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<465497BF-CB4A-46F3-B438-3B60C1FF7E3A@gmail.com>
Message-ID: <1afaf6160806031505h1ac5975bt6a0f9f7b4e8e922b@mail.gmail.com>

On Tue, Jun 3, 2008 at 5:02 PM, Jesse Noller <jnoller at gmail.com> wrote:
> What about also including a patch to fix the threading api to be pep 8
> compliant?

I doubt that will be accepted because of the closeness threading has
to Java's APIs.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From jnoller at gmail.com  Wed Jun  4 00:08:05 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 3 Jun 2008 18:08:05 -0400
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <F58430C304134512908AE6768974761A@RaymondLaptop1>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
Message-ID: <C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>

Also - we could leave in stubs to match the threading api - Guido,  
David Goodger and others really prefer not to continue the "broken"  
API of the threading API

On Jun 3, 2008, at 5:43 PM, "Raymond Hettinger" <python at rcn.com> wrote:

>>> * The API will become PEP 8 compliant
>> Doesn't that kill the intent that it's a drop-in replacement for
>> threading?
>
> IMO, it is essential that the API match the theading module, PEP 8  
> be damned.
>
>
> Raymond
>
>

From solipsis at pitrou.net  Wed Jun  4 00:27:42 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Jun 2008 22:27:42 +0000 (UTC)
Subject: [Python-Dev] PEP 371: Additional Discussion
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<465497BF-CB4A-46F3-B438-3B60C1FF7E3A@gmail.com>
	<1afaf6160806031505h1ac5975bt6a0f9f7b4e8e922b@mail.gmail.com>
Message-ID: <loom.20080603T222401-583@post.gmane.org>

Benjamin Peterson <musiccomposition <at> gmail.com> writes:
> On Tue, Jun 3, 2008 at 5:02 PM, Jesse Noller <jnoller <at> gmail.com> wrote:
> > What about also including a patch to fix the threading api to be pep 8
> > compliant?
> 
> I doubt that will be accepted because of the closeness threading has
> to Java's APIs.

Is this really a serious concern for some people?
If you come from the Java world and are learning Python, I suppose there are
things more difficult to assimilate than the change in method naming style...



From brett at python.org  Wed Jun  4 00:28:02 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 3 Jun 2008 15:28:02 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
Message-ID: <bbaeab100806031528g1a4843d9o856cca6987d25b8e@mail.gmail.com>

On Tue, Jun 3, 2008 at 3:08 PM, Jesse Noller <jnoller at gmail.com> wrote:
> Also - we could leave in stubs to match the threading api - Guido, David
> Goodger and others really prefer not to continue the "broken" API of the
> threading API
>

+1 from me. Gives a transition plan for people to move over to the new API.

-Brett

> On Jun 3, 2008, at 5:43 PM, "Raymond Hettinger" <python at rcn.com> wrote:
>
>>>> * The API will become PEP 8 compliant
>>>
>>> Doesn't that kill the intent that it's a drop-in replacement for
>>> threading?
>>
>> IMO, it is essential that the API match the theading module, PEP 8 be
>> damned.
>>
>>
>> Raymond
>>
>>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From musiccomposition at gmail.com  Wed Jun  4 00:53:37 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 3 Jun 2008 17:53:37 -0500
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
Message-ID: <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>

On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller <jnoller at gmail.com> wrote:
> Also - we could leave in stubs to match the threading api - Guido, David
> Goodger and others really prefer not to continue the "broken" API of the
> threading API
I agree that the threading the the pyprocessing APIs should be PEP 8
compliant, but I think 2 APIs is almost worse than one wrong one.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From brett at python.org  Wed Jun  4 01:00:56 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 3 Jun 2008 16:00:56 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
	<1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
Message-ID: <bbaeab100806031600nec9c89ase14aece7f075cb7d@mail.gmail.com>

On Tue, Jun 3, 2008 at 3:53 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller <jnoller at gmail.com> wrote:
>> Also - we could leave in stubs to match the threading api - Guido, David
>> Goodger and others really prefer not to continue the "broken" API of the
>> threading API
> I agree that the threading the the pyprocessing APIs should be PEP 8
> compliant, but I think 2 APIs is almost worse than one wrong one.
>

I disagree. If you state upfront that one of them is only for
backwards-compatibility/transitioning and will be going away in the
future, then I think it is fine to have the extra API.

-Brett

From mike.klaas at gmail.com  Wed Jun  4 01:01:49 2008
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 3 Jun 2008 16:01:49 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
	<1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
Message-ID: <1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com>


On 3-Jun-08, at 3:53 PM, Benjamin Peterson wrote:

> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller <jnoller at gmail.com>  
> wrote:
>> Also - we could leave in stubs to match the threading api - Guido,  
>> David
>> Goodger and others really prefer not to continue the "broken" API  
>> of the
>> threading API
> I agree that the threading the the pyprocessing APIs should be PEP 8
> compliant, but I think 2 APIs is almost worse than one wrong one.

A cleaner way to effectuate the transition would be to leave the  
camelCase API in 2.6 (for both modules), switch to PEP 8 in py3k (for  
both modules), and provide threading3k and multiprocessing3k modules  
in 2.6 that fa?ade the 2.6 API with the PEP 8 API.

2to3 would rewrite 'import threading3k' to 'import threading' and  
everything would work (it would warn about 'import threading' in 2.6  
code).

-Mike

From musiccomposition at gmail.com  Wed Jun  4 01:20:48 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 3 Jun 2008 18:20:48 -0500
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <bbaeab100806031600nec9c89ase14aece7f075cb7d@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
	<1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
	<bbaeab100806031600nec9c89ase14aece7f075cb7d@mail.gmail.com>
Message-ID: <1afaf6160806031620g42a31199g1d284783a07a21b2@mail.gmail.com>

On Tue, Jun 3, 2008 at 6:00 PM, Brett Cannon <brett at python.org> wrote:
> On Tue, Jun 3, 2008 at 3:53 PM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller <jnoller at gmail.com> wrote:
>>> Also - we could leave in stubs to match the threading api - Guido, David
>>> Goodger and others really prefer not to continue the "broken" API of the
>>> threading API
>> I agree that the threading the the pyprocessing APIs should be PEP 8
>> compliant, but I think 2 APIs is almost worse than one wrong one.
>>
>
> I disagree. If you state upfront that one of them is only for
> backwards-compatibility/transitioning and will be going away in the
> future, then I think it is fine to have the extra API.

In that case, I'm +1 as long as we implement a full DeprecationWarning on one.

>
> -Brett
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From guido at python.org  Wed Jun  4 01:21:10 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Jun 2008 16:21:10 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
	<1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
	<1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com>
Message-ID: <ca471dc20806031621k112db84di64ee052a33a367e@mail.gmail.com>

I'm curious why people thing that strict API compatibility is
important at all. In my view, having the APIs be similar is really
helpful because it helps people quickly understand what you can do
with the new module. But I honestly don't expect anyone to take an
existing app using threading and turning it into one using
multiprocessing by changing a single line. There are too many things
that threads let you do that aren't supported by processes (e.g.
shared mutable objects protected by locks).

IMO the multiprocessing module should only provide a PEP-8-compliant
API, and optionally we could start adding such an API to the threading
module. Strict compatibility with the Java API really isn't important
there either -- that's just how I got started with it, and I gladly
admit I went overboard there.

--Guido

On Tue, Jun 3, 2008 at 4:01 PM, Mike Klaas <mike.klaas at gmail.com> wrote:
>
> On 3-Jun-08, at 3:53 PM, Benjamin Peterson wrote:
>
>> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller <jnoller at gmail.com> wrote:
>>>
>>> Also - we could leave in stubs to match the threading api - Guido, David
>>> Goodger and others really prefer not to continue the "broken" API of the
>>> threading API
>>
>> I agree that the threading the the pyprocessing APIs should be PEP 8
>> compliant, but I think 2 APIs is almost worse than one wrong one.
>
> A cleaner way to effectuate the transition would be to leave the camelCase
> API in 2.6 (for both modules), switch to PEP 8 in py3k (for both modules),
> and provide threading3k and multiprocessing3k modules in 2.6 that fa?ade the
> 2.6 API with the PEP 8 API.
>
> 2to3 would rewrite 'import threading3k' to 'import threading' and everything
> would work (it would warn about 'import threading' in 2.6 code).
>
> -Mike
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Wed Jun  4 03:48:59 2008
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Jun 2008 18:48:59 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com><79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com><F58430C304134512908AE6768974761A@RaymondLaptop1><C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com><1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com><1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com>
	<ca471dc20806031621k112db84di64ee052a33a367e@mail.gmail.com>
Message-ID: <5062908B5F0144528DC9F35A19D425B3@RaymondLaptop1>

I think its a small disaster to have the APIs be almost
 the same but with trivial differences in spelling.

PEP 8 is a nice guideline but it seems to have become
an end in an of itself.  The point of the PEP is to use
consistency as a memory cue, but having two sets of
method names that are almost parallel but spelled
differently is at odds with that goal.

Moving both modules to PEP 8 compliance makes more
sense, but I would hope that gets left for Py3.0.  There's
too much existing code, too many existing tutorials, and
too many hardcopy books already in Py2.x.


Raymond


----- Original Message ----- 
From: "Guido van Rossum" <guido at python.org>
To: "Mike Klaas" <mike.klaas at gmail.com>
Cc: <python-dev at python.org>
Sent: Tuesday, June 03, 2008 4:21 PM
Subject: Re: [Python-Dev] PEP 371: Additional Discussion


I'm curious why people thing that strict API compatibility is
important at all. In my view, having the APIs be similar is really
helpful because it helps people quickly understand what you can do
with the new module. But I honestly don't expect anyone to take an
existing app using threading and turning it into one using
multiprocessing by changing a single line. There are too many things
that threads let you do that aren't supported by processes (e.g.
shared mutable objects protected by locks).

IMO the multiprocessing module should only provide a PEP-8-compliant
API, and optionally we could start adding such an API to the threading
module. Strict compatibility with the Java API really isn't important
there either -- that's just how I got started with it, and I gladly
admit I went overboard there.

--Guido

On Tue, Jun 3, 2008 at 4:01 PM, Mike Klaas <mike.klaas at gmail.com> wrote:
>
> On 3-Jun-08, at 3:53 PM, Benjamin Peterson wrote:
>
>> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller <jnoller at gmail.com> wrote:
>>>
>>> Also - we could leave in stubs to match the threading api - Guido, David
>>> Goodger and others really prefer not to continue the "broken" API of the
>>> threading API
>>
>> I agree that the threading the the pyprocessing APIs should be PEP 8
>> compliant, but I think 2 APIs is almost worse than one wrong one.
>
> A cleaner way to effectuate the transition would be to leave the camelCase
> API in 2.6 (for both modules), switch to PEP 8 in py3k (for both modules),
> and provide threading3k and multiprocessing3k modules in 2.6 that fa?ade the
> 2.6 API with the PEP 8 API.
>
> 2to3 would rewrite 'import threading3k' to 'import threading' and everything
> would work (it would warn about 'import threading' in 2.6 code).
>
> -Mike
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com 


From python at rcn.com  Wed Jun  4 03:56:28 2008
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Jun 2008 18:56:28 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com><79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com><F58430C304134512908AE6768974761A@RaymondLaptop1><C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com><1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
	<1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com>
Message-ID: <3DADCFCEEBBB4695A6A50646D1FD15C1@RaymondLaptop1>

From: "Mike Klaas" <mike.klaas at gmail.com>
> A cleaner way to effectuate the transition would be to leave
> the camelCase API in 2.6 (for both modules), switch to PEP 8
> in py3k (for  both modules)

+1
That makes good sense.


> , and provide threading3k and multiprocessing3k modules  in 2.6 that fa?ade the 2.6 API with the PEP 8 API.

+0
PEP 8 is nice but it's not that important.


Raymond 


From greg.ewing at canterbury.ac.nz  Wed Jun  4 04:28:22 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 04 Jun 2008 14:28:22 +1200
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
	<1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
Message-ID: <4845FDC6.2050909@canterbury.ac.nz>

Benjamin Peterson wrote:
> I agree that the threading the the pyprocessing APIs should be PEP 8
> compliant, but I think 2 APIs is almost worse than one wrong one.

So change them both to be PEP 8 compliant, and leave
aliases in both for existing code to use.

-- 
Greg

From guido at python.org  Wed Jun  4 06:36:34 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Jun 2008 21:36:34 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <5062908B5F0144528DC9F35A19D425B3@RaymondLaptop1>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
	<1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
	<1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com>
	<ca471dc20806031621k112db84di64ee052a33a367e@mail.gmail.com>
	<5062908B5F0144528DC9F35A19D425B3@RaymondLaptop1>
Message-ID: <ca471dc20806032136y4b29b818r78b93dc0e03b55a3@mail.gmail.com>

On Tue, Jun 3, 2008 at 6:48 PM, Raymond Hettinger <python at rcn.com> wrote:
> I think its a small disaster to have the APIs be almost
> the same but with trivial differences in spelling.

Strong words -- and it now seems just your gut feelings against mine.
I have come to trust mine quite a bit.

> PEP 8 is a nice guideline but it seems to have become
> an end in an of itself.  The point of the PEP is to use
> consistency as a memory cue, but having two sets of
> method names that are almost parallel but spelled
> differently is at odds with that goal.

PEP 8 helps by making exact method spellings easier to remember. The
recommendation against abbreviations also comes from this goal. It
should be used for all new APIs. (Except of course when adding to a
set of existing APIs that use a different convention.)

I consider multiprocessing a new API -- while it bears a superficial
resemblance with the threading API the similarities are just that, and
it should not be constrained by mistakes in that API.

> Moving both modules to PEP 8 compliance makes more
> sense, but I would hope that gets left for Py3.0.  There's
> too much existing code, too many existing tutorials, and
> too many hardcopy books already in Py2.x.

It was decided not to "fix" APIs for Py3k beyond the module naming
issue. We can add a new PEP-8-compliant API to the threading module in
3.0 but the old API should stay for another release or more. I'm
neutral on whether it makes sense to backport the new threading.py
APIs to 2.6.

--Guido

> Raymond
>
>
> ----- Original Message ----- From: "Guido van Rossum" <guido at python.org>
> To: "Mike Klaas" <mike.klaas at gmail.com>
> Cc: <python-dev at python.org>
> Sent: Tuesday, June 03, 2008 4:21 PM
> Subject: Re: [Python-Dev] PEP 371: Additional Discussion
>
>
> I'm curious why people thing that strict API compatibility is
> important at all. In my view, having the APIs be similar is really
> helpful because it helps people quickly understand what you can do
> with the new module. But I honestly don't expect anyone to take an
> existing app using threading and turning it into one using
> multiprocessing by changing a single line. There are too many things
> that threads let you do that aren't supported by processes (e.g.
> shared mutable objects protected by locks).
>
> IMO the multiprocessing module should only provide a PEP-8-compliant
> API, and optionally we could start adding such an API to the threading
> module. Strict compatibility with the Java API really isn't important
> there either -- that's just how I got started with it, and I gladly
> admit I went overboard there.
>
> --Guido
>
> On Tue, Jun 3, 2008 at 4:01 PM, Mike Klaas <mike.klaas at gmail.com> wrote:
>>
>> On 3-Jun-08, at 3:53 PM, Benjamin Peterson wrote:
>>
>>> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller <jnoller at gmail.com> wrote:
>>>>
>>>> Also - we could leave in stubs to match the threading api - Guido, David
>>>> Goodger and others really prefer not to continue the "broken" API of the
>>>> threading API
>>>
>>> I agree that the threading the the pyprocessing APIs should be PEP 8
>>> compliant, but I think 2 APIs is almost worse than one wrong one.
>>
>> A cleaner way to effectuate the transition would be to leave the camelCase
>> API in 2.6 (for both modules), switch to PEP 8 in py3k (for both modules),
>> and provide threading3k and multiprocessing3k modules in 2.6 that fa?ade
>> the
>> 2.6 API with the PEP 8 API.
>>
>> 2to3 would rewrite 'import threading3k' to 'import threading' and
>> everything
>> would work (it would warn about 'import threading' in 2.6 code).
>>
>> -Mike
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/python%40rcn.com
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jun  4 06:37:27 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Jun 2008 21:37:27 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <3DADCFCEEBBB4695A6A50646D1FD15C1@RaymondLaptop1>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com>
	<F58430C304134512908AE6768974761A@RaymondLaptop1>
	<C28B24D9-2060-4182-B65D-A455EEE33CB5@gmail.com>
	<1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com>
	<1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com>
	<3DADCFCEEBBB4695A6A50646D1FD15C1@RaymondLaptop1>
Message-ID: <ca471dc20806032137p6ea7a951pbb74ad7b351017a2@mail.gmail.com>

On Tue, Jun 3, 2008 at 6:56 PM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Mike Klaas" <mike.klaas at gmail.com>
>>
>> A cleaner way to effectuate the transition would be to leave
>> the camelCase API in 2.6 (for both modules), switch to PEP 8
>> in py3k (for  both modules)
>
> +1
> That makes good sense.

No. It makes *no* sense to introduce a new API in 2.6 and change it
again in 3.0. That sounds like another foolish consistency.

>> , and provide threading3k and multiprocessing3k modules  in 2.6 that
>> fa?ade the 2.6 API with the PEP 8 API.
>
> +0
> PEP 8 is nice but it's not that important.
>
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Wed Jun  4 10:54:25 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 4 Jun 2008 01:54:25 -0700
Subject: [Python-Dev] Mini-Pep:  Simplifying the Integral ABC
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
Message-ID: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>

The only comment so far was to keep the __index__ method.

Other than that, is this good to go?

Raymond


----- Original Message ----- 
> Target:  Py2.6 and Py3.0
> Author: Raymond Hettinger
> Date: May 31, 2008
> 
> Motivation
> ----------
> The principal purpose of an abstract base class is to support multiple
> implementations of an API; thereby allowing one concrete class to be
> substitutable for another. This purpose is defeated when useful substitutions
> are precluded because the ABC is too aggressive in its behavioral requirements
> -- mandating too many methods, mandating methods that are difficult to
> implement, or too closely following the full API of a single concrete type.
> 
> The Integral ABC is somewhat extensive and requires essentially every behavior
> exhibited by the int concrete class.  Usefully, it provides for basic integer
> behaviors like simple arithmetic and ordering relations.  However, the current
> ABC goes beyond that and requires bit-flipping and other operations that are
> not fundamental to the notion of being an integer.  That makes it challenging
> to define a new Integral class that isn't already an int.
> 
> Proposal
> --------
> Remove non-essential abstract methods like __index__, three argument __pow__,
> __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__, __xor__,
> __rxor__, __or__, __ror__, and __invert__, numerator, and denominator.
> 
> Discussion
> ----------
> The only known use cases for variants of int are types that limit the range of
> values to those that fit in a fixed storage width.
> 
> One existing implementation is in numpy which has types like int0, int8,
> int16, int32, and int16.  The numpy integral types implement all the methods
> present in Integral except for the new methods like __trunc__, __index__,
> real, imag, conjugate, numerator, and denominator.  For the most part, they
> are fully substitutable into code that expects regular ints; however, they do
> have wrap-around behaviors such as int8(200)+int8(100) --> int8(44). The
> wrap-around behaviors make the numpy types totally unsuitable for some
> applications of Integral such as the fractions module which accepts any
> integral numerator and denominator.
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com

From ncoghlan at gmail.com  Wed Jun  4 12:23:08 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Jun 2008 20:23:08 +1000
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <4845934A.6030708@v.loewis.de>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>
	<4845934A.6030708@v.loewis.de>
Message-ID: <48466D0C.40709@gmail.com>

Martin v. L?wis wrote:
>>> Please don't - not before % is actually deprecated (which I hope won't
>>> happen until Python 4, with removal of % in Python 5, in the year
>>> when I retire, i.e. 2037).
>> Now this is news to me -- was there a discussion that changed the
>> lifetime expectancy of str.__mod__? I'd always supposed it being
>> deprecated at some point in 3.x.
> 
> The PEP doesn't specify anything, and I don't recall any discussion,
> either - the specific timing suggested above is merely my own hopes.

While str.format has an awful lot of points in its favour (no tuple/dict 
special casing, much cleaner handling of named arguments, access to 
format options for non-builtin types), there are some debugging cases 
where I suspect the basic version of % formatting will prove to be more 
convenient.

At this point in time, my personal preference would be that later in the 
3.x series certain aspects of % formatting will be deprecated 
(acceptance of non-tuples in favour of the format() builtin, acceptance 
of dicts in favour of str.format), but that simple % formatting of a 
tuple of values will still be permitted.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Wed Jun  4 12:33:07 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Jun 2008 20:33:07 +1000
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
Message-ID: <48466F63.3020902@gmail.com>

Jesse Noller wrote:
> Per feedback from Guido, the python-dev list and others, I have sent
> in an updated version of PEP 371 - the inclusion of the pyprocessing
> module into the standard library.
> 
> URL: http://www.python.org/dev/peps/pep-0371/
> 
> New highlights:
>  * The module will be renamed to "multiprocessing"
>  * The API will become PEP 8 compliant

My 2 cents on this: PEP 8 compliance in the multiprocessing API is a 
good idea, but threading should be updated at the same time to provide 
PEP 8 compliant aliases for the existing names. Using the old Java-based 
names should provoke a Py3k warning in 2.6 (note: to avoid a runtime 
performance hit when not using -3, the actual function definitions 
should be made conditional, rather than checking whether or not to emit 
the warning every time the legacy API is invoked)

The PyGILState_* bug in debug builds that RMO pointed out should also be 
mentioned in the PEP as something that needs to be fixed in order to 
implement the PEP.

Cheers,
Nick.

[1] http://bugs.python.org/issue1683

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From fuzzyman at voidspace.org.uk  Wed Jun  4 13:36:48 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 04 Jun 2008 12:36:48 +0100
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <48466D0C.40709@gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>
	<48466D0C.40709@gmail.com>
Message-ID: <48467E50.9030202@voidspace.org.uk>

Nick Coghlan wrote:
> Martin v. L?wis wrote:
>>>> Please don't - not before % is actually deprecated (which I hope won't
>>>> happen until Python 4, with removal of % in Python 5, in the year
>>>> when I retire, i.e. 2037).
>>> Now this is news to me -- was there a discussion that changed the
>>> lifetime expectancy of str.__mod__? I'd always supposed it being
>>> deprecated at some point in 3.x.
>>
>> The PEP doesn't specify anything, and I don't recall any discussion,
>> either - the specific timing suggested above is merely my own hopes.
>
> While str.format has an awful lot of points in its favour (no 
> tuple/dict special casing, much cleaner handling of named arguments, 
> access to format options for non-builtin types), there are some 
> debugging cases where I suspect the basic version of % formatting will 
> prove to be more convenient.
>
> At this point in time, my personal preference would be that later in 
> the 3.x series certain aspects of % formatting will be deprecated 
> (acceptance of non-tuples in favour of the format() builtin, 
> acceptance of dicts in favour of str.format), but that simple % 
> formatting of a tuple of values will still be permitted.
>

+1

Simple string formatting with %s and a single object or a tuple meets 
 >90% of my string formatting needs.

Michael Foord

> Cheers,
> Nick.
>


From solipsis at pitrou.net  Wed Jun  4 14:44:10 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 4 Jun 2008 12:44:10 +0000 (UTC)
Subject: [Python-Dev] converting the stdlib to str.format
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>
	<48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk>
Message-ID: <loom.20080604T124312-888@post.gmane.org>

Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
> Simple string formatting with %s and a single object or a tuple meets 
>  >90% of my string formatting needs.

Not to mention that e.g. "%r" % s is much simpler than "{0!r}".format(s)
(if I got the format spec right).

cheers

Antoine.



From p.f.moore at gmail.com  Wed Jun  4 14:50:48 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 4 Jun 2008 13:50:48 +0100
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <loom.20080604T124312-888@post.gmane.org>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<4844F5B4.5090703@v.loewis.de> <g22unc$hbu$1@ger.gmane.org>
	<4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com>
	<48467E50.9030202@voidspace.org.uk>
	<loom.20080604T124312-888@post.gmane.org>
Message-ID: <79990c6b0806040550w1550194crbe2dc4d16ed45c39@mail.gmail.com>

2008/6/4 Antoine Pitrou <solipsis at pitrou.net>:
> Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
>> Simple string formatting with %s and a single object or a tuple meets
>>  >90% of my string formatting needs.
>
> Not to mention that e.g. "%r" % s is much simpler than "{0!r}".format(s)
> (if I got the format spec right).

repr(s) is simpler still, and doesn't break if s is a tuple :-)

Paul.

From ncoghlan at gmail.com  Wed Jun  4 15:28:40 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Jun 2008 23:28:40 +1000
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <loom.20080604T124312-888@post.gmane.org>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>	<48466D0C.40709@gmail.com>
	<48467E50.9030202@voidspace.org.uk>
	<loom.20080604T124312-888@post.gmane.org>
Message-ID: <48469888.4070603@gmail.com>

Antoine Pitrou wrote:
> Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
>> Simple string formatting with %s and a single object or a tuple meets 
>>  >90% of my string formatting needs.
> 
> Not to mention that e.g. "%r" % s is much simpler than "{0!r}".format(s)
> (if I got the format spec right).

repr(s) is even simpler :)

The basic idea for conversions to the new formatting:

'%r' % v --> repr(v) # or ascii(v) if more appropriate
'%s' % v --> str(v) # or format(v), or format(v, '')

The str/unicode stability that the latter provided in 2.x is no longer 
needed in 3.x. The first one never had any advantage that I can see over 
invoking repr() directly (since repr() and %r always produce an 8-bit 
string in 2.x)

The conversion of fmt % v depends on both fmt and v:

v is a single value, fmt is only a formatting code:
   '%x' % v--> format(v, 'x')

v is a single value, fmt is both a formatting code and additional text:
   'Value: %x' % v--> 'Value: {0:x}'.format(v)
Note that the old code in this case wouldn't display dict or tuple 
instances correctly. Avoiding that ambiguity is a major advantage of the 
new approach.

v is a tuple* (more on this below):
   'Key: %s Value: %s' % k, v --> 'Key: {0} Value: {1}'.format(k, v)

fmt uses named parameters:
   'Value: %(val)s' % dict(val=v) --> 'Value: {val}'.format(val=v)

* I still think the new str.format approach is too verbose and requires 
too much thought for simple use cases where the printf-style % 
replacement hits a sweet spot between being easy to write and easy to read.

So I see the introduction of str.format in 3.0 as an opportunity to 
*fix* %-formatting later in the 3.x series rather than get rid of it 
entirely. The fixes I would apply:

- remove support for %(name)s based formatting (str.format is vastly 
superior once you start naming the parameters).

- remove support for passing a single value to a format string without 
wrapping it in an iterable first

- accept any iterable as the right hand argument str.__mod__

This approach would eliminate the current ambiguity in fmt.v, and would 
allow fmt % [x] to be used to box an argument for safe use in a 
single-parameter format string instead of having to muck around with 
singleton tuples.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From steve at pearwood.info  Wed Jun  4 16:19:10 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 5 Jun 2008 00:19:10 +1000
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <48469888.4070603@gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<loom.20080604T124312-888@post.gmane.org>
	<48469888.4070603@gmail.com>
Message-ID: <200806050019.11367.steve@pearwood.info>

On Wed, 4 Jun 2008 11:28:40 pm Nick Coghlan wrote:

> > Not to mention that e.g. "%r" % s is much simpler than
> > "{0!r}".format(s) (if I got the format spec right).
>
> repr(s) is even simpler :)

Fair enough, and I see your smiley, but consider:

"X %r X" % s

"X {0!r} X".format(s)

"X " + repr(s) + " X"


I for one am excited to see advanced string formatting via .format(), 
but would be very disappointed if future Python versions removed the 
simplicity of %.


-- 
Steven

From solipsis at pitrou.net  Wed Jun  4 16:08:20 2008
From: solipsis at pitrou.net (Antoine)
Date: Wed, 4 Jun 2008 16:08:20 +0200 (CEST)
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <48469888.4070603@gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>	<48466D0C.40709@gmail.com>
	<48467E50.9030202@voidspace.org.uk>
	<loom.20080604T124312-888@post.gmane.org>
	<48469888.4070603@gmail.com>
Message-ID: <33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net>

> Antoine Pitrou wrote:
>>
>> Not to mention that e.g. "%r" % s is much simpler than "{0!r}".format(s)
>> (if I got the format spec right).
>
> repr(s) is even simpler :)

Yes, of course, but in the non-trivial case, e.g. "value=%r" % my_value,
it's much easier to use %-style formatting than playing with repr() and
the other string formatting facilities.

> Note that the old code in this case wouldn't display dict or tuple
> instances correctly. Avoiding that ambiguity is a major advantage of the
> new approach.

I know. For me the problem is not about ditching the % operator for an
intuitively-named method like format(). It's the format syntax which has
become much more complicated and error-prone without any clear advantage.

Regards

Antoine.



From jyasskin at gmail.com  Wed Jun  4 17:05:09 2008
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Wed, 4 Jun 2008 08:05:09 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
Message-ID: <5d44f72f0806040805l5cd8f2dbn1fcaa5e9b29ad793@mail.gmail.com>

Because .numerator and .denominator are defined by the Rational ABC,
and Integral inherits from that, removing them from Integral is a
larger change than removing the other methods. You'd have to remove
them from Rational or break the inheritance relation.

Removing the bitwise operators sounds fine to me.

On Wed, Jun 4, 2008 at 1:54 AM, Raymond Hettinger <python at rcn.com> wrote:
> The only comment so far was to keep the __index__ method.
>
> Other than that, is this good to go?
>
> Raymond
>
>
> ----- Original Message -----
>>
>> Target:  Py2.6 and Py3.0
>> Author: Raymond Hettinger
>> Date: May 31, 2008
>>
>> Motivation
>> ----------
>> The principal purpose of an abstract base class is to support multiple
>> implementations of an API; thereby allowing one concrete class to be
>> substitutable for another. This purpose is defeated when useful
>> substitutions
>> are precluded because the ABC is too aggressive in its behavioral
>> requirements
>> -- mandating too many methods, mandating methods that are difficult to
>> implement, or too closely following the full API of a single concrete
>> type.
>>
>> The Integral ABC is somewhat extensive and requires essentially every
>> behavior
>> exhibited by the int concrete class.  Usefully, it provides for basic
>> integer
>> behaviors like simple arithmetic and ordering relations.  However, the
>> current
>> ABC goes beyond that and requires bit-flipping and other operations that
>> are
>> not fundamental to the notion of being an integer.  That makes it
>> challenging
>> to define a new Integral class that isn't already an int.
>>
>> Proposal
>> --------
>> Remove non-essential abstract methods like __index__, three argument
>> __pow__,
>> __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__,
>> __xor__,
>> __rxor__, __or__, __ror__, and __invert__, numerator, and denominator.
>>
>> Discussion
>> ----------
>> The only known use cases for variants of int are types that limit the
>> range of
>> values to those that fit in a fixed storage width.
>>
>> One existing implementation is in numpy which has types like int0, int8,
>> int16, int32, and int16.  The numpy integral types implement all the
>> methods
>> present in Integral except for the new methods like __trunc__, __index__,
>> real, imag, conjugate, numerator, and denominator.  For the most part,
>> they
>> are fully substitutable into code that expects regular ints; however, they
>> do
>> have wrap-around behaviors such as int8(200)+int8(100) --> int8(44). The
>> wrap-around behaviors make the numpy types totally unsuitable for some
>> applications of Integral such as the fractions module which accepts any
>> integral numerator and denominator.
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/python%40rcn.com
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jyasskin%40gmail.com
>



-- 
Namast?,
Jeffrey Yasskin
http://jeffrey.yasskin.info/

From guido at python.org  Wed Jun  4 17:33:48 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Jun 2008 08:33:48 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
Message-ID: <ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>

I haven't seen anyone applaud either.

Unless more folks actually say they agree I don't want to go forward
with this. There was quite a bit of discussion about PEP 3141 and it
was accepted; striking this much from it with virtually no discussion
seems wrong to me.

Jeffrey made a good point: .numerator and .denominator need to be kept
in the interface. I really don't want to divorce Integer from
Rational. They're trivial to implement, and I won't complain if you
leave them unimplemented while claiming conformance. :-)

My position: I'm -1 on removing __index__, numerator, denominator, and
on removing anything you didn't mention explicitly. You used the
phrase "methods like", which seems inappropriate for a specification.
Note also that these happen to be concrete methods, not abstract ones
like you claim.

I'm -0 on removing the bitwise operators, i.e. could be swayed by some +1 votes.

--Guido

On Wed, Jun 4, 2008 at 1:54 AM, Raymond Hettinger <python at rcn.com> wrote:
> The only comment so far was to keep the __index__ method.
>
> Other than that, is this good to go?
>
> Raymond
>
>
> ----- Original Message -----
>>
>> Target:  Py2.6 and Py3.0
>> Author: Raymond Hettinger
>> Date: May 31, 2008
>>
>> Motivation
>> ----------
>> The principal purpose of an abstract base class is to support multiple
>> implementations of an API; thereby allowing one concrete class to be
>> substitutable for another. This purpose is defeated when useful
>> substitutions
>> are precluded because the ABC is too aggressive in its behavioral
>> requirements
>> -- mandating too many methods, mandating methods that are difficult to
>> implement, or too closely following the full API of a single concrete
>> type.
>>
>> The Integral ABC is somewhat extensive and requires essentially every
>> behavior
>> exhibited by the int concrete class.  Usefully, it provides for basic
>> integer
>> behaviors like simple arithmetic and ordering relations.  However, the
>> current
>> ABC goes beyond that and requires bit-flipping and other operations that
>> are
>> not fundamental to the notion of being an integer.  That makes it
>> challenging
>> to define a new Integral class that isn't already an int.
>>
>> Proposal
>> --------
>> Remove non-essential abstract methods like __index__, three argument
>> __pow__,
>> __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__,
>> __xor__,
>> __rxor__, __or__, __ror__, and __invert__, numerator, and denominator.
>>
>> Discussion
>> ----------
>> The only known use cases for variants of int are types that limit the
>> range of
>> values to those that fit in a fixed storage width.
>>
>> One existing implementation is in numpy which has types like int0, int8,
>> int16, int32, and int16.  The numpy integral types implement all the
>> methods
>> present in Integral except for the new methods like __trunc__, __index__,
>> real, imag, conjugate, numerator, and denominator.  For the most part,
>> they
>> are fully substitutable into code that expects regular ints; however, they
>> do
>> have wrap-around behaviors such as int8(200)+int8(100) --> int8(44). The
>> wrap-around behaviors make the numpy types totally unsuitable for some
>> applications of Integral such as the fractions module which accepts any
>> integral numerator and denominator.
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/python%40rcn.com
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jun  4 19:42:54 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Jun 2008 10:42:54 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <48466F63.3020902@gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<48466F63.3020902@gmail.com>
Message-ID: <ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>

Sounds good. (Maybe you want to contribute a patch to threading.py?
Your implementation notes are important. It could be quite independent
from PEP 371.)

On Wed, Jun 4, 2008 at 3:33 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Jesse Noller wrote:
>>
>> Per feedback from Guido, the python-dev list and others, I have sent
>> in an updated version of PEP 371 - the inclusion of the pyprocessing
>> module into the standard library.
>>
>> URL: http://www.python.org/dev/peps/pep-0371/
>>
>> New highlights:
>>  * The module will be renamed to "multiprocessing"
>>  * The API will become PEP 8 compliant
>
> My 2 cents on this: PEP 8 compliance in the multiprocessing API is a good
> idea, but threading should be updated at the same time to provide PEP 8
> compliant aliases for the existing names. Using the old Java-based names
> should provoke a Py3k warning in 2.6 (note: to avoid a runtime performance
> hit when not using -3, the actual function definitions should be made
> conditional, rather than checking whether or not to emit the warning every
> time the legacy API is invoked)
>
> The PyGILState_* bug in debug builds that RMO pointed out should also be
> mentioned in the PEP as something that needs to be fixed in order to
> implement the PEP.
>
> Cheers,
> Nick.
>
> [1] http://bugs.python.org/issue1683
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>            http://www.boredomandlaziness.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Wed Jun  4 21:57:11 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 4 Jun 2008 12:57:11 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
Message-ID: <2745D7EB7C064B16A88E433588021756@RaymondLaptop1>

From: "Guido van Rossum" <guido at python.org>
> Unless more folks actually say they agree I don't want to go forward
> with this. There was quite a bit of discussion about PEP 3141 and it
> was accepted; striking this much from it with virtually no discussion
> seems wrong to me.

Not sure how to generate more discussion.  It seems self-evident
that an abc with lots of abstract methods is inherently less usable
and that bitwise operations go beyond the basic notion of "integeriness".
Requiring all those methods to be defined makes it harder to write
a compliant class.  Other than int/long, no currently existing type 
matches-up with Integral.  Does anyone care about this?

If something like numpy's int8 eventually grows the required methods,
it is worriesome that the numerator/denominator properties will be
automatically supplied for fractions.py to accept as inputs eventhough
int8's wrap-around behavior makes them entirely unsuitable.  I don't
know if this bothers anyone.  It would seem that a consumer of an Integral
can assume the existence of methods but nothing about whether the
result is usable.  That might not be a big deal except that numpy is t
he only known use case.

Hopefully, some discussion gets generated.  But if no one cares, 
I'll happily drop it.


Raymond



From python at rcn.com  Wed Jun  4 22:28:10 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 4 Jun 2008 13:28:10 -0700
Subject: [Python-Dev] converting the stdlib to str.format
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>	<48466D0C.40709@gmail.com><48467E50.9030202@voidspace.org.uk><loom.20080604T124312-888@post.gmane.org><48469888.4070603@gmail.com>
	<33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net>
Message-ID: <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1>

From: "Antoine" <solipsis at pitrou.net
>  For me the problem is not about ditching the % operator for an
> intuitively-named method like format(). It's the format syntax which has
> become much more complicated and error-prone without any clear advantage.

It's seems that way to me too.  But, it may be one of those things that you quickly get used to.


Raymond

From guido at python.org  Wed Jun  4 22:50:37 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Jun 2008 13:50:37 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
Message-ID: <ca471dc20806041350w78d39281i86b84695ccf13bad@mail.gmail.com>

On Wed, Jun 4, 2008 at 12:57 PM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Guido van Rossum" <guido at python.org>
>>
>> Unless more folks actually say they agree I don't want to go forward
>> with this. There was quite a bit of discussion about PEP 3141 and it
>> was accepted; striking this much from it with virtually no discussion
>> seems wrong to me.
>
> Not sure how to generate more discussion.  It seems self-evident
> that an abc with lots of abstract methods is inherently less usable
> and that bitwise operations go beyond the basic notion of "integeriness".
> Requiring all those methods to be defined makes it harder to write
> a compliant class.

In general it is good to require that more thought goes into the
design and implementation of a class, so that less thought needs to go
into using it.

> Other than int/long, no currently existing type
> matches-up with Integral.  Does anyone care about this?

Does "this" refer to "the Integral ABC" or "no type matches up with it" ?

> If something like numpy's int8 eventually grows the required methods,
> it is worrysome that the numerator/denominator properties will be
> automatically supplied for fractions.py to accept as inputs eventhough
> int8's wrap-around behavior makes them entirely unsuitable.

I'm not sure what you mean. This is probably just my lack of
imagination. Can you give a small code example where using an int8
with the fractions module would cause problems? The Fraction class
appears to be calling __index__ on its arguments, which would convert
the int8 to a proper int (or long, in Python 2.6).

> I don't
> know if this bothers anyone.  It would seem that a consumer of an Integral
> can assume the existence of methods but nothing about whether the
> result is usable.  That might not be a big deal except that numpy is t
> he only known use case.

Any integer type that performs arithmetic modulo some number is
problematic. Perhaps a totally separate ABC needs to be defined for
this purpose, one that doesn't inherit from Rational.

> Hopefully, some discussion gets generated.  But if no one cares, I'll
> happily drop it.

Have you asked the numpy folks?

If enough people care, we could easily create a BinaryInteger ABC that
defines the bitwise operations.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jun  4 22:52:14 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Jun 2008 13:52:14 -0700
Subject: [Python-Dev] Mini-Pep: An Empty String ABC
In-Reply-To: <0D0F7B10924D4B19BB45DDCC121A7435@RaymondLaptop1>
References: <A8708637239F4FEEA25DE934DBFE639C@RaymondLaptop1>
	<ca471dc20806011606vece88b8g679c38f089040440@mail.gmail.com>
	<07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1>
	<loom.20080603T084123-121@post.gmane.org>
	<0D0F7B10924D4B19BB45DDCC121A7435@RaymondLaptop1>
Message-ID: <ca471dc20806041352t7db82719r74f075f29f2d7e56@mail.gmail.com>

Given the lack of responses to my questions, let's reject this.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From amcnabb at mcnabbs.org  Thu Jun  5 00:24:17 2008
From: amcnabb at mcnabbs.org (Andrew McNabb)
Date: Wed, 4 Jun 2008 16:24:17 -0600
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <ca471dc20806041350w78d39281i86b84695ccf13bad@mail.gmail.com>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<ca471dc20806041350w78d39281i86b84695ccf13bad@mail.gmail.com>
Message-ID: <20080604222416.GD26992@mcnabbs.org>

On Wed, Jun 4, 2008 at 12:57 PM, Raymond Hettinger <python at rcn.com> wrote:
>
> Not sure how to generate more discussion.  It seems self-evident
> that an abc with lots of abstract methods is inherently less usable
> and that bitwise operations go beyond the basic notion of "integeriness".
> Requiring all those methods to be defined makes it harder to write
> a compliant class.
>
> Other than int/long, no currently existing type
> matches-up with Integral.  Does anyone care about this?
>
> Hopefully, some discussion gets generated.  But if no one cares, I'll
> happily drop it.

I'm just responding out of duty to add more discussion on the topic.

I think I agree with Raymond on the basic principle that simple ABC's
are easier to use than simple ones.  Anyway, it's hard to add anything
to the discussion when I haven't seen many applications of the Integral
ABC, but in principle, but if I had to vote I'd say that the bitwise
operators aren't particularly integery.


-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080604/8f7cb973/attachment.pgp>

From henning.vonbargen at arcor.de  Wed Jun  4 21:37:22 2008
From: henning.vonbargen at arcor.de (Henning von Bargen)
Date: Wed, 4 Jun 2008 21:37:22 +0200
Subject: [Python-Dev] converting the stdlib to str.format
References: <mailman.1777.1212590197.1043.python-dev@python.org>
Message-ID: <002801c8c67a$68aa7350$6401a8c0@max>

Please, please, please: Keep the % formatting!

In 99% of the Python code I have seen,
the str.format wouldn't gain any advantage
(not even regarding the code readability).

Yes, there may be special cases where str.format
offers more flexibility, but until now I never missed
anything in the % formatting.

One thing to keep in mind:
What if you have a lot of  %-format strings translated
and stored them inside a database, for example?
If  % would be deprecated, these database entries
would have to be converted, so you would have to
write yet another program to do this.

So if the % formating really is going to vanish, then please
not before April 2099 (or when Duke Nukem Forever 
will be released ;-).

Just my 0,02 EUR

Henning


From tjreedy at udel.edu  Thu Jun  5 02:59:47 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 4 Jun 2008 20:59:47 -0400
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1><90B6514494134BEAB0191E8423A986D6@RaymondLaptop1><ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
Message-ID: <g27dq3$t3l$1@ger.gmane.org>


"Raymond Hettinger" <python at rcn.com> wrote in message 
news:2745D7EB7C064B16A88E433588021756 at RaymondLaptop1...
| From: "Guido van Rossum" <guido at python.org>
| > Unless more folks actually say they agree I don't want to go forward
| > with this. There was quite a bit of discussion about PEP 3141 and it
| > was accepted; striking this much from it with virtually no discussion
| > seems wrong to me.
|
| Not sure how to generate more discussion.  It seems self-evident
| that an abc with lots of abstract methods is inherently less usable
| and that bitwise operations go beyond the basic notion of "integeriness".

On reading PEP3141 some months ago and again today, I thought and still do 
that all the methods that depend on a 2s-complement representation and 
implementation really belong to an implentation-defined subclass of 
Integral.  But I am not sure of the purpose of the class and of including 
such concrete methods in an ABC, and so said nothing ;-).

I notice that the similarly implementation-based frexp and ldexp math 
methods are *not* in the Real class.

| Requiring all those methods to be defined makes it harder to write
| a compliant class.  Other than int/long, no currently existing type
| matches-up with Integral.  Does anyone care about this?

I might some day, for didactic purposes, write integer classes based on 
prime-factorization or fibonacci representations.  I certainly would skip 
all the 2s-complement methods.  But I do not know that I would care much 
about being 'non-compliant'.

| If something like numpy's int8 eventually grows the required methods,
| it is worriesome that the numerator/denominator properties will be
| automatically supplied for fractions.py to accept as inputs eventhough
| int8's wrap-around behavior makes them entirely unsuitable.

In my view, the members of residue/remainder classes are not really 
integers unless their usage is carefully restricted to avoid invocation of 
the mod operation.

Terry Jan Reedy




From greg.ewing at canterbury.ac.nz  Thu Jun  5 03:48:07 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 05 Jun 2008 13:48:07 +1200
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <48469888.4070603@gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<4844F5B4.5090703@v.loewis.de> <g22unc$hbu$1@ger.gmane.org>
	<4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com>
	<48467E50.9030202@voidspace.org.uk>
	<loom.20080604T124312-888@post.gmane.org>
	<48469888.4070603@gmail.com>
Message-ID: <484745D7.3000808@canterbury.ac.nz>

Nick Coghlan wrote:

> - remove support for passing a single value to a format string without 
> wrapping it in an iterable first

But won't that clobber a large number of the simple
use cases that you want to keep %-formatting for?

-- 
Greg

From steve at pearwood.info  Thu Jun  5 04:30:33 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 5 Jun 2008 12:30:33 +1000
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <484745D7.3000808@canterbury.ac.nz>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz>
Message-ID: <200806051230.33243.steve@pearwood.info>

On Thu, 5 Jun 2008 11:48:07 am Greg Ewing wrote:
> Nick Coghlan wrote:
> > - remove support for passing a single value to a format string
> > without wrapping it in an iterable first
>
> But won't that clobber a large number of the simple
> use cases that you want to keep %-formatting for?

Yes, I don't particularly see the advantage of writing:

"spam %s spam" % (value,)

over 

"spam %s spam" % value

Why require the first version?


-- 
Steven

From andymac at bullseye.apana.org.au  Thu Jun  5 04:32:48 2008
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Thu, 05 Jun 2008 13:32:48 +1100
Subject: [Python-Dev] wrt the beta deadline and freelist management
In-Reply-To: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org>
References: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org>
Message-ID: <48475050.9090602@bullseye.andymac.org>

There are 2 disparate approaches to clearing/compacting free lists for
basic types:
- APIs of the form Py<type>_ClearFreeList() called from gc.collect()
   [class, frame, method, tuple & unicode types];
- APIs of the form Py<type>_CompactFreeList() called from
   sys._compact_freelists()  [int & float types];

Both approaches are new for 2.6 & 3.0.

The patch at http://bugs.python.org/issue2862 is geared towards bring
the int/float free list management into line with the others.

The patch at http://bugs.python.org/issue3029 adds free list management
to the dict, list & set types.  The value of this is probably minor,
and this patch is thus not significant in its own right other than for
consistency.

However Raymond's comment to issue 3029 that the management routines
shouldn't be public APIs is IMO important (& I happen to agree).

It would be nice to reduce the 2 approaches to one.

I would rather the gc.collect() approach, as this seems to be more
obvious to many users who have gone looking for free list management in
prior versions, however my opinion isn't particularly valuable on this.

Can this be resolved for 2.6/3.0?

Regards,
Andrew.

-- 
-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac at pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia

From dickinsm at gmail.com  Thu Jun  5 05:44:09 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 4 Jun 2008 23:44:09 -0400
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
Message-ID: <5c6f2a5d0806042044p73cd93c2j39b37f214fa91c05@mail.gmail.com>

On Sun, Jun 1, 2008 at 2:15 AM, Raymond Hettinger <python at rcn.com> wrote:

> Proposal
> --------
> Remove non-essential abstract methods like __index__, three argument
> __pow__,
> __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__,
> __xor__,
> __rxor__, __or__, __ror__, and __invert__, numerator, and denominator.
>

+1 from me.

I'd support removing all these, minus the exceptions already pointed out
(__index__, numerator, denominator).  As a (so far incomplete) effort
to speed up the Decimal type I recently implemented a decimal-based
integer type;  this type would seem a natural candidate to inherit
from Integral, but the logical and shift operators above make less sense
for this type.

The other odd man out here is three-argument pow;  this *is* a
method that makes sense for integers without reference to the
way they're stored.  So maybe this should stay.  (Though I've
occasionally wondered why three-argument pow is part of the
core language, rather than being in the standard library.)

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080604/10d51011/attachment.htm>

From guido at python.org  Thu Jun  5 06:13:34 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Jun 2008 21:13:34 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <5c6f2a5d0806042044p73cd93c2j39b37f214fa91c05@mail.gmail.com>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<5c6f2a5d0806042044p73cd93c2j39b37f214fa91c05@mail.gmail.com>
Message-ID: <ca471dc20806042113y49b6cdb7odf010a9807851dfd@mail.gmail.com>

I think it's fine to remove 3-arg pow() from the ABC; implementations
are of course free to provide it.

Raymond, why don't you cook up a patch? It should patch both the PEP
and numbers.py.

On Wed, Jun 4, 2008 at 8:44 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Sun, Jun 1, 2008 at 2:15 AM, Raymond Hettinger <python at rcn.com> wrote:
>>
>> Proposal
>> --------
>> Remove non-essential abstract methods like __index__, three argument
>> __pow__,
>> __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__,
>> __xor__,
>> __rxor__, __or__, __ror__, and __invert__, numerator, and denominator.
>
> +1 from me.
> I'd support removing all these, minus the exceptions already pointed out
> (__index__, numerator, denominator).  As a (so far incomplete) effort
> to speed up the Decimal type I recently implemented a decimal-based
> integer type;  this type would seem a natural candidate to inherit
> from Integral, but the logical and shift operators above make less sense
> for this type.
> The other odd man out here is three-argument pow;  this *is* a
> method that makes sense for integers without reference to the
> way they're stored.  So maybe this should stay.  (Though I've
> occasionally wondered why three-argument pow is part of the
> core language, rather than being in the standard library.)
> Mark
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From p.f.moore at gmail.com  Thu Jun  5 12:50:32 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 5 Jun 2008 11:50:32 +0100
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <200806051230.33243.steve@pearwood.info>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz>
	<200806051230.33243.steve@pearwood.info>
Message-ID: <79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com>

On 05/06/2008, Steven D'Aprano <steve at pearwood.info> wrote:
> Yes, I don't particularly see the advantage of writing:
>
> "spam %s spam" % (value,)
>
> over
>
> "spam %s spam" % value
>
> Why require the first version?

Because the second breaks if value is a tuple:

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> value = (1,2)
>>> "spam %s spam" % value
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: not all arguments converted during string formatting

Paul.

From ncoghlan at gmail.com  Thu Jun  5 14:19:44 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Jun 2008 22:19:44 +1000
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <484745D7.3000808@canterbury.ac.nz>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>
	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>
	<48466D0C.40709@gmail.com>	<48467E50.9030202@voidspace.org.uk>	<loom.20080604T124312-888@post.gmane.org>	<48469888.4070603@gmail.com>
	<484745D7.3000808@canterbury.ac.nz>
Message-ID: <4847D9E0.4020805@gmail.com>

Greg Ewing wrote:
> Nick Coghlan wrote:
> 
>> - remove support for passing a single value to a format string without 
>> wrapping it in an iterable first
> 
> But won't that clobber a large number of the simple
> use cases that you want to keep %-formatting for?

Note the part of the proposal that allows *any* iterable on the right 
hand side rather than the current insistence on a tuple. So the 
single-value use cases can be boxed easily with a list.

To my mind salvaging %-formatting requires removing the ambiguity over 
whether or not the right hand side will be iterated over.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From fuzzyman at voidspace.org.uk  Thu Jun  5 14:29:21 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 05 Jun 2008 13:29:21 +0100
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <4847D9E0.4020805@gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>	<48466D0C.40709@gmail.com>	<48467E50.9030202@voidspace.org.uk>	<loom.20080604T124312-888@post.gmane.org>	<48469888.4070603@gmail.com>	<484745D7.3000808@canterbury.ac.nz>
	<4847D9E0.4020805@gmail.com>
Message-ID: <4847DC21.8030508@voidspace.org.uk>

Nick Coghlan wrote:
> Greg Ewing wrote:
>> Nick Coghlan wrote:
>>
>>> - remove support for passing a single value to a format string 
>>> without wrapping it in an iterable first
>>
>> But won't that clobber a large number of the simple
>> use cases that you want to keep %-formatting for?
>
> Note the part of the proposal that allows *any* iterable on the right 
> hand side rather than the current insistence on a tuple. So the 
> single-value use cases can be boxed easily with a list.
>
> To my mind salvaging %-formatting requires removing the ambiguity over 
> whether or not the right hand side will be iterated over.
>

But that either needs doing for 3.0 or waiting until 4.0 right?

Personally I only *occasionally* find the tuple interpolation a problem 
and am perfectly happy with the current % string formatting...

Michael Foord

> Cheers,
> Nick.
>


From ncoghlan at gmail.com  Thu Jun  5 14:43:20 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Jun 2008 22:43:20 +1000
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>	<48466D0C.40709@gmail.com><48467E50.9030202@voidspace.org.uk><loom.20080604T124312-888@post.gmane.org><48469888.4070603@gmail.com>
	<33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net>
	<2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1>
Message-ID: <4847DF68.1090205@gmail.com>

Raymond Hettinger wrote:
> From: "Antoine" <solipsis at pitrou.net
>>  For me the problem is not about ditching the % operator for an
>> intuitively-named method like format(). It's the format syntax which has
>> become much more complicated and error-prone without any clear advantage.
> 
> It's seems that way to me too.  But, it may be one of those things that 
> you quickly get used to.

%r is about the worst case for the new syntax relative to the old - two 
characters become 5. It's worth looking at what those extra characters 
buy us though:

{0!r}

{}: Conversion to a bracketed format is what allows us the flexibility 
to permit arbitrary format strings (such as datetime formatting), as 
well as the

0: Explicit positional argument references allow arguments to be re-used 
(not quite sold on this one personally - surely named arguments are even 
better for that?)

!: Explicit separators (: or !) allow the option of flexible 
object-controlled formatting, while still permitting the basic 
formatting of str/repr/ascii if desired.

I'm really starting to wonder if supporting positional arguments to 
str.format() *at all* is a mistake. Maybe we should ditch support for 
positional arguments and just accept a single dictionary as the sole 
parameter to format().

For dictionary formatting, str.format() is a clear winner over 
str.__mod__(). For positional formatting I'm not so sure - if someone 
decided to convert from %-formatting to str.format, would it be such a 
burden to ask them to name their substitution variables in the process?

Silly example:

"%s occurs %s times in this format string" % (2, 2)

"{0} occurs {0} times in this format string".format(2)

"{num} occurs {num} times in this format string".format(dict(num=2))

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Thu Jun  5 14:59:38 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Jun 2008 22:59:38 +1000
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>	
	<48466F63.3020902@gmail.com>
	<ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>
Message-ID: <4847E33A.1050806@gmail.com>

Guido van Rossum wrote:
> Sounds good. (Maybe you want to contribute a patch to threading.py?
> Your implementation notes are important. It could be quite independent
> from PEP 371.)

I created issue 3042 as an RFE to add PEP 8 compliant aliases to the 
threading module (including the notes about how to avoid the runtime 
performance hit on the old names). Jesse may want to reference that from 
the PEP as well.

I'm not sure I'll be able to get to it for the first beta though - I'm 
about to go away for a few days, and plan to enjoy my computer-free 
holiday :)

Cheers,
Nick.

http://bugs.python.org/issue3042

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From jnoller at gmail.com  Thu Jun  5 15:15:51 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 5 Jun 2008 09:15:51 -0400
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <4847E33A.1050806@gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<48466F63.3020902@gmail.com>
	<ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>
	<4847E33A.1050806@gmail.com>
Message-ID: <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>

On Thu, Jun 5, 2008 at 8:59 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>>
>> Sounds good. (Maybe you want to contribute a patch to threading.py?
>> Your implementation notes are important. It could be quite independent
>> from PEP 371.)
>
> I created issue 3042 as an RFE to add PEP 8 compliant aliases to the
> threading module (including the notes about how to avoid the runtime
> performance hit on the old names). Jesse may want to reference that from the
> PEP as well.
>
> I'm not sure I'll be able to get to it for the first beta though - I'm about
> to go away for a few days, and plan to enjoy my computer-free holiday :)
>
> Cheers,
> Nick.
>
> http://bugs.python.org/issue3042

I'll add it to the PEP. I had to take a day to do "real work" so I can
circle back to this today. The continuing outstanding question is if
we should put Processing into 2.6 with the threading-like API or PEP 8
compliant names. Richard has already converted the package to PEP 8
style naming, which means I'll need to add aliases to for the original
API.

Ideally, both threading and processing would loose the non-PEP 8 APIs
in py3k or 2.7

Before I go back to the PEP though - I'd like to see if we can reach
some consensus on the API naming. My personal thought is that for many
tasks, the processing module *is* a drop-in replacement (I had to do
this very thing yesterday) which means that putting it in with an API
which matches the current threading module's API is a Good Thing.
However, the flip side of this is that no one really "likes" the
threading API as-is, so putting the module into the standard library
with the matching API simply adds another "broken" API.

Other than the API naming - I don't think there are any other issues
which need to be addressed, minus some cleanup within the PEP for
consistency.

-Jesse

From ncoghlan at gmail.com  Thu Jun  5 15:20:40 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Jun 2008 23:20:40 +1000
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <g27dq3$t3l$1@ger.gmane.org>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1><90B6514494134BEAB0191E8423A986D6@RaymondLaptop1><ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org>
Message-ID: <4847E828.9060001@gmail.com>

Terry Reedy wrote:
> "Raymond Hettinger" <python at rcn.com> wrote in message 
> news:2745D7EB7C064B16A88E433588021756 at RaymondLaptop1...
> | From: "Guido van Rossum" <guido at python.org>
> | > Unless more folks actually say they agree I don't want to go forward
> | > with this. There was quite a bit of discussion about PEP 3141 and it
> | > was accepted; striking this much from it with virtually no discussion
> | > seems wrong to me.
> |
> | Not sure how to generate more discussion.  It seems self-evident
> | that an abc with lots of abstract methods is inherently less usable
> | and that bitwise operations go beyond the basic notion of "integeriness".
> 
> On reading PEP3141 some months ago and again today, I thought and still do 
> that all the methods that depend on a 2s-complement representation and 
> implementation really belong to an implentation-defined subclass of 
> Integral.  But I am not sure of the purpose of the class and of including 
> such concrete methods in an ABC, and so said nothing ;-).

I think it definitely makes sense to separate out the 
number-as-sequence-of-bits operations from the main Integral ABC. This 
would involve moving:

lshift, rshift, and, or, xor, invert (along with their reversed and 
in-place counterparts)

Note that this leaves the Integral ABC adding only __long__, __index__ 
and 3-argument __pow__ over and above the Rational ABC. If 3-argument 
__pow__ goes (which appears likely), we're left with __long__ and __index__.

However, there's still a few additional public properties and methods 
inherited from higher up in the numeric stack which most existing 
integral types are unlikely to provide: .real, .imag, .conjugate(). 
Unlike the methods being relocated, however, these are absolutely 
trivial for all non-complex numeric types.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Thu Jun  5 15:53:51 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Jun 2008 23:53:51 +1000
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>	
	<48466F63.3020902@gmail.com>	
	<ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>	
	<4847E33A.1050806@gmail.com>
	<4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>
Message-ID: <4847EFEF.6040302@gmail.com>

Jesse Noller wrote:
> However, the flip side of this is that no one really "likes" the
> threading API as-is, so putting the module into the standard library
> with the matching API simply adds another "broken" API.

Provided threading gets a PEP 8 style API in 2.6 (which it looks like it 
is going to), then I don't see a problem with multiprocessing only 
providing the new API.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From steve at pearwood.info  Thu Jun  5 16:05:57 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 6 Jun 2008 00:05:57 +1000
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<200806051230.33243.steve@pearwood.info>
	<79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com>
Message-ID: <200806060005.58301.steve@pearwood.info>

On Thu, 5 Jun 2008 08:50:32 pm Paul Moore wrote:
> On 05/06/2008, Steven D'Aprano <steve at pearwood.info> wrote:
> > Yes, I don't particularly see the advantage of writing:
> >
> > "spam %s spam" % (value,)
> >
> > over
> >
> > "spam %s spam" % value
> >
> > Why require the first version?
>
> Because the second breaks if value is a tuple:

That's a good reason for not using the second form if value is an 
arbitrary object that could be any type at all. But how often does that 
happen in real code? It's rare that the right hand argument can be any 
arbitrary type.

I've looked at my code, and there's not one occasion that I've needed to 
write "foo %s" % (obj,) to guard against obj unexpectedly being a 
tuple. If obj was a tuple, it would fail long before it reached the 
string expression. Of course, your mileage may vary.

As I see it, your argument is one for consistency: since *sometimes* you 
need to wrap the right-hand argument in a tuple, then we should insist 
in *always* wrapping it in a tuple. But of course you can do that right 
now:

>>> n = 1
>>> "number %d" % (n,)  # just in case n is a tuple
'number 1'


Just don't force me to do it.



-- 
Steven

From solipsis at pitrou.net  Thu Jun  5 16:49:35 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 5 Jun 2008 14:49:35 +0000 (UTC)
Subject: [Python-Dev] converting the stdlib to str.format
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>	<48466D0C.40709@gmail.com><48467E50.9030202@voidspace.org.uk><loom.20080604T124312-888@post.gmane.org><48469888.4070603@gmail.com>
	<33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net>
	<2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1>
	<4847DF68.1090205@gmail.com>
Message-ID: <loom.20080605T144705-472@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
> %r is about the worst case for the new syntax relative to the old - two 
> characters become 5.

Well there are other very common "worst cases" - namely %d, %f (and probably
a few less commonly used ones such as %X).

> For dictionary formatting, str.format() is a clear winner over 
> str.__mod__().

Agreed. %(something)s is quirky - you are bound to forget the final "s"
(or whatever other specifier) from time to time.

Antoine.




From guido at python.org  Thu Jun  5 16:54:41 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Jun 2008 07:54:41 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <4847E828.9060001@gmail.com>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
Message-ID: <ca471dc20806050754r245c88bfjb8490b887b8b59eb@mail.gmail.com>

On Thu, Jun 5, 2008 at 6:20 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Terry Reedy wrote:
>>
>> "Raymond Hettinger" <python at rcn.com> wrote in message
>> news:2745D7EB7C064B16A88E433588021756 at RaymondLaptop1...
>> | From: "Guido van Rossum" <guido at python.org>
>> | > Unless more folks actually say they agree I don't want to go forward
>> | > with this. There was quite a bit of discussion about PEP 3141 and it
>> | > was accepted; striking this much from it with virtually no discussion
>> | > seems wrong to me.
>> |
>> | Not sure how to generate more discussion.  It seems self-evident
>> | that an abc with lots of abstract methods is inherently less usable
>> | and that bitwise operations go beyond the basic notion of
>> "integeriness".
>>
>> On reading PEP3141 some months ago and again today, I thought and still do
>> that all the methods that depend on a 2s-complement representation and
>> implementation really belong to an implentation-defined subclass of
>> Integral.  But I am not sure of the purpose of the class and of including
>> such concrete methods in an ABC, and so said nothing ;-).
>
> I think it definitely makes sense to separate out the
> number-as-sequence-of-bits operations from the main Integral ABC. This would
> involve moving:
>
> lshift, rshift, and, or, xor, invert (along with their reversed and in-place
> counterparts)

Agreed. Let's move these into a separate BinaryInteger class.

> Note that this leaves the Integral ABC adding only __long__, __index__ and
> 3-argument __pow__ over and above the Rational ABC. If 3-argument __pow__
> goes (which appears likely), we're left with __long__ and __index__.

Let's ditch 3-arg pow, but keep __long__ (in 2.6) and __index__.
Actually __long__ can go too.

> However, there's still a few additional public properties and methods
> inherited from higher up in the numeric stack which most existing integral
> types are unlikely to provide: .real, .imag, .conjugate(). Unlike the
> methods being relocated, however, these are absolutely trivial for all
> non-complex numeric types.

I definitely want to keep these. They're essential for people who want
to use the higher-up classes in the numeric tower.

I think this is settled now; Raymond can update PEP 3141 (if he
refrains from editorializing) and patch numbers.py.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From theller at ctypes.org  Thu Jun  5 17:08:10 2008
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 05 Jun 2008 17:08:10 +0200
Subject: [Python-Dev] PEP3118, python 2.6
Message-ID: <4848015A.5080009@ctypes.org>

Hi Travis,

I'm currently trying to port the pep3118 ctypes changes which are already in
the py3k branch to the trunk.

In py3k the tests for this stuff (in Lib/ctypes/test/test_pep3118.py) use
the memoryview object which exposes attributes like .format, .shape, .strides
and so on for objects implementing the new buffer interface.

In Python 2.6 memoryview does not exist so the question is how to write a test
that checks for the correct attributes.  My idea is to implement the
__array_interface__ property for ctypes instances, as described in this document
http://numpy.scipy.org/array_interface.shtml.

Do you think this is a good idea, or would it be better to invent another,
private, mechanism for the tests?  In other words:  Would the __array_interface__
property (version 3) on these objects be useful for external code or would
it disrupt external code?

BTW: Did you have a chance to look over the test_pep3118 module in py3k,
to make sure that I got things correctly (it should be pretty easy to read, imo)?
-- 
Thanks,
Thomas

From jnoller at gmail.com  Thu Jun  5 17:10:41 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 5 Jun 2008 11:10:41 -0400
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <4847EFEF.6040302@gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<48466F63.3020902@gmail.com>
	<ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>
	<4847E33A.1050806@gmail.com>
	<4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>
	<4847EFEF.6040302@gmail.com>
Message-ID: <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com>

On Thu, Jun 5, 2008 at 9:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Jesse Noller wrote:
>>
>> However, the flip side of this is that no one really "likes" the
>> threading API as-is, so putting the module into the standard library
>> with the matching API simply adds another "broken" API.
>
> Provided threading gets a PEP 8 style API in 2.6 (which it looks like it is
> going to), then I don't see a problem with multiprocessing only providing
> the new API.
>
> Cheers,
> Nick.
>

Fantastic. If no one has any other arguments to the contrary, I'll run
with the new API naming scheme.

Now I have a lot of tests and documentation to rewrite.

-jesse

From steve at pearwood.info  Thu Jun  5 17:21:22 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 6 Jun 2008 01:21:22 +1000
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <4847DF68.1090205@gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1>
	<4847DF68.1090205@gmail.com>
Message-ID: <200806060121.22560.steve@pearwood.info>

On Thu, 5 Jun 2008 10:43:20 pm Nick Coghlan wrote:

> I'm really starting to wonder if supporting positional arguments to
> str.format() *at all* is a mistake. Maybe we should ditch support for
> positional arguments and just accept a single dictionary as the sole
> parameter to format().
>
> For dictionary formatting, str.format() is a clear winner over
> str.__mod__(). For positional formatting I'm not so sure - if someone
> decided to convert from %-formatting to str.format, would it be such
> a burden to ask them to name their substitution variables in the
> process?

If positional arguments are dropped, I would expect to see a 
proliferation of meaningless names used in the dicts:

"items {a} {b}".format(dict(a=spam, b=eggs))

In other words, the names will be just aliases for the position.

I would also expect a performance hit. On my computer, dict(a=spam, 
b=eggs) is two times slower than {'a':spam, 'b':eggs} and nearly three 
times slower than (spam, eggs). I don't imagine building a dict will 
ever be faster than building a tuple. Given that format() is already 
significantly slower than %, why make it slower still?

-1 on dropping positional arguments.


-- 
Steven

From theller at ctypes.org  Thu Jun  5 17:42:45 2008
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 05 Jun 2008 17:42:45 +0200
Subject: [Python-Dev] PEP3118, python 2.6
In-Reply-To: <4848015A.5080009@ctypes.org>
References: <4848015A.5080009@ctypes.org>
Message-ID: <g291hm$r22$1@ger.gmane.org>

Thomas Heller schrieb:
> I'm currently trying to port the pep3118 ctypes changes which are already in
> the py3k branch to the trunk.
> 
> In py3k the tests for this stuff (in Lib/ctypes/test/test_pep3118.py) use
> the memoryview object which exposes attributes like .format, .shape, .strides
> and so on for objects implementing the new buffer interface.
> 
> In Python 2.6 memoryview does not exist so the question is how to write a test
> that checks for the correct attributes.  My idea is to implement the
> __array_interface__ property for ctypes instances, as described in this document
> http://numpy.scipy.org/array_interface.shtml.

In private email Travis told me that he has no time to backport the memoryview
object to Python 2.6.  Maybe there is someone who could do this (Myself I have
no time either for this)?

Thomas


From musiccomposition at gmail.com  Thu Jun  5 18:23:15 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 5 Jun 2008 11:23:15 -0500
Subject: [Python-Dev] on Python's tests (and making them better)
Message-ID: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>

Hi,
This summer, I am going to be revamping Python's test suite. Major
things I plan to do include
- rewriting regrtest.py to be a simple test driver
- implementing CPython only decorators
- moving skipping data to the tests themselves (perhaps with decorators)
- completing the transition to unittest/docttest
- removing old test_support crud (like verify)
- increasing the testing of unicode inputs and fixing bugs the appear
- write new testing docs for newbie contributors
- merging related tests
- reorganizing the tests into separate directories

Other suggestions?

I also wanted to ask about the API compatibility requirements for
test.test_support. IMO, we shouldn't include the testing tools as part
of the stdlib. However, test_support is documented.

Does anyone know any projects using test_support?

-- 
Thanks,
Benjamin Peterson
"There's no place like 127.0.0.1."

From fijall at gmail.com  Thu Jun  5 18:41:33 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 5 Jun 2008 18:41:33 +0200
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
Message-ID: <693bc9ab0806050941q10a73354iccca0d7694ed9a5e@mail.gmail.com>

On Thu, Jun 5, 2008 at 6:23 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> Hi,
> This summer, I am going to be revamping Python's test suite. Major
> things I plan to do include
> - rewriting regrtest.py to be a simple test driver
> - implementing CPython only decorators
> - moving skipping data to the tests themselves (perhaps with decorators)
> - completing the transition to unittest/docttest
> - removing old test_support crud (like verify)
> - increasing the testing of unicode inputs and fixing bugs the appear
> - write new testing docs for newbie contributors
> - merging related tests
> - reorganizing the tests into separate directories
>
> Other suggestions?

Cool! I can help you with some info which tests are unlikely to pass
on top of ie pypy (and *we* decided those are implementation-specific,
but I might be wrong).

Cheers,
fijal

From janssen at parc.com  Thu Jun  5 18:41:44 2008
From: janssen at parc.com (Bill Janssen)
Date: Thu, 5 Jun 2008 09:41:44 PDT
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <20080604222416.GD26992@mcnabbs.org> 
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<ca471dc20806041350w78d39281i86b84695ccf13bad@mail.gmail.com>
	<20080604222416.GD26992@mcnabbs.org>
Message-ID: <08Jun5.094145pdt."58698"@synergy1.parc.xerox.com>

> I think I agree with Raymond on the basic principle that simple ABC's
> are easier to use than simple ones.

I don't think he was saying that.  He was saying that simple ABC's are
easier to implement to.

It's not at all clear to me that simple ABC's are good in and of
themselves.  I think a String ABC should probably include all the
methods that basestring provides, perhaps by reducing the methods that
basestring provides, and moving the removed methods to the String ABC,
with implementations that call the methods provided by basestring.
That way, String classes could be implemented by overriding the "base"
set of methods, but still inherit from String to get the other
methods.  But, then, isn't basestring the true simple ABC for strings?

Bill

From amcnabb at mcnabbs.org  Thu Jun  5 18:57:13 2008
From: amcnabb at mcnabbs.org (Andrew McNabb)
Date: Thu, 5 Jun 2008 10:57:13 -0600
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <08Jun5.094145pdt."58698"@synergy1.parc.xerox.com>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<ca471dc20806041350w78d39281i86b84695ccf13bad@mail.gmail.com>
	<20080604222416.GD26992@mcnabbs.org>
	<08Jun5.094145pdt."58698"@synergy1.parc.xerox.com>
Message-ID: <20080605165713.GH30223@mcnabbs.org>

On Thu, Jun 05, 2008 at 09:41:44AM -0700, Bill Janssen wrote:
> > I think I agree with Raymond on the basic principle that simple ABC's
> > are easier to use than simple ones.
> 
> I don't think he was saying that.  He was saying that simple ABC's are
> easier to implement to.

Sorry; I used the wrong word.  I should have said "implement to" rather
than "use." I agree with you.


-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080605/ac457be7/attachment.pgp>

From guido at python.org  Thu Jun  5 19:22:13 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Jun 2008 10:22:13 -0700
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<48466F63.3020902@gmail.com>
	<ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>
	<4847E33A.1050806@gmail.com>
	<4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>
	<4847EFEF.6040302@gmail.com>
	<4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com>
Message-ID: <ca471dc20806051022o44828883id5dee70266d32f46@mail.gmail.com>

I've accepted your PEP. I think it still needs some clean-up and
perhaps clarification of the agreement reached about API style, but
there is nothing now that keeps you from implementing it! Hopefully
you'll make the beta release early next week.

--Guido

On Thu, Jun 5, 2008 at 8:10 AM, Jesse Noller <jnoller at gmail.com> wrote:
> On Thu, Jun 5, 2008 at 9:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Jesse Noller wrote:
>>>
>>> However, the flip side of this is that no one really "likes" the
>>> threading API as-is, so putting the module into the standard library
>>> with the matching API simply adds another "broken" API.
>>
>> Provided threading gets a PEP 8 style API in 2.6 (which it looks like it is
>> going to), then I don't see a problem with multiprocessing only providing
>> the new API.
>>
>> Cheers,
>> Nick.
>>
>
> Fantastic. If no one has any other arguments to the contrary, I'll run
> with the new API naming scheme.
>
> Now I have a lot of tests and documentation to rewrite.
>
> -jesse
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dalcinl at gmail.com  Thu Jun  5 19:28:36 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 5 Jun 2008 14:28:36 -0300
Subject: [Python-Dev] PEP3118, python 2.6
In-Reply-To: <g291hm$r22$1@ger.gmane.org>
References: <4848015A.5080009@ctypes.org> <g291hm$r22$1@ger.gmane.org>
Message-ID: <e7ba66e40806051028h399bfc44h5847fb2ccbfdfd31@mail.gmail.com>

On 6/5/08, Thomas Heller <theller at ctypes.org> wrote:
Thomas, Iff this helps, you have attached the backport but as an
extension module.  Sorry, I do not have time for the full backport.
But at least, I've found that the Py3K implementation works

You will need to build it with 'python2.6 setup.py build' and copy
'memvwr.so' somewhere in your $PYTHONPATH, then in you 'site.py' or
whatever, you can do 'import memvwr; del memvwr' . This will
automaticall add the module to the __builtin__ module.

Some notes:

* I had to add Py_TPFLAGS_HAVE_NEWBUFFER to the type object structure.
Should'nt this be already included in Py_TPFLAGS_DEFAULT ?

* I have to monkey-copy two auxiliary functions 'abstrac.c' for Py3k,
as ther are employed in memoryobject.c Travis, I believe you should
have to review this. I'm not sure iff that is the right approach,
regarding symbol visibility in shared libs in different OS's,
compilers, etc.. But perhaps all is OK. Just a warning about possible
problems :-).

* I've not implemented the former buffer interface, I just put NULL's
for the slots. Not sure if implementing the former interface is relly
needed.

* I've used PyString_XXX API calls.  Not sure what will be the final
status of the C API cleanup for Py2.6. Perhaps this should be replaced
with PyBytes_XXXX

> Thomas Heller schrieb:
>
> > I'm currently trying to port the pep3118 ctypes changes which are already in
>  > the py3k branch to the trunk.
>  >
>  > In py3k the tests for this stuff (in Lib/ctypes/test/test_pep3118.py) use
>  > the memoryview object which exposes attributes like .format, .shape, .strides
>  > and so on for objects implementing the new buffer interface.
>  >
>  > In Python 2.6 memoryview does not exist so the question is how to write a test
>  > that checks for the correct attributes.  My idea is to implement the
>  > __array_interface__ property for ctypes instances, as described in this document
>  > http://numpy.scipy.org/array_interface.shtml.
>
>
> In private email Travis told me that he has no time to backport the memoryview
>  object to Python 2.6.  Maybe there is someone who could do this (Myself I have
>  no time either for this)?
>
>
>  Thomas
>
>  _______________________________________________
>  Python-Dev mailing list
>  Python-Dev at python.org
>  http://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe: http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com
>


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
-------------- next part --------------
A non-text attachment was scrubbed...
Name: memoryview-py2.6.tar.gz
Type: application/x-gzip
Size: 6427 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080605/b1c7df6e/attachment.bin>

From jnoller at gmail.com  Thu Jun  5 19:48:12 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 5 Jun 2008 13:48:12 -0400
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <ca471dc20806051022o44828883id5dee70266d32f46@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<48466F63.3020902@gmail.com>
	<ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>
	<4847E33A.1050806@gmail.com>
	<4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>
	<4847EFEF.6040302@gmail.com>
	<4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com>
	<ca471dc20806051022o44828883id5dee70266d32f46@mail.gmail.com>
Message-ID: <4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com>

I'll have a cleaned version to you by end of day, and by day, I mean
by 10:00 EST :)

On Thu, Jun 5, 2008 at 1:22 PM, Guido van Rossum <guido at python.org> wrote:
> I've accepted your PEP. I think it still needs some clean-up and
> perhaps clarification of the agreement reached about API style, but
> there is nothing now that keeps you from implementing it! Hopefully
> you'll make the beta release early next week.
>
> --Guido
>
> On Thu, Jun 5, 2008 at 8:10 AM, Jesse Noller <jnoller at gmail.com> wrote:
>> On Thu, Jun 5, 2008 at 9:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> Jesse Noller wrote:
>>>>
>>>> However, the flip side of this is that no one really "likes" the
>>>> threading API as-is, so putting the module into the standard library
>>>> with the matching API simply adds another "broken" API.
>>>
>>> Provided threading gets a PEP 8 style API in 2.6 (which it looks like it is
>>> going to), then I don't see a problem with multiprocessing only providing
>>> the new API.
>>>
>>> Cheers,
>>> Nick.
>>>
>>
>> Fantastic. If no one has any other arguments to the contrary, I'll run
>> with the new API naming scheme.
>>
>> Now I have a lot of tests and documentation to rewrite.
>>
>> -jesse
>>
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

From musiccomposition at gmail.com  Thu Jun  5 20:54:01 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 5 Jun 2008 13:54:01 -0500
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<48466F63.3020902@gmail.com>
	<ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>
	<4847E33A.1050806@gmail.com>
	<4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>
	<4847EFEF.6040302@gmail.com>
	<4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com>
	<ca471dc20806051022o44828883id5dee70266d32f46@mail.gmail.com>
	<4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com>
Message-ID: <1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com>

On Thu, Jun 5, 2008 at 12:48 PM, Jesse Noller <jnoller at gmail.com> wrote:
> I'll have a cleaned version to you by end of day, and by day, I mean
> by 10:00 EST :)

Great! Please post that on the bug tracker when you're ready.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From martin at v.loewis.de  Thu Jun  5 21:32:00 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Jun 2008 21:32:00 +0200
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <loom.20080605T144705-472@post.gmane.org>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>	<48466D0C.40709@gmail.com><48467E50.9030202@voidspace.org.uk><loom.20080604T124312-888@post.gmane.org><48469888.4070603@gmail.com>	<33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net>	<2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1>	<4847DF68.1090205@gmail.com>
	<loom.20080605T144705-472@post.gmane.org>
Message-ID: <48483F30.6030101@v.loewis.de>

> Agreed. %(something)s is quirky - you are bound to forget the final "s"
> (or whatever other specifier) from time to time.

But that gives a ValueError the first time you try it, no?

Regards,
Martin

From theller at ctypes.org  Thu Jun  5 22:05:30 2008
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 05 Jun 2008 22:05:30 +0200
Subject: [Python-Dev] PEP3118, python 2.6
In-Reply-To: <e7ba66e40806051028h399bfc44h5847fb2ccbfdfd31@mail.gmail.com>
References: <4848015A.5080009@ctypes.org> <g291hm$r22$1@ger.gmane.org>
	<e7ba66e40806051028h399bfc44h5847fb2ccbfdfd31@mail.gmail.com>
Message-ID: <g29gub$lml$1@ger.gmane.org>

Lisandro Dalcin schrieb:
> On 6/5/08, Thomas Heller <theller at ctypes.org> wrote:
> Thomas, Iff this helps, you have attached the backport but as an
> extension module.  Sorry, I do not have time for the full backport.
> But at least, I've found that the Py3K implementation works

Thanks for the effort.  Maybe I will look into that later.  (And, if
you want to make sure that your patch doesn't get lost and hopefully
reviewed you should add it to the bug tracker.)

In the meantime I found that I already had a _ctypes private function
that can expose the needed info to Python, for ctypes pep3118 testing.

Thanks again,
Thomas


From jnoller at gmail.com  Thu Jun  5 23:44:36 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 5 Jun 2008 17:44:36 -0400
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<48466F63.3020902@gmail.com>
	<ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>
	<4847E33A.1050806@gmail.com>
	<4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>
	<4847EFEF.6040302@gmail.com>
	<4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com>
	<ca471dc20806051022o44828883id5dee70266d32f46@mail.gmail.com>
	<4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com>
	<1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com>
Message-ID: <4222a8490806051444qd561e68j72f034d214412aa@mail.gmail.com>

On Thu, Jun 5, 2008 at 2:54 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Thu, Jun 5, 2008 at 12:48 PM, Jesse Noller <jnoller at gmail.com> wrote:
>> I'll have a cleaned version to you by end of day, and by day, I mean
>> by 10:00 EST :)
>
> Great! Please post that on the bug tracker when you're ready.
>
>
> --
> Cheers,
> Benjamin Peterson
> "There's no place like 127.0.0.1."
>

I meant a cleaned version of the PEP - I still have docs and tests to redo

From ncoghlan at gmail.com  Fri Jun  6 04:06:41 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 06 Jun 2008 12:06:41 +1000
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <4222a8490806051444qd561e68j72f034d214412aa@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>	
	<48466F63.3020902@gmail.com>	
	<ca471dc20806041042k2926b129ue0c519413cff7318@mail.gmail.com>	
	<4847E33A.1050806@gmail.com>	
	<4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>	
	<4847EFEF.6040302@gmail.com>	
	<4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com>	
	<ca471dc20806051022o44828883id5dee70266d32f46@mail.gmail.com>	
	<4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com>	
	<1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com>
	<4222a8490806051444qd561e68j72f034d214412aa@mail.gmail.com>
Message-ID: <48489BB1.7080702@gmail.com>

Jesse Noller wrote:
> I meant a cleaned version of the PEP - I still have docs and tests to redo
> 
It would also be good if you could check Benjamin's patch on issue 3402 
to give threading a PEP 8 compliant API and make sure the names are the 
same as the new names in multiprocessing.

Cheers,
Nick.

P.S. OK, now I really am going to be offline for the next few days ;)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Fri Jun  6 04:07:53 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 06 Jun 2008 12:07:53 +1000
Subject: [Python-Dev] PEP3118, python 2.6
In-Reply-To: <g29gub$lml$1@ger.gmane.org>
References: <4848015A.5080009@ctypes.org>
	<g291hm$r22$1@ger.gmane.org>	<e7ba66e40806051028h399bfc44h5847fb2ccbfdfd31@mail.gmail.com>
	<g29gub$lml$1@ger.gmane.org>
Message-ID: <48489BF9.3070407@gmail.com>

Thomas Heller wrote:
> Lisandro Dalcin schrieb:
>> On 6/5/08, Thomas Heller <theller at ctypes.org> wrote:
>> Thomas, Iff this helps, you have attached the backport but as an
>> extension module.  Sorry, I do not have time for the full backport.
>> But at least, I've found that the Py3K implementation works
> 
> Thanks for the effort.  Maybe I will look into that later.  (And, if
> you want to make sure that your patch doesn't get lost and hopefully
> reviewed you should add it to the bug tracker.)

There's an existing issue for the memoryview backport. Maybe it's 
priority should be bumped up a bit?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From jnoller at gmail.com  Fri Jun  6 04:08:35 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 5 Jun 2008 22:08:35 -0400
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <48489BB1.7080702@gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<4847E33A.1050806@gmail.com>
	<4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>
	<4847EFEF.6040302@gmail.com>
	<4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com>
	<ca471dc20806051022o44828883id5dee70266d32f46@mail.gmail.com>
	<4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com>
	<1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com>
	<4222a8490806051444qd561e68j72f034d214412aa@mail.gmail.com>
	<48489BB1.7080702@gmail.com>
Message-ID: <4222a8490806051908r20c93293v2761ed61c70dbeef@mail.gmail.com>

On Thu, Jun 5, 2008 at 10:06 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Jesse Noller wrote:
>>
>> I meant a cleaned version of the PEP - I still have docs and tests to redo
>>
> It would also be good if you could check Benjamin's patch on issue 3402 to
> give threading a PEP 8 compliant API and make sure the names are the same as
> the new names in multiprocessing.
>
> Cheers,
> Nick.
>
> P.S. OK, now I really am going to be offline for the next few days ;)
>

Good idea, I'll add that to the queue - right now I'm focusing on
getting the module into the build on trunk (locally) and getting all
of the docs/tests updated and up to par.

-jesse

From jimjjewett at gmail.com  Fri Jun  6 05:19:34 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 5 Jun 2008 23:19:34 -0400
Subject: [Python-Dev] PEP 8 vs PEP 371: Additional Discussion
Message-ID: <fb6fbf560806052019v46948c8eyc9d1d68694f6f0c9@mail.gmail.com>

Guido van Rossum wrote:

> I consider multiprocessing a new API -- while it bears
> a superficial resemblance with the threading API the
> similarities are just that, and it should not be
> constrained by mistakes in that API.

The justification for including it is precisely that it is *not* a new API.

For multiple processes in general, there are competing APIs, which may
well be better.  The advantage of this API is that (in many cases) it
is a drop-in replacement for threading.  If that breaks, then there
really isn't any reason to include it in the stdlib yet.

This doesn't prevent changing the joint API to conform with PEP 8.
But why clean this module while leaving the rest of the stdlib?

"Because there is a volunteer" only makes sense if changes to the
other modules would also be welcomed.  Is there some reason to believe
that changes in the threading API are much less disruptive than
changes elsewhere in the stdlib?

-jJ

From greg.ewing at canterbury.ac.nz  Fri Jun  6 05:32:35 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Jun 2008 15:32:35 +1200
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz>
	<200806051230.33243.steve@pearwood.info>
	<79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com>
Message-ID: <4848AFD3.5020606@canterbury.ac.nz>

Paul Moore wrote:

> Because the second breaks if value is a tuple:

However, changing it now is going to break a huge
amount of existing code that uses %-formatting,
and in ways that 2to3 can't reliably fix.

Keeping %-formatting but breaking a large
proportion of its uses doesn't seem like a good
idea to me.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Jun  6 05:37:50 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Jun 2008 15:37:50 +1200
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <4847DF68.1090205@gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<4844F5B4.5090703@v.loewis.de> <g22unc$hbu$1@ger.gmane.org>
	<4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com>
	<48467E50.9030202@voidspace.org.uk>
	<loom.20080604T124312-888@post.gmane.org>
	<48469888.4070603@gmail.com>
	<33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net>
	<2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1>
	<4847DF68.1090205@gmail.com>
Message-ID: <4848B10E.5000902@canterbury.ac.nz>

Nick Coghlan wrote:
> %r is about the worst case for the new syntax relative to the old - two 
> characters become 5. It's worth looking at what those extra characters 
> buy us though:

However, those benefits are only realised some of the
time, and it's only in rare cases that they're all
realised at once. The cost-benefit analysis needs to
take the entropy into account somehow.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Jun  6 05:39:33 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Jun 2008 15:39:33 +1200
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <4847DF68.1090205@gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<4844F5B4.5090703@v.loewis.de> <g22unc$hbu$1@ger.gmane.org>
	<4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com>
	<48467E50.9030202@voidspace.org.uk>
	<loom.20080604T124312-888@post.gmane.org>
	<48469888.4070603@gmail.com>
	<33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net>
	<2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1>
	<4847DF68.1090205@gmail.com>
Message-ID: <4848B175.9050307@canterbury.ac.nz>

Nick Coghlan wrote:
> Maybe we should ditch support for 
> positional arguments and just accept a single dictionary as the sole 
> parameter to format().
> 
> "{num} occurs {num} times in this format string".format(dict(num=2))

If named arguments were to become mandatory, I'd want
to be able to write that as

   "...{num}...".format(num = 2)

-- 
Greg

From alexandre at peadrop.com  Fri Jun  6 05:43:59 2008
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Thu, 5 Jun 2008 23:43:59 -0400
Subject: [Python-Dev] [Python-3000] How to specify keyword-only
	arguments from C?
In-Reply-To: <acd65fa20806052018h510d6244m3db0b2ff953dac5a@mail.gmail.com>
References: <acd65fa20806051347u11250050pc23c1a9628170d54@mail.gmail.com>
	<02ee01c8c77b$1553d6e0$3ffb84a0$@com.au>
	<acd65fa20806052018h510d6244m3db0b2ff953dac5a@mail.gmail.com>
Message-ID: <acd65fa20806052043u2ba15998n21fea0263cb47c37@mail.gmail.com>

On Thu, Jun 5, 2008 at 11:18 PM, Alexandre Vassalotti
<alexandre at peadrop.com> wrote:
> On Thu, Jun 5, 2008 at 10:14 PM, Mark Hammond <mhammond at skippinet.com.au> wrote:
>> Set an error if the 'arg' tuple doesn't have a length of zero?
>>
>
> Oh, that isn't a bad idea at all. I will try this. Thanks!
>

Worked flawlessly!

Just for the archives, here's how it looks like:

static int
Unpickler_init(UnpicklerObject *self, PyObject *args, PyObject *kwds)
{
    static char *kwlist[] = {"file", "encoding", "errors", 0};
    PyObject *file;
    char *encoding = NULL;
    char *errors = NULL;

    if (Py_SIZE(args) != 1) {
        PyErr_Format(PyExc_TypeError,
                     "%s takes exactly one 1 positional argument (%zd given)",
                     Py_TYPE(self)->tp_name, Py_SIZE(args));
        return -1;
    }

    if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|ss:Unpickler", kwlist,
                                     &file, &encoding, &errors))
        return -1;
...


Thank you, Mark, for the tip!

-- Alexandre

From python at rcn.com  Fri Jun  6 05:45:27 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 5 Jun 2008 20:45:27 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1><90B6514494134BEAB0191E8423A986D6@RaymondLaptop1><ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1><g27dq3$t3l$1@ger.gmane.org>
	<4847E828.9060001@gmail.com>
Message-ID: <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>

[Terry Reedy]
>> On reading PEP3141 some months ago and again today, I thought and still do 
>> that all the methods that depend on a 2s-complement representation and 
>> implementation really belong to an implentation-defined subclass of 
>> Integral.  But I am not sure of the purpose of the class and of including 
>> such concrete methods in an ABC, and so said nothing ;-).

[Nick Coghlan]
> I think it definitely makes sense to separate out the 
> number-as-sequence-of-bits operations from the main Integral ABC. This 
> would involve moving:
> 
> lshift, rshift, and, or, xor, invert (along with their reversed and 
> in-place counterparts)

I vote for complete removal of the bit flipping methods.

There's a cost to creating new ABCs that are devoid of use cases.
When Terry says "I am not sure of the purpose of the class ...", I think he 
meant that a binary abc wasn't proposed because of its questionable utility.

At some point, if you want an int type, you just use an int type.
What is the purpose of having yet another layer in the "numeric tower"?
Does anyone actually need an int lookalike with binary methods but
cannot just inherit from int?

Looking back at PEP 3119, I see there was an early decision
"between standardizing more, fine-grained ABCs or fewer, course-grained
ones" and that the choice was resolved in favor the few.  That is how we 
avoided urban sprawl with the likes of "ComposableSet, MutableSet, 
HashableSet, MutableComposableSet, HashableComposableSet".

I had thought everyone was on-board with the notion that thin abcs
that capture the essence of a type are better than fat abcs
that seek emulate every aspect of a single concrete type.
Without agreement on that premise, simplification is a lost cause.

If the group wants to go forward with a Binary subclass of Integral,
then I would like to withdraw this simplification mini-pep.  Better
to stick with what we have now than to addcomplexity to a module
that is already use case challenged.


Raymond


From jimjjewett at gmail.com  Fri Jun  6 05:39:39 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 5 Jun 2008 23:39:39 -0400
Subject: [Python-Dev] Mini-Pep: An Empty String ABC
Message-ID: <fb6fbf560806052039w72c273aep232c622612e629d7@mail.gmail.com>

> So, apart from compatibility purposes, what is the
> point currently of *not* directly subclassing str?

To provide your own storage format, such as a views into existing data.

Whether or not this is actually practical is a different question;
plenty C code tends to assume it can use the internals of str
directly, which breaks on even some subclasses.

-jJ

From g.brandl at gmx.net  Fri Jun  6 13:08:25 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 06 Jun 2008 11:08:25 +0000
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <4848B175.9050307@canterbury.ac.nz>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>
	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>
	<48466D0C.40709@gmail.com>	<48467E50.9030202@voidspace.org.uk>	<loom.20080604T124312-888@post.gmane.org>	<48469888.4070603@gmail.com>	<33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net>	<2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1>	<4847DF68.1090205@gmail.com>
	<4848B175.9050307@canterbury.ac.nz>
Message-ID: <g2auqt$g4f$1@ger.gmane.org>

Greg Ewing schrieb:
> Nick Coghlan wrote:
>> Maybe we should ditch support for 
>> positional arguments and just accept a single dictionary as the sole 
>> parameter to format().
>> 
>> "{num} occurs {num} times in this format string".format(dict(num=2))
> 
> If named arguments were to become mandatory, I'd want
> to be able to write that as
> 
>    "...{num}...".format(num = 2)

You already are -- in fact, that is the only way this works.
(I'd wondered why nobody brought this error up in this discussion
before. Probably it wouldn't have been ugly enough.)

I don't see a significant speed difference between positional and keyword
arguments to format() when testing with two arguments.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From g.brandl at gmx.net  Fri Jun  6 13:12:27 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 06 Jun 2008 11:12:27 +0000
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <4848AFD3.5020606@canterbury.ac.nz>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<48469888.4070603@gmail.com>
	<484745D7.3000808@canterbury.ac.nz>	<200806051230.33243.steve@pearwood.info>	<79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com>
	<4848AFD3.5020606@canterbury.ac.nz>
Message-ID: <g2av2b$g4f$2@ger.gmane.org>

Greg Ewing schrieb:
> Paul Moore wrote:
> 
>> Because the second breaks if value is a tuple:
> 
> However, changing it now is going to break a huge
> amount of existing code that uses %-formatting,
> and in ways that 2to3 can't reliably fix.
> 
> Keeping %-formatting but breaking a large
> proportion of its uses doesn't seem like a good
> idea to me.

Exactly. If % formatting is removed in 3k, code breaks, so we keep it.
Changing it now so that code breaks anyway is not productive :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From mal at egenix.com  Fri Jun  6 11:19:29 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 06 Jun 2008 11:19:29 +0200
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com>
References: <48397ECC.9070805@cheimes.de>
	<483ECA52.6040000@egenix.com>	<483ECF94.7060607@cheimes.de>
	<483EF139.8000606@egenix.com>	<483F34C3.3050402@gmail.com>
	<483FBCB4.5020007@egenix.com>	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>	<4843E884.1060705@egenix.com>	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>	<ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>
	<52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com>
Message-ID: <48490121.5030701@egenix.com>

On 2008-06-03 01:29, Gregory P. Smith wrote:
> On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum <guido at python.org> wrote:
> 
>> I will freely admit that I haven't followed this thread in any detail,
>> but if it were up to me, I'd have the 2.6 internal code use PyString
> 
> ...
> 
> Should we read this as a BDFL pronouncement and make it so?
> 
> All that would mean change wise is that trunk r63675 as well as possibly
> r63672 and r63677 would need to be rolled back and this whole discussion
> over if such a big change should have happened would turn into a moot point.

I would certainly welcome reverting the change.

All that's needed to support PyBytes API in 2.x is a set of #defines
that map the new APIs to the PyString names. That's a clean and
easily understandable solution.

Programmers interested in the code
for a PyString API can then still look up the code in stringobject.c,
e.g. to find out how a certain special case is handled or to check
the ref counting - just like they did for years.

Developer who want to start differentiating between mixed byte/text
data and bytes-only can start using PyBytes for byte data.

>> I would also add macros that map the PyBytes_* APIs to PyString_*, but
>> I would not start using these internally except in code newly written
>> for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString
>> for 8-bit strings containing text, and PyBytes for 8-bit strings
>> containing binary data. For 8-bit strings that could contain either
>> text or data, I'd use PyString, in the spirit of 2.x.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 06 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            30 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From g.brandl at gmx.net  Fri Jun  6 13:27:29 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 06 Jun 2008 11:27:29 +0000
Subject: [Python-Dev] Modules for 2.6 inclusion
Message-ID: <g2avui$jjo$1@ger.gmane.org>

Hi,

PEP 361 lists the following modules for possible inclusion in 2.6 (next
to pyprocessing, which is now accepted):

- winerror
   http://python.org/sf/1505257
   (Owner: MAL)

This patch has been marked as rejected, so I'll remove the entry from
the PEP.

- setuptools
   BDFL pronouncement for inclusion in 2.5:
   http://mail.python.org/pipermail/python-dev/2006-April/063964.html

   PJE's withdrawal from 2.5 for inclusion in 2.6:
   http://mail.python.org/pipermail/python-dev/2006-April/064145.html

I guess this will be deferred?

- ast
   http://mail.python.org/pipermail/python-dev/2008-April/078950.html

If there's no objection, I'll go over the interface with Thomas, who's
working on AST optimization for 2.7, to make sure the ast interface can
stay the same after his branch is merged, write the docs and commit it
before beta1.

- bdist_deb in distutils package
   http://mail.python.org/pipermail/python-dev/2006-February/060926.html

- bdist_egg in distutils package

- pure python pgen module
   (Owner: Guido)
   Deferral to 2.6:
   http://mail.python.org/pipermail/python-dev/2006-April/064528.html


There are also several other "possible" todo items in PEP 361, but they
all look as if they are not required to be in before beta1.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From p.f.moore at gmail.com  Fri Jun  6 11:31:11 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 6 Jun 2008 10:31:11 +0100
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <g2av2b$g4f$2@ger.gmane.org>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz>
	<200806051230.33243.steve@pearwood.info>
	<79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com>
	<4848AFD3.5020606@canterbury.ac.nz> <g2av2b$g4f$2@ger.gmane.org>
Message-ID: <79990c6b0806060231k4316e21aof9b9e3103d64c79@mail.gmail.com>

On 06/06/2008, Georg Brandl <g.brandl at gmx.net> wrote:
> Greg Ewing schrieb:
> > Paul Moore wrote:
> >
> >
> > > Because the second breaks if value is a tuple:
> > >
> >
> > However, changing it now is going to break a huge
> > amount of existing code that uses %-formatting,
> > and in ways that 2to3 can't reliably fix.
> >
> > Keeping %-formatting but breaking a large
> > proportion of its uses doesn't seem like a good
> > idea to me.
> >
>
> Exactly. If % formatting is removed in 3k, code breaks, so we keep it.
> Changing it now so that code breaks anyway is not productive :)

Just to clarify, I'm not advocating change here. My comment (above)
about code breaking was in reference to an issue with a chunk of
user-level code which got clipped from the quote, and is completely
unrelated to any diiscussion of changing Python.

Paul.

From mal at egenix.com  Fri Jun  6 13:07:31 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 06 Jun 2008 13:07:31 +0200
Subject: [Python-Dev] Modules for 2.6 inclusion
In-Reply-To: <g2avui$jjo$1@ger.gmane.org>
References: <g2avui$jjo$1@ger.gmane.org>
Message-ID: <48491A73.7060104@egenix.com>

On 2008-06-06 13:27, Georg Brandl wrote:
> Hi,
> 
> PEP 361 lists the following modules for possible inclusion in 2.6 (next
> to pyprocessing, which is now accepted):
> 
> - winerror
>   http://python.org/sf/1505257
>   (Owner: MAL)
> 
> This patch has been marked as rejected, so I'll remove the entry from
> the PEP.

Note that the idea is still valid - the implementation of the module
should be written in C and the patch only comes with a Python module.

If anyone would like to work on a (generated) C module, please have
a look at the patch.

winerror is meant to provide access to the Windows error codes which
are currently not available in Python.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 06 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            30 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From steve at holdenweb.com  Fri Jun  6 13:14:07 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 06 Jun 2008 07:14:07 -0400
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <4847D9E0.4020805@gmail.com>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>	<4844F5B4.5090703@v.loewis.de>	<g22unc$hbu$1@ger.gmane.org>	<4845934A.6030708@v.loewis.de>	<48466D0C.40709@gmail.com>	<48467E50.9030202@voidspace.org.uk>	<loom.20080604T124312-888@post.gmane.org>	<48469888.4070603@gmail.com>	<484745D7.3000808@canterbury.ac.nz>
	<4847D9E0.4020805@gmail.com>
Message-ID: <g2b65v$714$1@ger.gmane.org>

Nick Coghlan wrote:
> Greg Ewing wrote:
>> Nick Coghlan wrote:
>>
>>> - remove support for passing a single value to a format string 
>>> without wrapping it in an iterable first
>>
>> But won't that clobber a large number of the simple
>> use cases that you want to keep %-formatting for?
> 
> Note the part of the proposal that allows *any* iterable on the right 
> hand side rather than the current insistence on a tuple. So the 
> single-value use cases can be boxed easily with a list.
> 
> To my mind salvaging %-formatting requires removing the ambiguity over 
> whether or not the right hand side will be iterated over.
> 
But then this breaks code where iterables are intended to be output 
using a single %s format specifier, for example. I don't see why this 
would be regarded as helpful. To avoid breakage I'd rather keep the 
%-formatting ability as it is, and label it a legacy feature, rather 
than "salvaging" it. It's going to be too tricky to convert using 2to3 
otherwise.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From ggpolo at gmail.com  Fri Jun  6 13:43:10 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Fri, 6 Jun 2008 08:43:10 -0300
Subject: [Python-Dev] Modules for 2.6 inclusion
In-Reply-To: <g2avui$jjo$1@ger.gmane.org>
References: <g2avui$jjo$1@ger.gmane.org>
Message-ID: <ac2200130806060443k2d8fc4fbnf26f78d371890081@mail.gmail.com>

2008/6/6 Georg Brandl <g.brandl at gmx.net>:
> Hi,
>
> PEP 361 lists the following modules for possible inclusion in 2.6 (next
> to pyprocessing, which is now accepted):
>
> - winerror
>  http://python.org/sf/1505257
>  (Owner: MAL)
>
> This patch has been marked as rejected, so I'll remove the entry from
> the PEP.
>
> - setuptools
>  BDFL pronouncement for inclusion in 2.5:
>  http://mail.python.org/pipermail/python-dev/2006-April/063964.html
>
>  PJE's withdrawal from 2.5 for inclusion in 2.6:
>  http://mail.python.org/pipermail/python-dev/2006-April/064145.html
>
> I guess this will be deferred?
>
> - ast
>  http://mail.python.org/pipermail/python-dev/2008-April/078950.html
>
> If there's no objection, I'll go over the interface with Thomas, who's
> working on AST optimization for 2.7, to make sure the ast interface can
> stay the same after his branch is merged, write the docs and commit it
> before beta1.
>
> - bdist_deb in distutils package
>  http://mail.python.org/pipermail/python-dev/2006-February/060926.html
>
> - bdist_egg in distutils package
>
> - pure python pgen module
>  (Owner: Guido)
>  Deferral to 2.6:
>  http://mail.python.org/pipermail/python-dev/2006-April/064528.html
>

I created an issue 1 week ago (http://bugs.python.org/issue2983)
suggesting the addition of the ttk module to lib-tk, and to the new
tkinter package. Is there any chance to this be accepted for Python
2.6 ?

>
> There are also several other "possible" todo items in PEP 361, but they
> all look as if they are not required to be in before beta1.
>
> Georg
>
> --
> Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
> Four shall be the number of spaces thou shalt indent, and the number of thy
> indenting shall be four. Eight shalt thou not indent, nor either indent thou
> two, excepting that thou then proceed to four. Tabs are right out.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>



-- 
-- Guilherme H. Polo Goncalves

From jnoller at gmail.com  Fri Jun  6 14:05:20 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 6 Jun 2008 08:05:20 -0400
Subject: [Python-Dev] PEP 371: Additional Discussion
In-Reply-To: <4222a8490806051908r20c93293v2761ed61c70dbeef@mail.gmail.com>
References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com>
	<4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com>
	<4847EFEF.6040302@gmail.com>
	<4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com>
	<ca471dc20806051022o44828883id5dee70266d32f46@mail.gmail.com>
	<4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com>
	<1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com>
	<4222a8490806051444qd561e68j72f034d214412aa@mail.gmail.com>
	<48489BB1.7080702@gmail.com>
	<4222a8490806051908r20c93293v2761ed61c70dbeef@mail.gmail.com>
Message-ID: <4222a8490806060505h2be38ef4k27f346e8981a2391@mail.gmail.com>

On Thu, Jun 5, 2008 at 10:08 PM, Jesse Noller <jnoller at gmail.com> wrote:
> On Thu, Jun 5, 2008 at 10:06 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Jesse Noller wrote:
>>>
>>> I meant a cleaned version of the PEP - I still have docs and tests to redo
>>>
>> It would also be good if you could check Benjamin's patch on issue 3402 to
>> give threading a PEP 8 compliant API and make sure the names are the same as
>> the new names in multiprocessing.
>>
>> Cheers,
>> Nick.
>>
>> P.S. OK, now I really am going to be offline for the next few days ;)
>>
>
> Good idea, I'll add that to the queue - right now I'm focusing on
> getting the module into the build on trunk (locally) and getting all
> of the docs/tests updated and up to par.
>
> -jesse
>

Just to follow up - I have added a ticket to the tracker
(http://bugs.python.org/issue3050) to track current status and work,
as issues are resolve both the ticket and the PEP will be updated to
reflect status.

I still need to (or have someone with dev privs) add dependencies to
issue 3042 and 1683

-jesse

From guido at python.org  Fri Jun  6 17:52:26 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Jun 2008 08:52:26 -0700
Subject: [Python-Dev] converting the stdlib to str.format
In-Reply-To: <g2b65v$714$1@ger.gmane.org>
References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com>
	<g22unc$hbu$1@ger.gmane.org> <4845934A.6030708@v.loewis.de>
	<48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk>
	<loom.20080604T124312-888@post.gmane.org> <48469888.4070603@gmail.com>
	<484745D7.3000808@canterbury.ac.nz> <4847D9E0.4020805@gmail.com>
	<g2b65v$714$1@ger.gmane.org>
Message-ID: <ca471dc20806060852u1335cffbh59b8e596d9d3aa8@mail.gmail.com>

+1

On 6/6/08, Steve Holden <steve at holdenweb.com> wrote:
> Nick Coghlan wrote:
>> Greg Ewing wrote:
>>> Nick Coghlan wrote:
>>>
>>>> - remove support for passing a single value to a format string
>>>> without wrapping it in an iterable first
>>>
>>> But won't that clobber a large number of the simple
>>> use cases that you want to keep %-formatting for?
>>
>> Note the part of the proposal that allows *any* iterable on the right
>> hand side rather than the current insistence on a tuple. So the
>> single-value use cases can be boxed easily with a list.
>>
>> To my mind salvaging %-formatting requires removing the ambiguity over
>> whether or not the right hand side will be iterated over.
>>
> But then this breaks code where iterables are intended to be output
> using a single %s format specifier, for example. I don't see why this
> would be regarded as helpful. To avoid breakage I'd rather keep the
> %-formatting ability as it is, and label it a legacy feature, rather
> than "salvaging" it. It's going to be too tricky to convert using 2to3
> otherwise.
>
> regards
>   Steve
> --
> Steve Holden        +1 571 484 6266   +1 800 494 3119
> Holden Web LLC              http://www.holdenweb.com/
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>

-- 
Sent from Gmail for mobile | mobile.google.com

--Guido van Rossum (home page: http://www.python.org/~guido/)

From status at bugs.python.org  Fri Jun  6 18:06:28 2008
From: status at bugs.python.org (Python tracker)
Date: Fri,  6 Jun 2008 18:06:28 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20080606160628.DE7D078096@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (05/30/08 - 06/06/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1895 open (+28) / 12981 closed (+15) / 14876 total (+43)

Open issues with patches:   575

Average duration of open issues: 708 days.
Median duration of open issues: 1451 days.

Open Issues Breakdown
   open  1872 (+27)
pending    23 ( +1)

Issues Created Or Reopened (45)
_______________________________

tkinter, assorted fixes                                          06/02/08
CLOSED http://bugs.python.org/issue2906    reopened gpolo                     
       patch, patch                                                            

PyNumberMethods has left-over fields in Py3                      06/02/08
       http://bugs.python.org/issue2997    reopened georg.brandl              
       patch                                                                   

locale module alias table needs to be updated                    05/30/08
CLOSED http://bugs.python.org/issue3011    created  lemburg                   
                                                                               

disutils fails with GNU ld (GNU Binutils) 2.18.50.20080523       05/30/08
CLOSED http://bugs.python.org/issue3012    created  jameinel                  
                                                                               

disutils fails with GNU ld (GNU Binutils) 2.18.50.20080523       05/30/08
       http://bugs.python.org/issue3013    created  jameinel                  
       patch, easy                                                             

file_dealloc() assumes errno is set when EOF is returned         05/30/08
       http://bugs.python.org/issue3014    created  johansen                  
                                                                               

tkinter with wantobjects=False has been broken for some time     05/30/08
       http://bugs.python.org/issue3015    created  gpolo                     
       patch                                                                   

tarfile.py incurs exception from self.chmod() when tarball has g 05/30/08
CLOSED http://bugs.python.org/issue3016    created  zooko                     
                                                                               

Verify doc updates for the decimal module                        05/31/08
       http://bugs.python.org/issue3017    created  rhettinger                
                                                                               

tkinter demos fixed                                              05/31/08
       http://bugs.python.org/issue3018    created  gpolo                     
       patch                                                                   

Python3a5 compile failing due to high memory usage               05/31/08
CLOSED http://bugs.python.org/issue3019    created  segfaulthunter            
                                                                               

doctest should have lib2to3 integration                          06/01/08
       http://bugs.python.org/issue3020    created  scoder                    
                                                                               

Lexical exception handlers                                       06/01/08
       http://bugs.python.org/issue3021    created  pitrou                    
       patch                                                                   

mailbox module, two small fixes                                  06/01/08
       http://bugs.python.org/issue3022    created  gpolo                     
       patch, patch                                                            

Problem with invalidly-encoded command-line arguments (Unix)     06/01/08
       http://bugs.python.org/issue3023    created  baikie                    
                                                                               

Integer conversion inconsistent                                  06/01/08
CLOSED http://bugs.python.org/issue3024    created  helminthe                 
                                                                               

batch/IDLE differ: print broken for chraracters>ascii            06/01/08
CLOSED http://bugs.python.org/issue3025    created  jimjjewett                
                                                                               

mmap broken with large files on 64bit system                     06/02/08
       http://bugs.python.org/issue3026    created  donut                     
                                                                               

if extended c module is calling Windows API waitForSingleObject, 06/02/08
CLOSED http://bugs.python.org/issue3027    created  patrick                   
                                                                               

tokenize module: normal lines, not "logical"                     06/02/08
       http://bugs.python.org/issue3028    created  noam                      
                                                                               

free list management - list, dict, set                           06/02/08
       http://bugs.python.org/issue3029    created  aimacintyre               
       patch, patch                                                            

compiler warning on HP-UX                                        06/03/08
       http://bugs.python.org/issue3030    created  theller                   
                                                                               

distutils package_dir/package_data failure                       06/03/08
       http://bugs.python.org/issue3031    created  fma                       
                                                                               

tkFont added displayof where necessary                           06/03/08
CLOSED http://bugs.python.org/issue3032    created  gpolo                     
       patch                                                                   

tkFont added displayof where necessary                           06/03/08
       http://bugs.python.org/issue3033    created  gpolo                     
       patch, patch                                                            

tkFont added displayof where necessary                           06/03/08
CLOSED http://bugs.python.org/issue3034    created  gpolo                     
       patch, patch                                                            

Removing apparently unwanted functions from Tkinter              06/03/08
       http://bugs.python.org/issue3035    created  gpolo                     
       patch, patch                                                            

docs - print() example wrong in whatsnew                         06/04/08
CLOSED http://bugs.python.org/issue3036    created  evilnick                  
                                                                               

<generator> in output                                            06/04/08
CLOSED http://bugs.python.org/issue3037    created  ianb                      
                                                                               

Return results from Python callbacks to Tcl as Tcl objects, plea 06/05/08
       http://bugs.python.org/issue3038    created  gpolo                     
                                                                               

tarfile.TarFileCompat.writestr(ZipInfo, str) raises AttributeErr 06/05/08
       http://bugs.python.org/issue3039    created  jkankiewicz               
       patch                                                                   

optparse documentation: variable arguments example doesn't work  06/05/08
       http://bugs.python.org/issue3040    created  meyerowitz                
                                                                               

autodoc does not support unicode docstrings                      06/05/08
CLOSED http://bugs.python.org/issue3041    created  cdevienne                 
                                                                               

Add PEP 8 compliant aliases to threading module                  06/05/08
       http://bugs.python.org/issue3042    created  ncoghlan                  
       patch                                                                   

Recursion bug in deepcopy                                        06/05/08
       http://bugs.python.org/issue3043    created  Zeroth                    
                                                                               

Simplify the Integral ABC                                        06/05/08
CLOSED http://bugs.python.org/issue3044    created  rhettinger                
       patch                                                                   

Windows online help broken when spaces in TEMP environ           06/06/08
       http://bugs.python.org/issue3045    created  peta                      
                                                                               

Locking should be removed from the new buffer protocol           06/06/08
       http://bugs.python.org/issue3046    created  scoder                    
       patch                                                                   

string object's lstrip function                                  06/06/08
CLOSED http://bugs.python.org/issue3047    created  jsostok                   
                                                                               

getsizeof incorrect for Unicode strings                          06/06/08
       http://bugs.python.org/issue3048    created  loewis                    
                                                                               

Some 3k sizeof tests fail                                        06/06/08
CLOSED http://bugs.python.org/issue3049    created  loewis                    
                                                                               

Implement PEP 371: multiprocessing module                        06/06/08
       http://bugs.python.org/issue3050    created  jnoller                   
                                                                               

heapq change breaking compatibility                              06/06/08
       http://bugs.python.org/issue3051    created  therve                    
                                                                               

Mac Modules failing to compile                                   06/06/08
       http://bugs.python.org/issue3052    created  benjamin.peterson         
                                                                               

test_shutil fails under AIX                                      06/06/08
       http://bugs.python.org/issue3053    created  pitrou                    
       patch                                                                   



Issues Now Closed (42)
______________________

py3k: Completely remove nb_coerce slot                            260 days
       http://bugs.python.org/issue1185    scoder                    
       patch                                                                   

Add ctypes calling convention that allows safe access of errno    147 days
       http://bugs.python.org/issue1798    theller                   
       patch                                                                   

Adapt pydoc to new doc system                                     132 days
       http://bugs.python.org/issue1883    georg.brandl              
       patch                                                                   

Read support for Records in msilib                                108 days
       http://bugs.python.org/issue2125    loewis                    
       patch                                                                   

Backport oct() and hex() to use __index__                          20 days
       http://bugs.python.org/issue2337    benjamin.peterson         
       patch, 26backport                                                       

numeric overflow in IDLE                                           53 days
       http://bugs.python.org/issue2584    kbk                       
                                                                               

PyOS_vsnprintf() underflow leads to memory corruption              54 days
       http://bugs.python.org/issue2588    gregory.p.smith           
       patch                                                                   

PyOS_vsnprintf() potential integer overflow leads to memory corr   54 days
       http://bugs.python.org/issue2589    gregory.p.smith           
                                                                               

inconsistency with bare * in parameter list                        54 days
       http://bugs.python.org/issue2613    bgolemon                  
                                                                               

datetime/date strftime() method and time.strftime() inconsistenc   26 days
       http://bugs.python.org/issue2782    gregory.p.smith           
                                                                               

subprocess.py leaks fd in communicate                              24 days
       http://bugs.python.org/issue2791    gregory.p.smith           
       patch                                                                   

store thread.get_ident() thread identifier inside threading.Thre   17 days
       http://bugs.python.org/issue2871    gregory.p.smith           
       patch                                                                   

Remove htmllib use in the stdlib                                   17 days
       http://bugs.python.org/issue2873    georg.brandl              
                                                                               

Create the tkinter package                                         16 days
       http://bugs.python.org/issue2884    kbk                       
       patch                                                                   

Add memory footprint query                                         15 days
       http://bugs.python.org/issue2898    schuppenies               
       patch                                                                   

tkinter, assorted fixes                                             0 days
       http://bugs.python.org/issue2906    georg.brandl              
       patch, patch                                                            

ctypes.util.find_library() doesn't consult LD_LIBRARY_PATH         12 days
       http://bugs.python.org/issue2936    theller                   
                                                                               

EasyDialogs - documentation enhancement                             2 days
       http://bugs.python.org/issue3005    georg.brandl              
                                                                               

Module cmd documentation enhancement                                1 days
       http://bugs.python.org/issue3010    georg.brandl              
       patch                                                                   

locale module alias table needs to be updated                       0 days
       http://bugs.python.org/issue3011    lemburg                   
                                                                               

disutils fails with GNU ld (GNU Binutils) 2.18.50.20080523          0 days
       http://bugs.python.org/issue3012    benjamin.peterson         
                                                                               

tarfile.py incurs exception from self.chmod() when tarball has g    1 days
       http://bugs.python.org/issue3016    lars.gustaebel            
                                                                               

Python3a5 compile failing due to high memory usage                  0 days
       http://bugs.python.org/issue3019    georg.brandl              
                                                                               

Integer conversion inconsistent                                     0 days
       http://bugs.python.org/issue3024    rhettinger                
                                                                               

batch/IDLE differ: print broken for chraracters>ascii               0 days
       http://bugs.python.org/issue3025    loewis                    
                                                                               

if extended c module is calling Windows API waitForSingleObject,    1 days
       http://bugs.python.org/issue3027    georg.brandl              
                                                                               

tkFont added displayof where necessary                              0 days
       http://bugs.python.org/issue3032    gpolo                     
       patch                                                                   

tkFont added displayof where necessary                              0 days
       http://bugs.python.org/issue3034    gpolo                     
       patch, patch                                                            

docs - print() example wrong in whatsnew                            0 days
       http://bugs.python.org/issue3036    georg.brandl              
                                                                               

<generator> in output                                               0 days
       http://bugs.python.org/issue3037    georg.brandl              
                                                                               

autodoc does not support unicode docstrings                         0 days
       http://bugs.python.org/issue3041    georg.brandl              
                                                                               

Simplify the Integral ABC                                           0 days
       http://bugs.python.org/issue3044    gvanrossum                
       patch                                                                   

string object's lstrip function                                     0 days
       http://bugs.python.org/issue3047    jsostok                   
                                                                               

Some 3k sizeof tests fail                                           0 days
       http://bugs.python.org/issue3049    schuppenies               
                                                                               

Tk.quit and sys.exit cause Fatal Error                           1669 days
       http://bugs.python.org/issue837234  georg.brandl              
                                                                               

patch 1079734 broke cgi.FieldStorage w/ multipart post req.      1218 days
       http://bugs.python.org/issue1112856 hdiogenes                 
                                                                               

new turtle module                                                 707 days
       http://bugs.python.org/issue1513695 loewis                    
       patch                                                                   

Prevent textwrap from breaking words at hyphens                   413 days
       http://bugs.python.org/issue1702681 gpolo                     
                                                                               

access to unicodedata (via codepoints or 2-char surrogates)       406 days
       http://bugs.python.org/issue1706460 doerwalter                
                                                                               

tiny addition to peephole optimizer                               305 days
       http://bugs.python.org/issue1764087 rhettinger                
       patch                                                                   

generic and more efficient removal of unreachable code            304 days
       http://bugs.python.org/issue1764986 rhettinger                
       patch                                                                   

small improvement for peephole conditional jump optimizer         304 days
       http://bugs.python.org/issue1765558 rhettinger                
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 16 Let bin() show floats                                              7 days
open    http://bugs.python.org/issue3008   

 14 repr() should not escape non-ASCII characters                     53 days
open    http://bugs.python.org/issue2630   

  6 Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1          518 days
open    http://bugs.python.org/issue1628484

  5 Integer conversion inconsistent                                    0 days
closed  http://bugs.python.org/issue3024   

  5 PyNumberMethods has left-over fields in Py3                        4 days
open    http://bugs.python.org/issue2997   

  5 Remove PyUnicode_AsString(), rework PyUnicode_AsStringAndSize()   28 days
open    http://bugs.python.org/issue2799   

  5 Add ctypes calling convention that allows safe access of errno   147 days
closed  http://bugs.python.org/issue1798   

  4 New class special method lookup change                          2020 days
open    http://bugs.python.org/issue643841 

  4 Recursion bug in deepcopy                                          1 days
pending http://bugs.python.org/issue3043   

  4 Add PEP 8 compliant aliases to threading module                    1 days
open    http://bugs.python.org/issue3042   




From guido at python.org  Fri Jun  6 19:22:24 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Jun 2008 10:22:24 -0700
Subject: [Python-Dev] PEP 8 vs PEP 371: Additional Discussion
In-Reply-To: <fb6fbf560806052019v46948c8eyc9d1d68694f6f0c9@mail.gmail.com>
References: <fb6fbf560806052019v46948c8eyc9d1d68694f6f0c9@mail.gmail.com>
Message-ID: <ca471dc20806061022l31bc08d3o1e6eedb0e816f4e9@mail.gmail.com>

On Thu, Jun 5, 2008 at 8:19 PM, Jim Jewett <jimjjewett at gmail.com> wrote:
> Guido van Rossum wrote:
>
>> I consider multiprocessing a new API -- while it bears
>> a superficial resemblance with the threading API the
>> similarities are just that, and it should not be
>> constrained by mistakes in that API.
>
> The justification for including it is precisely that it is *not* a new API.

It is new in the stdlib. Insofar it is not new, renaming the methods
is a very superficial thing.

> For multiple processes in general, there are competing APIs, which may
> well be better.  The advantage of this API is that (in many cases) it
> is a drop-in replacement for threading.  If that breaks, then there
> really isn't any reason to include it in the stdlib yet.

I've said before, and nobody has disagreed, that while familiarity
with the threading module helps people understand how to use the
multiprocessing module (as opposed to other APIs, which differ more
deeply than just in their method names), but that I don't expect
anyone to take an existing multi-threaded app of any complexity and
turn it into a multi-processing app by changing one import.

> This doesn't prevent changing the joint API to conform with PEP 8.
> But why clean this module while leaving the rest of the stdlib?

Have you actually read the thread? Many arguments were put forward for
cleaning up the API -- there is no need to perpetuate mistakes of the
past when introducing anything "new" to the stdlib.

> "Because there is a volunteer" only makes sense if changes to the
> other modules would also be welcomed.  Is there some reason to believe
> that changes in the threading API are much less disruptive than
> changes elsewhere in the stdlib?

We'll keep the compatible API as long as needed. This means in 3.0 too.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun  6 20:01:20 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Jun 2008 11:01:20 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
Message-ID: <ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>

On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote:
> Does anyone actually need an int lookalike with binary methods but
> cannot just inherit from int?

Does anyone actually need an int lookalike with operations like +, -
etc. but cannot just inherit from int? If the answer is yes, is there
a compelling reason why they wouldn't want to support binary methods
as well?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From p.f.moore at gmail.com  Fri Jun  6 20:36:03 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 6 Jun 2008 19:36:03 +0100
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
	<ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
Message-ID: <79990c6b0806061136h67d54fd7yaab4db9d216e1d6c@mail.gmail.com>

2008/6/6 Guido van Rossum <guido at python.org>:
> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote:
>> Does anyone actually need an int lookalike with binary methods but
>> cannot just inherit from int?
>
> Does anyone actually need an int lookalike with operations like +, -
> etc. but cannot just inherit from int? If the answer is yes, is there
> a compelling reason why they wouldn't want to support binary methods
> as well?

I have no vested interest either way, but I think someone mentioned
creating a long int equivalent based on Decimal earlier in the thread.
Wrappers for the gmp library might also want to do this. It's hardly
the world's biggest use case, though...

Paul.

From aleaxit at gmail.com  Fri Jun  6 20:40:01 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Fri, 6 Jun 2008 11:40:01 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
	<ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
Message-ID: <e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>

On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote:
>> Does anyone actually need an int lookalike with binary methods but
>> cannot just inherit from int?
>
> Does anyone actually need an int lookalike with operations like +, -
> etc. but cannot just inherit from int? If the answer is yes, is there
> a compelling reason why they wouldn't want to support binary methods
> as well?

Yes, there's a use case for implementing long integers as arrays of
decimal digits -- addition is roughly as efficient as for binary
integers (x86 chips still have instructions to help with that), and
emitting as decimal digits is MUCH more efficient of course -- so if
I/O in decimal form is the most common operation, with a little
arithmetic (particularly sums), you could gain performance; binary
operations, however, would be as inefficient as decimal form
conversion is for ordinary binary ints, and not needed for the typical
applications that would use these "decimal coded integers"
(accounting), so why not save the implementer of such an extension
from having to write that unneeded and slow extra code?


Alex

From jjl at pobox.com  Fri Jun  6 21:12:27 2008
From: jjl at pobox.com (John J Lee)
Date: Fri, 6 Jun 2008 20:12:27 +0100 (BST)
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
Message-ID: <alpine.DEB.1.00.0806062010470.6207@alice>

On Thu, 5 Jun 2008, Benjamin Peterson wrote:

> - reorganizing the tests into separate directories

Why this one?


John


From musiccomposition at gmail.com  Fri Jun  6 21:20:24 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 6 Jun 2008 14:20:24 -0500
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <alpine.DEB.1.00.0806062010470.6207@alice>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
	<alpine.DEB.1.00.0806062010470.6207@alice>
Message-ID: <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>

On Fri, Jun 6, 2008 at 2:12 PM, John J Lee <jjl at pobox.com> wrote:
> On Thu, 5 Jun 2008, Benjamin Peterson wrote:
>
>> - reorganizing the tests into separate directories
>
> Why this one?

I always find it hard to find a test I'm looking for in a directory
with 365 different tests in it. Also grouping tests by function will
hopefully help reduce duplication and it more intuitive.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From jflatow at northwestern.edu  Fri Jun  6 22:10:58 2008
From: jflatow at northwestern.edu (Jared Flatow)
Date: Fri, 6 Jun 2008 15:10:58 -0500
Subject: [Python-Dev] Assumed reflexivity of rich comparison operators
Message-ID: <57C82822-A384-47F3-9E2C-9730E1471B43@northwestern.edu>

Hi all,

PEP 207 (http://www.python.org/dev/peps/pep-0207/) states in the  
fourth clause of the proposed resolutions to concerns:

"The reflexivity rules *are* assumed by Python.  Thus, the interpreter  
may swap y>x with x<y, y>=x with x<=y, and may swap the arguments of  
x==y and x!=y."

However, if this is the case, why does Python allow the definition of  
both pairs of __le__, __ge__ and __lt__, __gt__ for a single class,  
since users have no guarantee over which will be called? Currently, if  
I do not want x >= y to mean the same thing as y <= x (and believe it  
or not I believe I have a good use case for doing this), there is no  
reliable way of doing this. However, if the decision is to not allow  
users to do this at all using operators (and force them to create  
specially named methods), what is the point of allowing the definition  
of both? It seems very confusing to me (and indeed I was at first very  
confused what was going on), to tempt users to be able to define both  
but give no promise that if they do, the appropriate one will be  
called. Does anyone have a good explanation for this?

Thanks!
jared


From guido at python.org  Fri Jun  6 22:18:13 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Jun 2008 13:18:13 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
	<ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
	<e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>
Message-ID: <ca471dc20806061318y6030a7dbs292f5368730eedb4@mail.gmail.com>

On Fri, Jun 6, 2008 at 11:40 AM, Alex Martelli <aleaxit at gmail.com> wrote:
> On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote:
>>> Does anyone actually need an int lookalike with binary methods but
>>> cannot just inherit from int?
>>
>> Does anyone actually need an int lookalike with operations like +, -
>> etc. but cannot just inherit from int? If the answer is yes, is there
>> a compelling reason why they wouldn't want to support binary methods
>> as well?
>
> Yes, there's a use case for implementing long integers as arrays of
> decimal digits -- addition is roughly as efficient as for binary
> integers (x86 chips still have instructions to help with that), and
> emitting as decimal digits is MUCH more efficient of course -- so if
> I/O in decimal form is the most common operation, with a little
> arithmetic (particularly sums), you could gain performance;

I tried this with ABC in '83. We didn't see any of the hoped-for
benefits though. That's why Python has binary long integers. :-)

> binary
> operations, however, would be as inefficient as decimal form
> conversion is for ordinary binary ints, and not needed for the typical
> applications that would use these "decimal coded integers"
> (accounting), so why not save the implementer of such an extension
> from having to write that unneeded and slow extra code?

You could just raise an exception. This is common in Java when an
Interface requires you implement a method you can't. Or use virtual
inheritance from the Integral class and leave them unimplemented. See
if anyone cares. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun  6 22:24:32 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Jun 2008 13:24:32 -0700
Subject: [Python-Dev] Assumed reflexivity of rich comparison operators
In-Reply-To: <57C82822-A384-47F3-9E2C-9730E1471B43@northwestern.edu>
References: <57C82822-A384-47F3-9E2C-9730E1471B43@northwestern.edu>
Message-ID: <ca471dc20806061324m7977cf59taead197782d3e167@mail.gmail.com>

On Fri, Jun 6, 2008 at 1:10 PM, Jared Flatow <jflatow at northwestern.edu> wrote:
> PEP 207 (http://www.python.org/dev/peps/pep-0207/) states in the fourth
> clause of the proposed resolutions to concerns:
>
> "The reflexivity rules *are* assumed by Python.  Thus, the interpreter may
> swap y>x with x<y, y>=x with x<=y, and may swap the arguments of x==y and
> x!=y."
>
> However, if this is the case, why does Python allow the definition of both
> pairs of __le__, __ge__ and __lt__, __gt__ for a single class, since users
> have no guarantee over which will be called? Currently, if I do not want x
>>= y to mean the same thing as y <= x (and believe it or not I believe I
> have a good use case for doing this),

I find it hard to believe that your users will be happy with that though. :-)

> there is no reliable way of doing this.

Does it help if I tell you that for "x <binop> y" we always try
x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the
case where y is an instance of a subclass of the class of x?

> However, if the decision is to not allow users to do this at all using
> operators (and force them to create specially named methods), what is the
> point of allowing the definition of both?

The same reason we allow (require) you to define __add__ and __radd_.
It is quite possible that for any particular binary operation "x
<binop> y", the class of x doesn't know how to implement it, and then
the class of y is tried with the reversed operation.

> It seems very confusing to me (and
> indeed I was at first very confused what was going on), to tempt users to be
> able to define both but give no promise that if they do, the appropriate one
> will be called. Does anyone have a good explanation for this?

I have explained it as well as I can.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From cesare at pronto.it  Fri Jun  6 22:18:44 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Fri, 06 Jun 2008 22:18:44 +0200
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
	<ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
	<e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>
Message-ID: <op.uccf1ixhhlrjc9@conan>

In data 06 giugno 2008 alle ore 20:40:01, Alex Martelli <aleaxit at gmail.com> ha scritto:

> On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote:
>>> Does anyone actually need an int lookalike with binary methods but
>>> cannot just inherit from int?
>>
>> Does anyone actually need an int lookalike with operations like +, -
>> etc. but cannot just inherit from int? If the answer is yes, is there
>> a compelling reason why they wouldn't want to support binary methods
>> as well?
>
> Yes, there's a use case for implementing long integers as arrays of
> decimal digits -- addition is roughly as efficient as for binary
> integers (x86 chips still have instructions to help with that), and
> emitting as decimal digits is MUCH more efficient of course -- so if
> I/O in decimal form is the most common operation, with a little
> arithmetic (particularly sums), you could gain performance; binary
> operations, however, would be as inefficient as decimal form
> conversion is for ordinary binary ints, and not needed for the typical
> applications that would use these "decimal coded integers"
> (accounting), so why not save the implementer of such an extension
> from having to write that unneeded and slow extra code?
>
>
> Alex

I don't know if you are talking about BCD numbers, but they are quite inefficient and slow in x86 architecture.
There are instructions only to add and subtract packed BCD numbers which uses just two decimal digits (packed in two nibbles into a single byte).
For unpacked BCDs, there are instructions to add, subtract, multiply and divide numbers, but which uses only one digit at the time.

So using packed BCDs to store 8 decimal digits in 32 bits, for example, requires 4 instructions to make addictions or subractions, plus the required shift & mask instructions to put every couple digits into the AL register to execute BCD operations.
Unpacked BCDs need double of them.

Also, these instructions still use microcode to execute on modern processors, slowing down the execution pipeline (most of the simpler instructions do not require microcode, and execute "directly").

Last but not least, on x86-64 architecture BCD instructions were completely removed from the ISA; opcodes are assigned to new instructions. Obviously, binary operations can be performed twice faster thanks to the 64 bit registers and ALUs.

The only practical advantage on using BCD numbers is the conversion-to-string operation, which can be done faster than binary numbers.

Binary addition, subtraction, multiplication and division are greatly faster than BCD ones, and should be the preferred way to do integer math.

Cesare

From python at rcn.com  Fri Jun  6 22:33:05 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 6 Jun 2008 13:33:05 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
	<ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
	<e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>
Message-ID: <EA74414F0FEB42158756F1D9036FB925@RaymondLaptop1>

New idea!  I missed an obvious solution. Let the binary methods in Integral be mixins instead of abstract methods.  That minimizes 
the burden on the class implementer while providing maximum support for clients.

class Integral(Rational):
    ...
    def __lshift__(self, other):
        """self << other"""
        return long(self) << long(other)
    def __xor__(self, other):
        """self ^ other"""
        return long(self) ^ long(other)

I worried a bit about changing type, but this kind of thing is already baked into numbers.py:

    @property
    def imag(self):
        """Real numbers have no imaginary component."""
        return 0

    @property
    def denominator(self):
        """Integers have a denominator of 1."""
        return 1

Raymond


----- Original Message ----- 
From: "Alex Martelli" <aleaxit at gmail.com>
To: "Guido van Rossum" <guido at python.org>
Cc: "Raymond Hettinger" <python at rcn.com>; <python-dev at python.org>
Sent: Friday, June 06, 2008 11:40 AM
Subject: Re: [Python-Dev] Mini-Pep: Simplifying the Integral ABC


> On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote:
>>> Does anyone actually need an int lookalike with binary methods but
>>> cannot just inherit from int?
>>
>> Does anyone actually need an int lookalike with operations like +, -
>> etc. but cannot just inherit from int? If the answer is yes, is there
>> a compelling reason why they wouldn't want to support binary methods
>> as well?
>
> Yes, there's a use case for implementing long integers as arrays of
> decimal digits -- addition is roughly as efficient as for binary
> integers (x86 chips still have instructions to help with that), and
> emitting as decimal digits is MUCH more efficient of course -- so if
> I/O in decimal form is the most common operation, with a little
> arithmetic (particularly sums), you could gain performance; binary
> operations, however, would be as inefficient as decimal form
> conversion is for ordinary binary ints, and not needed for the typical
> applications that would use these "decimal coded integers"
> (accounting), so why not save the implementer of such an extension
> from having to write that unneeded and slow extra code?
>
>
> Alex 


From jflatow at northwestern.edu  Fri Jun  6 22:44:14 2008
From: jflatow at northwestern.edu (Jared Flatow)
Date: Fri, 6 Jun 2008 15:44:14 -0500
Subject: [Python-Dev] Assumed reflexivity of rich comparison operators
In-Reply-To: <ca471dc20806061324m7977cf59taead197782d3e167@mail.gmail.com>
References: <57C82822-A384-47F3-9E2C-9730E1471B43@northwestern.edu>
	<ca471dc20806061324m7977cf59taead197782d3e167@mail.gmail.com>
Message-ID: <DFF6A1A7-4085-487B-B7DF-5F7D2DDD7E2D@northwestern.edu>

On Jun 6, 2008, at 3:24 PM, Guido van Rossum wrote:

> On Fri, Jun 6, 2008 at 1:10 PM, Jared Flatow  
> <jflatow at northwestern.edu> wrote:
>> PEP 207 (http://www.python.org/dev/peps/pep-0207/) states in the  
>> fourth
>> clause of the proposed resolutions to concerns:
>>
>> "The reflexivity rules *are* assumed by Python.  Thus, the  
>> interpreter may
>> swap y>x with x<y, y>=x with x<=y, and may swap the arguments of  
>> x==y and
>> x!=y."
>>
>> However, if this is the case, why does Python allow the definition  
>> of both
>> pairs of __le__, __ge__ and __lt__, __gt__ for a single class,  
>> since users
>> have no guarantee over which will be called? Currently, if I do not  
>> want x
>>> = y to mean the same thing as y <= x (and believe it or not I  
>>> believe I
>> have a good use case for doing this),
>
> I find it hard to believe that your users will be happy with that  
> though. :-)

In this case, I am my users and I would be very happy with it (but I  
won't try to justify it :).

>> there is no reliable way of doing this.
>
> Does it help if I tell you that for "x <binop> y" we always try
> x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the
> case where y is an instance of a subclass of the class of x?

Yes, actually that explains quite a bit, and now I see that is the  
case I am dealing with. y is an instance of a subclass of x, but the  
class of x is the one that defines both the binops. I suppose it is  
too much to ask to only call the __reverse_binop__ if the subclass  
overrides it.

>> However, if the decision is to not allow users to do this at all  
>> using
>> operators (and force them to create specially named methods), what  
>> is the
>> point of allowing the definition of both?
>
> The same reason we allow (require) you to define __add__ and __radd_.
> It is quite possible that for any particular binary operation "x
> <binop> y", the class of x doesn't know how to implement it, and then
> the class of y is tried with the reversed operation.
>
>> It seems very confusing to me (and
>> indeed I was at first very confused what was going on), to tempt  
>> users to be
>> able to define both but give no promise that if they do, the  
>> appropriate one
>> will be called. Does anyone have a good explanation for this?
>
> I have explained it as well as I can.

Thanks very much, at least that is enough information to work around  
reliably.

jared

From guido at python.org  Fri Jun  6 22:47:01 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Jun 2008 13:47:01 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <EA74414F0FEB42158756F1D9036FB925@RaymondLaptop1>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
	<ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
	<e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>
	<EA74414F0FEB42158756F1D9036FB925@RaymondLaptop1>
Message-ID: <ca471dc20806061347n4da42726r988824abdf29d482@mail.gmail.com>

Make that int() instead of long() and I'm okay with it.

On Fri, Jun 6, 2008 at 1:33 PM, Raymond Hettinger <python at rcn.com> wrote:
> New idea!  I missed an obvious solution. Let the binary methods in Integral
> be mixins instead of abstract methods.  That minimizes the burden on the
> class implementer while providing maximum support for clients.
>
> class Integral(Rational):
>   ...
>   def __lshift__(self, other):
>       """self << other"""
>       return long(self) << long(other)
>   def __xor__(self, other):
>       """self ^ other"""
>       return long(self) ^ long(other)
>
> I worried a bit about changing type, but this kind of thing is already baked
> into numbers.py:
>
>   @property
>   def imag(self):
>       """Real numbers have no imaginary component."""
>       return 0
>
>   @property
>   def denominator(self):
>       """Integers have a denominator of 1."""
>       return 1
>
> Raymond
>
>
> ----- Original Message ----- From: "Alex Martelli" <aleaxit at gmail.com>
> To: "Guido van Rossum" <guido at python.org>
> Cc: "Raymond Hettinger" <python at rcn.com>; <python-dev at python.org>
> Sent: Friday, June 06, 2008 11:40 AM
> Subject: Re: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
>
>
>> On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum <guido at python.org>
>> wrote:
>>>
>>> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote:
>>>>
>>>> Does anyone actually need an int lookalike with binary methods but
>>>> cannot just inherit from int?
>>>
>>> Does anyone actually need an int lookalike with operations like +, -
>>> etc. but cannot just inherit from int? If the answer is yes, is there
>>> a compelling reason why they wouldn't want to support binary methods
>>> as well?
>>
>> Yes, there's a use case for implementing long integers as arrays of
>> decimal digits -- addition is roughly as efficient as for binary
>> integers (x86 chips still have instructions to help with that), and
>> emitting as decimal digits is MUCH more efficient of course -- so if
>> I/O in decimal form is the most common operation, with a little
>> arithmetic (particularly sums), you could gain performance; binary
>> operations, however, would be as inefficient as decimal form
>> conversion is for ordinary binary ints, and not needed for the typical
>> applications that would use these "decimal coded integers"
>> (accounting), so why not save the implementer of such an extension
>> from having to write that unneeded and slow extra code?
>>
>>
>> Alex
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Fri Jun  6 23:13:49 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 6 Jun 2008 14:13:49 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<90B6514494134BEAB0191E8423A986D6@RaymondLaptop1>
	<ca471dc20806040833w4982ba20j46dd89efd34c60b4@mail.gmail.com>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
	<ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
	<e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>
	<EA74414F0FEB42158756F1D9036FB925@RaymondLaptop1>
	<ca471dc20806061347n4da42726r988824abdf29d482@mail.gmail.com>
Message-ID: <1A488C5E25A14DC5A0DAC585C073E8BC@RaymondLaptop1>

From: "Guido van Rossum" <guido at python.org>
> Make that int() instead of long() and I'm okay with it.

Does anyone know why Integral says that __long__ is a required abstract method, but not __int__?

Likewise, why is index() defined as long(self) instead of int(self)?

There may be some design nuance that I'm not seeing.


Raymond

From martin at v.loewis.de  Fri Jun  6 23:30:20 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Jun 2008 23:30:20 +0200
Subject: [Python-Dev] Modules for 2.6 inclusion
In-Reply-To: <ac2200130806060443k2d8fc4fbnf26f78d371890081@mail.gmail.com>
References: <g2avui$jjo$1@ger.gmane.org>
	<ac2200130806060443k2d8fc4fbnf26f78d371890081@mail.gmail.com>
Message-ID: <4849AC6C.7020004@v.loewis.de>

> I created an issue 1 week ago (http://bugs.python.org/issue2983)
> suggesting the addition of the ttk module to lib-tk, and to the new
> tkinter package. Is there any chance to this be accepted for Python
> 2.6 ?

Is it complete? In principle, it's for the mentor to decide (who
would need to decide before the first beta, of course - afterwards
it's for the release manager to decide).

Regards,
Martin

From martin at v.loewis.de  Fri Jun  6 23:32:40 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Jun 2008 23:32:40 +0200
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>	<alpine.DEB.1.00.0806062010470.6207@alice>
	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
Message-ID: <4849ACF8.20305@v.loewis.de>

>>> - reorganizing the tests into separate directories
>> Why this one?
> 
> I always find it hard to find a test I'm looking for in a directory
> with 365 different tests in it. Also grouping tests by function will
> hopefully help reduce duplication and it more intuitive.

Still, I don't think this should be done. Flat is better than nested,
and adding hierarchy will make it *more* difficult to find anything
(except perhaps for the one person who did the rearrangement).

I personally use grep to find the place where to add a new test.

Regards,
Martin


From ggpolo at gmail.com  Fri Jun  6 23:40:41 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Fri, 6 Jun 2008 18:40:41 -0300
Subject: [Python-Dev] Modules for 2.6 inclusion
In-Reply-To: <4849AC6C.7020004@v.loewis.de>
References: <g2avui$jjo$1@ger.gmane.org>
	<ac2200130806060443k2d8fc4fbnf26f78d371890081@mail.gmail.com>
	<4849AC6C.7020004@v.loewis.de>
Message-ID: <ac2200130806061440w2f196254q17005f387fea575c@mail.gmail.com>

2008/6/6 "Martin v. L?wis" <martin at v.loewis.de>:
>> I created an issue 1 week ago (http://bugs.python.org/issue2983)
>> suggesting the addition of the ttk module to lib-tk, and to the new
>> tkinter package. Is there any chance to this be accepted for Python
>> 2.6 ?
>
> Is it complete? In principle, it's for the mentor to decide (who
> would need to decide before the first beta, of course - afterwards
> it's for the release manager to decide).
>

It is complete Martin. But I will wait for Fredrik's decision then,
not sure if it will be soon enough tho.

> Regards,
> Martin
>



-- 
-- Guilherme H. Polo Goncalves

From jjt009 at yahoo.com  Fri Jun  6 23:40:24 2008
From: jjt009 at yahoo.com (James Thomas)
Date: Fri, 6 Jun 2008 14:40:24 -0700 (PDT)
Subject: [Python-Dev] New Developer
In-Reply-To: <mailman.3406.1212787831.1043.python-dev@python.org>
Message-ID: <443348.68183.qm@web30903.mail.mud.yahoo.com>

Hi All,
I'm a new developer with Python. I have a solid knowledge of both Python and C. Are there any tasks that would be suitable for a beginner to the Python codebase? I haven't been able to find a list of such tasks on python.org or bugs.python.org.
Thanks,
James Thomas

--- On Fri, 6/6/08, python-dev-request at python.org &lt;python-dev-request at python.org&gt; wrote:
From: python-dev-request at python.org &lt;python-dev-request at python.org&gt;
Subject: Python-Dev Digest, Vol 59, Issue 24
To: python-dev at python.org
Date: Friday, June 6, 2008, 2:30 PM

Send Python-Dev mailing list submissions to
	python-dev at python.org

To subscribe or unsubscribe via the World Wide Web, visit
	http://mail.python.org/mailman/listinfo/python-dev
or, via email, send a message with subject or body 'help' to
	python-dev-request at python.org

You can reach the person managing the list at
	python-dev-owner at python.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Python-Dev digest..."Today's Topics:

   1. Re: Assumed reflexivity of rich comparison operators
      (Guido van Rossum)
   2. Re: Mini-Pep: Simplifying the Integral ABC (Cesare Di Mauro)
   3. Re: Mini-Pep: Simplifying the Integral ABC (Raymond Hettinger)
   4. Re: Assumed reflexivity of rich comparison operators
      (Jared Flatow)
   5. Re: Mini-Pep: Simplifying the Integral ABC (Guido van Rossum)
   6. Re: Mini-Pep: Simplifying the Integral ABC (Raymond Hettinger)
   7. Re: Modules for 2.6 inclusion (Martin v. L?wis)On Fri, Jun 6, 2008 at 1:10 PM, Jared Flatow &lt;jflatow at northwestern.edu&gt;
wrote:
&gt; PEP 207 (http://www.python.org/dev/peps/pep-0207/) states in the fourth
&gt; clause of the proposed resolutions to concerns:
&gt;
&gt; "The reflexivity rules *are* assumed by Python.  Thus, the
interpreter may
&gt; swap y&gt;x with x&lt;y, y&gt;=x with x&lt;=y, and may swap the arguments
of x==y and
&gt; x!=y."
&gt;
&gt; However, if this is the case, why does Python allow the definition of both
&gt; pairs of __le__, __ge__ and __lt__, __gt__ for a single class, since users
&gt; have no guarantee over which will be called? Currently, if I do not want x
&gt;&gt;= y to mean the same thing as y &lt;= x (and believe it or not I
believe I
&gt; have a good use case for doing this),

I find it hard to believe that your users will be happy with that though. :-)

&gt; there is no reliable way of doing this.

Does it help if I tell you that for "x &lt;binop&gt; y" we always try
x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the
case where y is an instance of a subclass of the class of x?

&gt; However, if the decision is to not allow users to do this at all using
&gt; operators (and force them to create specially named methods), what is the
&gt; point of allowing the definition of both?

The same reason we allow (require) you to define __add__ and __radd_.
It is quite possible that for any particular binary operation "x
&lt;binop&gt; y", the class of x doesn't know how to implement it, and
then
the class of y is tried with the reversed operation.

&gt; It seems very confusing to me (and
&gt; indeed I was at first very confused what was going on), to tempt users to
be
&gt; able to define both but give no promise that if they do, the appropriate
one
&gt; will be called. Does anyone have a good explanation for this?

I have explained it as well as I can.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)In data 06 giugno 2008 alle ore 20:40:01, Alex Martelli
&lt;aleaxit at gmail.com&gt; ha scritto:

&gt; On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum &lt;guido at python.org&gt;
wrote:
&gt;&gt; On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger
&lt;python at rcn.com&gt; wrote:
&gt;&gt;&gt; Does anyone actually need an int lookalike with binary methods but
&gt;&gt;&gt; cannot just inherit from int?
&gt;&gt;
&gt;&gt; Does anyone actually need an int lookalike with operations like +, -
&gt;&gt; etc. but cannot just inherit from int? If the answer is yes, is there
&gt;&gt; a compelling reason why they wouldn't want to support binary
methods
&gt;&gt; as well?
&gt;
&gt; Yes, there's a use case for implementing long integers as arrays of
&gt; decimal digits -- addition is roughly as efficient as for binary
&gt; integers (x86 chips still have instructions to help with that), and
&gt; emitting as decimal digits is MUCH more efficient of course -- so if
&gt; I/O in decimal form is the most common operation, with a little
&gt; arithmetic (particularly sums), you could gain performance; binary
&gt; operations, however, would be as inefficient as decimal form
&gt; conversion is for ordinary binary ints, and not needed for the typical
&gt; applications that would use these "decimal coded integers"
&gt; (accounting), so why not save the implementer of such an extension
&gt; from having to write that unneeded and slow extra code?
&gt;
&gt;
&gt; Alex

I don't know if you are talking about BCD numbers, but they are quite
inefficient and slow in x86 architecture.
There are instructions only to add and subtract packed BCD numbers which uses
just two decimal digits (packed in two nibbles into a single byte).
For unpacked BCDs, there are instructions to add, subtract, multiply and divide
numbers, but which uses only one digit at the time.

So using packed BCDs to store 8 decimal digits in 32 bits, for example,
requires 4 instructions to make addictions or subractions, plus the required
shift &amp; mask instructions to put every couple digits into the AL register
to execute BCD operations.
Unpacked BCDs need double of them.

Also, these instructions still use microcode to execute on modern processors,
slowing down the execution pipeline (most of the simpler instructions do not
require microcode, and execute "directly").

Last but not least, on x86-64 architecture BCD instructions were completely
removed from the ISA; opcodes are assigned to new instructions. Obviously,
binary operations can be performed twice faster thanks to the 64 bit registers
and ALUs.

The only practical advantage on using BCD numbers is the conversion-to-string
operation, which can be done faster than binary numbers.

Binary addition, subtraction, multiplication and division are greatly faster
than BCD ones, and should be the preferred way to do integer math.

CesareNew idea!  I missed an obvious solution. Let the binary methods in Integral be
mixins instead of abstract methods.  That minimizes 
the burden on the class implementer while providing maximum support for
clients.

class Integral(Rational):
    ...
    def __lshift__(self, other):
        """self &lt;&lt; other"""
        return long(self) &lt;&lt; long(other)
    def __xor__(self, other):
        """self ^ other"""
        return long(self) ^ long(other)

I worried a bit about changing type, but this kind of thing is already baked
into numbers.py:

    @property
    def imag(self):
        """Real numbers have no imaginary
component."""
        return 0

    @property
    def denominator(self):
        """Integers have a denominator of 1."""
        return 1

Raymond


----- Original Message ----- 
From: "Alex Martelli" &lt;aleaxit at gmail.com&gt;
To: "Guido van Rossum" &lt;guido at python.org&gt;
Cc: "Raymond Hettinger" &lt;python at rcn.com&gt;;
&lt;python-dev at python.org&gt;
Sent: Friday, June 06, 2008 11:40 AM
Subject: Re: [Python-Dev] Mini-Pep: Simplifying the Integral ABC


&gt; On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum &lt;guido at python.org&gt;
wrote:
&gt;&gt; On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger
&lt;python at rcn.com&gt; wrote:
&gt;&gt;&gt; Does anyone actually need an int lookalike with binary methods but
&gt;&gt;&gt; cannot just inherit from int?
&gt;&gt;
&gt;&gt; Does anyone actually need an int lookalike with operations like +, -
&gt;&gt; etc. but cannot just inherit from int? If the answer is yes, is there
&gt;&gt; a compelling reason why they wouldn't want to support binary
methods
&gt;&gt; as well?
&gt;
&gt; Yes, there's a use case for implementing long integers as arrays of
&gt; decimal digits -- addition is roughly as efficient as for binary
&gt; integers (x86 chips still have instructions to help with that), and
&gt; emitting as decimal digits is MUCH more efficient of course -- so if
&gt; I/O in decimal form is the most common operation, with a little
&gt; arithmetic (particularly sums), you could gain performance; binary
&gt; operations, however, would be as inefficient as decimal form
&gt; conversion is for ordinary binary ints, and not needed for the typical
&gt; applications that would use these "decimal coded integers"
&gt; (accounting), so why not save the implementer of such an extension
&gt; from having to write that unneeded and slow extra code?
&gt;
&gt;
&gt; AlexOn Jun 6, 2008, at 3:24 PM, Guido van Rossum wrote:

&gt; On Fri, Jun 6, 2008 at 1:10 PM, Jared Flatow  
&gt; &lt;jflatow at northwestern.edu&gt; wrote:
&gt;&gt; PEP 207 (http://www.python.org/dev/peps/pep-0207/) states in the  
&gt;&gt; fourth
&gt;&gt; clause of the proposed resolutions to concerns:
&gt;&gt;
&gt;&gt; "The reflexivity rules *are* assumed by Python.  Thus, the  
&gt;&gt; interpreter may
&gt;&gt; swap y&gt;x with x&lt;y, y&gt;=x with x&lt;=y, and may swap the
arguments of  
&gt;&gt; x==y and
&gt;&gt; x!=y."
&gt;&gt;
&gt;&gt; However, if this is the case, why does Python allow the definition  
&gt;&gt; of both
&gt;&gt; pairs of __le__, __ge__ and __lt__, __gt__ for a single class,  
&gt;&gt; since users
&gt;&gt; have no guarantee over which will be called? Currently, if I do not  
&gt;&gt; want x
&gt;&gt;&gt; = y to mean the same thing as y &lt;= x (and believe it or not I  
&gt;&gt;&gt; believe I
&gt;&gt; have a good use case for doing this),
&gt;
&gt; I find it hard to believe that your users will be happy with that  
&gt; though. :-)

In this case, I am my users and I would be very happy with it (but I  
won't try to justify it :).

&gt;&gt; there is no reliable way of doing this.
&gt;
&gt; Does it help if I tell you that for "x &lt;binop&gt; y" we
always try
&gt; x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the
&gt; case where y is an instance of a subclass of the class of x?

Yes, actually that explains quite a bit, and now I see that is the  
case I am dealing with. y is an instance of a subclass of x, but the  
class of x is the one that defines both the binops. I suppose it is  
too much to ask to only call the __reverse_binop__ if the subclass  
overrides it.

&gt;&gt; However, if the decision is to not allow users to do this at all  
&gt;&gt; using
&gt;&gt; operators (and force them to create specially named methods), what  
&gt;&gt; is the
&gt;&gt; point of allowing the definition of both?
&gt;
&gt; The same reason we allow (require) you to define __add__ and __radd_.
&gt; It is quite possible that for any particular binary operation "x
&gt; &lt;binop&gt; y", the class of x doesn't know how to implement
it, and then
&gt; the class of y is tried with the reversed operation.
&gt;
&gt;&gt; It seems very confusing to me (and
&gt;&gt; indeed I was at first very confused what was going on), to tempt  
&gt;&gt; users to be
&gt;&gt; able to define both but give no promise that if they do, the  
&gt;&gt; appropriate one
&gt;&gt; will be called. Does anyone have a good explanation for this?
&gt;
&gt; I have explained it as well as I can.

Thanks very much, at least that is enough information to work around  
reliably.

jaredMake that int() instead of long() and I'm okay with it.

On Fri, Jun 6, 2008 at 1:33 PM, Raymond Hettinger &lt;python at rcn.com&gt; wrote:
&gt; New idea!  I missed an obvious solution. Let the binary methods in
Integral
&gt; be mixins instead of abstract methods.  That minimizes the burden on the
&gt; class implementer while providing maximum support for clients.
&gt;
&gt; class Integral(Rational):
&gt;   ...
&gt;   def __lshift__(self, other):
&gt;       """self &lt;&lt; other"""
&gt;       return long(self) &lt;&lt; long(other)
&gt;   def __xor__(self, other):
&gt;       """self ^ other"""
&gt;       return long(self) ^ long(other)
&gt;
&gt; I worried a bit about changing type, but this kind of thing is already
baked
&gt; into numbers.py:
&gt;
&gt;   @property
&gt;   def imag(self):
&gt;       """Real numbers have no imaginary
component."""
&gt;       return 0
&gt;
&gt;   @property
&gt;   def denominator(self):
&gt;       """Integers have a denominator of
1."""
&gt;       return 1
&gt;
&gt; Raymond
&gt;
&gt;
&gt; ----- Original Message ----- From: "Alex Martelli"
&lt;aleaxit at gmail.com&gt;
&gt; To: "Guido van Rossum" &lt;guido at python.org&gt;
&gt; Cc: "Raymond Hettinger" &lt;python at rcn.com&gt;;
&lt;python-dev at python.org&gt;
&gt; Sent: Friday, June 06, 2008 11:40 AM
&gt; Subject: Re: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
&gt;
&gt;
&gt;&gt; On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum
&lt;guido at python.org&gt;
&gt;&gt; wrote:
&gt;&gt;&gt;
&gt;&gt;&gt; On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger
&lt;python at rcn.com&gt; wrote:
&gt;&gt;&gt;&gt;
&gt;&gt;&gt;&gt; Does anyone actually need an int lookalike with binary methods
but
&gt;&gt;&gt;&gt; cannot just inherit from int?
&gt;&gt;&gt;
&gt;&gt;&gt; Does anyone actually need an int lookalike with operations like +,
-
&gt;&gt;&gt; etc. but cannot just inherit from int? If the answer is yes, is
there
&gt;&gt;&gt; a compelling reason why they wouldn't want to support binary
methods
&gt;&gt;&gt; as well?
&gt;&gt;
&gt;&gt; Yes, there's a use case for implementing long integers as arrays
of
&gt;&gt; decimal digits -- addition is roughly as efficient as for binary
&gt;&gt; integers (x86 chips still have instructions to help with that), and
&gt;&gt; emitting as decimal digits is MUCH more efficient of course -- so if
&gt;&gt; I/O in decimal form is the most common operation, with a little
&gt;&gt; arithmetic (particularly sums), you could gain performance; binary
&gt;&gt; operations, however, would be as inefficient as decimal form
&gt;&gt; conversion is for ordinary binary ints, and not needed for the typical
&gt;&gt; applications that would use these "decimal coded integers"
&gt;&gt; (accounting), so why not save the implementer of such an extension
&gt;&gt; from having to write that unneeded and slow extra code?
&gt;&gt;
&gt;&gt;
&gt;&gt; Alex
&gt;
&gt;



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)From: "Guido van Rossum" &lt;guido at python.org&gt;
&gt; Make that int() instead of long() and I'm okay with it.

Does anyone know why Integral says that __long__ is a required abstract method,
but not __int__?

Likewise, why is index() defined as long(self) instead of int(self)?

There may be some design nuance that I'm not seeing.


Raymond&gt; I created an issue 1 week ago (http://bugs.python.org/issue2983)
&gt; suggesting the addition of the ttk module to lib-tk, and to the new
&gt; tkinter package. Is there any chance to this be accepted for Python
&gt; 2.6 ?

Is it complete? In principle, it's for the mentor to decide (who
would need to decide before the first beta, of course - afterwards
it's for the release manager to decide).

Regards,
Martin_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080606/ffcea19a/attachment-0001.htm>

From facundobatista at gmail.com  Sat Jun  7 00:10:00 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 6 Jun 2008 19:10:00 -0300
Subject: [Python-Dev] New Developer
In-Reply-To: <443348.68183.qm@web30903.mail.mud.yahoo.com>
References: <mailman.3406.1212787831.1043.python-dev@python.org>
	<443348.68183.qm@web30903.mail.mud.yahoo.com>
Message-ID: <e04bdf310806061510k78d76ef0u320ac9bb92c901e0@mail.gmail.com>

2008/6/6 James Thomas <jjt009 at yahoo.com>:

> Hi All,
> I'm a new developer with Python. I have a solid knowledge of both Python and
> C. Are there any tasks that would be suitable for a beginner to the Python
> codebase? I haven't been able to find a list of such tasks on python.org or
> bugs.python.org.

Welcome!

The issue tracker [1] is *the* place to find such tasks! Note that
there're some tasks tagged as easy [2], take a look!

Next week we'll have a Python Bug Weekend [3], it's a good moment to gain speed.

(ah, what is not welcomed are HTML mails)

Regards,

[1] http://bugs.python.org/
[2] http://www.taniquetil.com.ar/cgi-bin/pytickets.py?nropag=0&priority=0&severity=0&component=0&version=0&keyword=6
[3] http://wiki.python.org/moin/PythonBugDay

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From guido at python.org  Sat Jun  7 00:23:12 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Jun 2008 15:23:12 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <1A488C5E25A14DC5A0DAC585C073E8BC@RaymondLaptop1>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<2745D7EB7C064B16A88E433588021756@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
	<ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
	<e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>
	<EA74414F0FEB42158756F1D9036FB925@RaymondLaptop1>
	<ca471dc20806061347n4da42726r988824abdf29d482@mail.gmail.com>
	<1A488C5E25A14DC5A0DAC585C073E8BC@RaymondLaptop1>
Message-ID: <ca471dc20806061523s2ce6f465l83bdad1a29161501@mail.gmail.com>

Both of these seem 2.6-specific quirks. Those lines wereJeffrey's;
maybe he remembers? I'm guessing that adding __long__ was done since
2.6 supports it, and the removal of __int__ was an oversight.  I also
think that there's no reason to change __index__ to call long(); int()
will automatically return a long as needed. Maybe changing __long__
back to __int__ is also harmless.

On Fri, Jun 6, 2008 at 2:13 PM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Guido van Rossum" <guido at python.org>
>>
>> Make that int() instead of long() and I'm okay with it.
>
> Does anyone know why Integral says that __long__ is a required abstract
> method, but not __int__?
>
> Likewise, why is index() defined as long(self) instead of int(self)?
>
> There may be some design nuance that I'm not seeing.
>
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From eric+python-dev at trueblade.com  Sat Jun  7 00:51:10 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Fri, 06 Jun 2008 18:51:10 -0400
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <4849ACF8.20305@v.loewis.de>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>	<alpine.DEB.1.00.0806062010470.6207@alice>	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
	<4849ACF8.20305@v.loewis.de>
Message-ID: <4849BF5E.9020205@trueblade.com>

Martin v. L?wis wrote:
>>>> - reorganizing the tests into separate directories
>>> Why this one?
>> I always find it hard to find a test I'm looking for in a directory
>> with 365 different tests in it. Also grouping tests by function will
>> hopefully help reduce duplication and it more intuitive.
> 
> Still, I don't think this should be done. Flat is better than nested,
> and adding hierarchy will make it *more* difficult to find anything
> (except perhaps for the one person who did the rearrangement).
> 
> I personally use grep to find the place where to add a new test.

I agree.  There's not much chance I'd know which directory a test for a 
given piece of functionality is in, so instead of grepping in a single 
directory, I'd have to grep in all of them.  Definitely more hassle.

Eric.

From musiccomposition at gmail.com  Sat Jun  7 00:52:05 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 6 Jun 2008 17:52:05 -0500
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <4849BF5E.9020205@trueblade.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
	<alpine.DEB.1.00.0806062010470.6207@alice>
	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
	<4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com>
Message-ID: <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com>

On Fri, Jun 6, 2008 at 5:51 PM, Eric Smith
<eric+python-dev at trueblade.com> wrote:
> Martin v. L?wis wrote:
>>>
>>> I always find it hard to find a test I'm looking for in a directory
>>> with 365 different tests in it. Also grouping tests by function will
>>> hopefully help reduce duplication and it more intuitive.
>>
>> Still, I don't think this should be done. Flat is better than nested,
>> and adding hierarchy will make it *more* difficult to find anything
>> (except perhaps for the one person who did the rearrangement).
>>
>> I personally use grep to find the place where to add a new test.
>
> I agree.  There's not much chance I'd know which directory a test for a
> given piece of functionality is in, so instead of grepping in a single
> directory, I'd have to grep in all of them.  Definitely more hassle.

Really? Given the choice between core_language (divided into syntax
and builtins) and stdlib you wouldn't know where to look?

>
> Eric.
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From eric+python-dev at trueblade.com  Sat Jun  7 00:58:06 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Fri, 06 Jun 2008 18:58:06 -0400
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>	
	<alpine.DEB.1.00.0806062010470.6207@alice>	
	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>	
	<4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com>
	<1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com>
Message-ID: <4849C0FE.3070208@trueblade.com>

Benjamin Peterson wrote:
> On Fri, Jun 6, 2008 at 5:51 PM, Eric Smith
> <eric+python-dev at trueblade.com> wrote:
>> Martin v. L?wis wrote:
>>>> I always find it hard to find a test I'm looking for in a directory
>>>> with 365 different tests in it. Also grouping tests by function will
>>>> hopefully help reduce duplication and it more intuitive.
>>> Still, I don't think this should be done. Flat is better than nested,
>>> and adding hierarchy will make it *more* difficult to find anything
>>> (except perhaps for the one person who did the rearrangement).
>>>
>>> I personally use grep to find the place where to add a new test.
>> I agree.  There's not much chance I'd know which directory a test for a
>> given piece of functionality is in, so instead of grepping in a single
>> directory, I'd have to grep in all of them.  Definitely more hassle.
> 
> Really? Given the choice between core_language (divided into syntax
> and builtins) and stdlib you wouldn't know where to look?

Really. I wouldn't bother thinking about it.  I'd just grep.

Eric.


From facundobatista at gmail.com  Sat Jun  7 00:58:48 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 6 Jun 2008 19:58:48 -0300
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
	<alpine.DEB.1.00.0806062010470.6207@alice>
	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
Message-ID: <e04bdf310806061558ybf0b745ncd2728ad8240301e@mail.gmail.com>

2008/6/6 Benjamin Peterson <musiccomposition at gmail.com>:

> I always find it hard to find a test I'm looking for in a directory
> with 365 different tests in it. Also grouping tests by function will
> hopefully help reduce duplication and it more intuitive.

Intuitive to who taking into account which grouping?  Start to group
using *your* intuitive way, and a lot of people will find it
unintuitive.

-1 to group the tests.

Regards,

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From jnoller at gmail.com  Sat Jun  7 01:02:46 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 6 Jun 2008 19:02:46 -0400
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
	<alpine.DEB.1.00.0806062010470.6207@alice>
	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
	<4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com>
	<1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com>
Message-ID: <49570F5A-35A9-4349-A26D-F1CF8FE14767@gmail.com>



On Jun 6, 2008, at 6:52 PM, "Benjamin Peterson" <musiccomposition at gmail.com 
 > wrote:

> On Fri, Jun 6, 2008 at 5:51 PM, Eric Smith
> <eric+python-dev at trueblade.com> wrote:
>> Martin v. L?wis wrote:
>>>>
>>>> I always find it hard to find a test I'm looking for in a directory
>>>> with 365 different tests in it. Also grouping tests by function  
>>>> will
>>>> hopefully help reduce duplication and it more intuitive.
>>>
>>> Still, I don't think this should be done. Flat is better than  
>>> nested,
>>> and adding hierarchy will make it *more* difficult to find anything
>>> (except perhaps for the one person who did the rearrangement).
>>>
>>> I personally use grep to find the place where to add a new test.
>>
>> I agree.  There's not much chance I'd know which directory a test  
>> for a
>> given piece of functionality is in, so instead of grepping in a  
>> single
>> directory, I'd have to grep in all of them.  Definitely more hassle.
>
> Really? Given the choice between core_language (divided into syntax
> and builtins) and stdlib you wouldn't know where to look?
>
>>
>> Eric.
>>
>
>
>
> -- 
> Cheers,
> Benjamin Peterson
> "There's no place like 127.0.0.1."
> ______________________________

I'd tend to agree with ben here - a single directory for all tests  
does scale well and makes the purpose of each test unclear.

Besides, grep -r could traverse the directory tree.

+1 nested :)

Jesse

From jnoller at gmail.com  Sat Jun  7 01:04:40 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 6 Jun 2008 19:04:40 -0400
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <e04bdf310806061558ybf0b745ncd2728ad8240301e@mail.gmail.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
	<alpine.DEB.1.00.0806062010470.6207@alice>
	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
	<e04bdf310806061558ybf0b745ncd2728ad8240301e@mail.gmail.com>
Message-ID: <E8DFD9BD-777F-43FE-B52E-621121210E36@gmail.com>

How does 1 directory scale when one day you have possibly thousands of  
tests?


On Jun 6, 2008, at 6:58 PM, "Facundo Batista"  
<facundobatista at gmail.com> wrote:

> 2008/6/6 Benjamin Peterson <musiccomposition at gmail.com>:
>
>> I always find it hard to find a test I'm looking for in a directory
>> with 365 different tests in it. Also grouping tests by function will
>> hopefully help reduce duplication and it more intuitive.
>
> Intuitive to who taking into account which grouping?  Start to group
> using *your* intuitive way, and a lot of people will find it
> unintuitive.
>
> -1 to group the tests.
>
> Regards,
>
> -- 
> . Facundo
>
> Blog: http://www.taniquetil.com.ar/plog/
> PyAr: http://www.python.org/ar/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com

From martin at v.loewis.de  Sat Jun  7 01:19:52 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 07 Jun 2008 01:19:52 +0200
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>	
	<alpine.DEB.1.00.0806062010470.6207@alice>	
	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>	
	<4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com>
	<1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com>
Message-ID: <4849C618.8000209@v.loewis.de>

> Really? Given the choice between core_language (divided into syntax
> and builtins) and stdlib you wouldn't know where to look?

Exactly so. I would not normally guess that the builtins belong to
the core language - they belong to the stdlib, IMO. "core language"
is just syntax to me, perhaps plus the data types that have literals
or displays, and perhaps plus a few selected exceptions.

(just to test your own intuition: when reversed() got added, was
that more of a language change than when sys.meta_path got
added?)

Regards,
Martin

From martin at v.loewis.de  Sat Jun  7 01:26:38 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Jun 2008 01:26:38 +0200
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <E8DFD9BD-777F-43FE-B52E-621121210E36@gmail.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>	<alpine.DEB.1.00.0806062010470.6207@alice>	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>	<e04bdf310806061558ybf0b745ncd2728ad8240301e@mail.gmail.com>
	<E8DFD9BD-777F-43FE-B52E-621121210E36@gmail.com>
Message-ID: <4849C7AE.4070408@v.loewis.de>

> How does 1 directory scale when one day you have possibly thousands of
> tests?

I find this a theoretical question. It took 18 years to arrive at 500
test files. Assuming a linear growth, we get 1000 tests in 2025, and
2000 tests in 2060. People can worry about reorganizing them then.

Regards,
Martin

From rrr at ronadam.com  Sat Jun  7 02:30:10 2008
From: rrr at ronadam.com (Ron Adam)
Date: Fri, 06 Jun 2008 19:30:10 -0500
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <4849C7AE.4070408@v.loewis.de>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>	<alpine.DEB.1.00.0806062010470.6207@alice>	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>	<e04bdf310806061558ybf0b745ncd2728ad8240301e@mail.gmail.com>	<E8DFD9BD-777F-43FE-B52E-621121210E36@gmail.com>
	<4849C7AE.4070408@v.loewis.de>
Message-ID: <4849D692.1050004@ronadam.com>



Martin v. L?wis wrote:
>> How does 1 directory scale when one day you have possibly thousands of
>> tests?
> 
> I find this a theoretical question. It took 18 years to arrive at 500
> test files. Assuming a linear growth, we get 1000 tests in 2025, and
> 2000 tests in 2060. People can worry about reorganizing them then.


Personally I'd like to see packages have their own test directory.  This 
keeps things related to each other together.  Top level modules of course 
would have their tests in the top level test directory as they are now.

I don't see any need to subdivide tests further than that.




From ggpolo at gmail.com  Sat Jun  7 02:40:15 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Fri, 6 Jun 2008 21:40:15 -0300
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <4849D692.1050004@ronadam.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
	<alpine.DEB.1.00.0806062010470.6207@alice>
	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
	<e04bdf310806061558ybf0b745ncd2728ad8240301e@mail.gmail.com>
	<E8DFD9BD-777F-43FE-B52E-621121210E36@gmail.com>
	<4849C7AE.4070408@v.loewis.de> <4849D692.1050004@ronadam.com>
Message-ID: <ac2200130806061740m6c4f25abw1324319ebb9e46b5@mail.gmail.com>

2008/6/6 Ron Adam <rrr at ronadam.com>:
>
>
> Martin v. L?wis wrote:
>>>
>>> How does 1 directory scale when one day you have possibly thousands of
>>> tests?
>>
>> I find this a theoretical question. It took 18 years to arrive at 500
>> test files. Assuming a linear growth, we get 1000 tests in 2025, and
>> 2000 tests in 2060. People can worry about reorganizing them then.
>
>
> Personally I'd like to see packages have their own test directory.  This
> keeps things related to each other together.  Top level modules of course
> would have their tests in the top level test directory as they are now.
>

I really dislike having a test directory inside a python package. You
have my -1 on that idea.

> I don't see any need to subdivide tests further than that.
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>



-- 
-- Guilherme H. Polo Goncalves

From python at rcn.com  Sat Jun  7 03:21:07 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 6 Jun 2008 18:21:07 -0700
Subject: [Python-Dev] on Python's tests (and making them better)
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com><alpine.DEB.1.00.0806062010470.6207@alice><1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com><4849ACF8.20305@v.loewis.de>
	<4849BF5E.9020205@trueblade.com>
	<1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com>
Message-ID: <F4A3B3C08BC14140A81CA297B70FA2BA@RaymondLaptop1>

>>> Still, I don't think this should be done. Flat is better than nested,
>>> and adding hierarchy will make it *more* difficult to find anything
>>> (except perhaps for the one person who did the rearrangement).

Yes.  Grep is your friend.


Raymond

From rrr at ronadam.com  Sat Jun  7 02:30:10 2008
From: rrr at ronadam.com (Ron Adam)
Date: Fri, 06 Jun 2008 19:30:10 -0500
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <4849C7AE.4070408@v.loewis.de>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>	<alpine.DEB.1.00.0806062010470.6207@alice>	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>	<e04bdf310806061558ybf0b745ncd2728ad8240301e@mail.gmail.com>	<E8DFD9BD-777F-43FE-B52E-621121210E36@gmail.com>
	<4849C7AE.4070408@v.loewis.de>
Message-ID: <4849D692.1050004@ronadam.com>



Martin v. L?wis wrote:
>> How does 1 directory scale when one day you have possibly thousands of
>> tests?
> 
> I find this a theoretical question. It took 18 years to arrive at 500
> test files. Assuming a linear growth, we get 1000 tests in 2025, and
> 2000 tests in 2060. People can worry about reorganizing them then.


Personally I'd like to see packages have their own test directory.  This 
keeps things related to each other together.  Top level modules of course 
would have their tests in the top level test directory as they are now.

I don't see any need to subdivide tests further than that.




From steve at pearwood.info  Sat Jun  7 05:24:21 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 7 Jun 2008 13:24:21 +1000
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <4849ACF8.20305@v.loewis.de>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
	<4849ACF8.20305@v.loewis.de>
Message-ID: <200806071324.21468.steve@pearwood.info>

On Sat, 7 Jun 2008 07:32:40 am Martin v. L?wis wrote:

> Flat is better than nested,
> and adding hierarchy will make it *more* difficult to find anything
> (except perhaps for the one person who did the rearrangement).

Do you have a filing cabinet with everything filed under "F" for File? 
*wink*

I've often thought that "Flat is better than nested" was one of the 
weaker Zens... often good advice, but nearly as often not. Lists of 
lists and binary trees can be useful. If you've ever used an operating 
system with a flat file system (and I have) you'll know what a pain it 
is dealing with the lack of hierarchical directories.

> I personally use grep to find the place where to add a new test.

I think, and this is just my opinion, that if you are forced to do a 
linear search through the entire test suite in order to find the file 
you want, that perhaps the test suite needs a tad better organisation.



-- 
Steven

From jyasskin at gmail.com  Sat Jun  7 05:25:03 2008
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Fri, 6 Jun 2008 20:25:03 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <ca471dc20806061523s2ce6f465l83bdad1a29161501@mail.gmail.com>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<g27dq3$t3l$1@ger.gmane.org> <4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
	<ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
	<e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>
	<EA74414F0FEB42158756F1D9036FB925@RaymondLaptop1>
	<ca471dc20806061347n4da42726r988824abdf29d482@mail.gmail.com>
	<1A488C5E25A14DC5A0DAC585C073E8BC@RaymondLaptop1>
	<ca471dc20806061523s2ce6f465l83bdad1a29161501@mail.gmail.com>
Message-ID: <5d44f72f0806062025s17fc0e26yc61cbf17e8b13172@mail.gmail.com>

Well, it seems like Integral instances should be able to be passed to
either int() or long(), so __long__ should probably stay. I have no
idea why I didn't include __int__, but its absence was probably the
only reason __index__ calls long() instead of int().

On Fri, Jun 6, 2008 at 3:23 PM, Guido van Rossum <guido at python.org> wrote:
> Both of these seem 2.6-specific quirks. Those lines wereJeffrey's;
> maybe he remembers? I'm guessing that adding __long__ was done since
> 2.6 supports it, and the removal of __int__ was an oversight.  I also
> think that there's no reason to change __index__ to call long(); int()
> will automatically return a long as needed. Maybe changing __long__
> back to __int__ is also harmless.
>
> On Fri, Jun 6, 2008 at 2:13 PM, Raymond Hettinger <python at rcn.com> wrote:
>> From: "Guido van Rossum" <guido at python.org>
>>>
>>> Make that int() instead of long() and I'm okay with it.
>>
>> Does anyone know why Integral says that __long__ is a required abstract
>> method, but not __int__?
>>
>> Likewise, why is index() defined as long(self) instead of int(self)?
>>
>> There may be some design nuance that I'm not seeing.
>>
>>
>> Raymond
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
Namast?,
Jeffrey Yasskin
http://jeffrey.yasskin.info/

From steve at pearwood.info  Sat Jun  7 05:42:19 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 7 Jun 2008 13:42:19 +1000
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <4849C0FE.3070208@trueblade.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
	<1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com>
	<4849C0FE.3070208@trueblade.com>
Message-ID: <200806071342.19766.steve@pearwood.info>

On Sat, 7 Jun 2008 08:58:06 am Eric Smith wrote:

> > Really? Given the choice between core_language (divided into syntax
> > and builtins) and stdlib you wouldn't know where to look?
>
> Really. I wouldn't bother thinking about it.  I'd just grep.

Since you'll do the same thing regardless of whether the tests are 
nested or not, I don't think you should object to the proposal. "I'm 
against it because it will make no difference to me" is not what I call 
a good objection.

Some people might prefer to cut down their false positives by splitting 
the tests into sub-directories. You can still get all the false 
positives you want by writing

grep -r -I parrot *

instead of 

grep -I parrot *



-- 
Steven

From cs at zip.com.au  Sat Jun  7 06:18:22 2008
From: cs at zip.com.au (Cameron Simpson)
Date: Sat, 7 Jun 2008 14:18:22 +1000
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <200806071324.21468.steve@pearwood.info>
Message-ID: <20080607041822.GA4843@cskk.homeip.net>

On 07Jun2008 13:24, Steven D'Aprano <steve at pearwood.info> wrote:
| On Sat, 7 Jun 2008 07:32:40 am Martin v. L?wis wrote:
| > Flat is better than nested,
| > and adding hierarchy will make it *more* difficult to find anything
| > (except perhaps for the one person who did the rearrangement).
| 
| Do you have a filing cabinet with everything filed under "F" for File? 
| *wink*

No, but I always want to put things in more than one folder in my filing
cabinet, and cannot. Choosing a primary axis for organisation can be
arbitrary.

| I've often thought that "Flat is better than nested" was one of the 
| weaker Zens... often good advice, but nearly as often not. Lists of 
| lists and binary trees can be useful. If you've ever used an operating 
| system with a flat file system (and I have) you'll know what a pain it 
| is dealing with the lack of hierarchical directories.

Aye, but I've frequently tripped over people blithely saying "general to
specific" for nesting without realising that the axes were orthoganal,
not inherently nested.

Not arguing for or against, btw.
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

From jjt009 at yahoo.com  Sat Jun  7 06:57:58 2008
From: jjt009 at yahoo.com (James Thomas)
Date: Fri, 6 Jun 2008 21:57:58 -0700 (PDT)
Subject: [Python-Dev] Location of uname in os.py
Message-ID: <7934.12022.qm@web30904.mail.mud.yahoo.com>

Can anyone give me an idea as to where the function uname is located in os? I am unable to find it explicitly defined within the os.py file in the current svn repository. I also need to know how a call to os.uname() interacts with platform.uname(). I apologize if there is an obvious answer to this question; I just began to work on the Python project a few days ago and therefore am relatively unacqainted with the codebase.
Thanks,
James Thomas



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080606/95535e51/attachment.htm>

From g.brandl at gmx.net  Sat Jun  7 10:51:22 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 07 Jun 2008 08:51:22 +0000
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <ac2200130806061740m6c4f25abw1324319ebb9e46b5@mail.gmail.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>	<alpine.DEB.1.00.0806062010470.6207@alice>	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>	<e04bdf310806061558ybf0b745ncd2728ad8240301e@mail.gmail.com>	<E8DFD9BD-777F-43FE-B52E-621121210E36@gmail.com>	<4849C7AE.4070408@v.loewis.de>
	<4849D692.1050004@ronadam.com>
	<ac2200130806061740m6c4f25abw1324319ebb9e46b5@mail.gmail.com>
Message-ID: <g2db5s$e4k$1@ger.gmane.org>

Guilherme Polo schrieb:
> 2008/6/6 Ron Adam <rrr at ronadam.com>:
>>
>>
>> Martin v. L?wis wrote:
>>>>
>>>> How does 1 directory scale when one day you have possibly thousands of
>>>> tests?
>>>
>>> I find this a theoretical question. It took 18 years to arrive at 500
>>> test files. Assuming a linear growth, we get 1000 tests in 2025, and
>>> 2000 tests in 2060. People can worry about reorganizing them then.
>>
>> Personally I'd like to see packages have their own test directory.  This
>> keeps things related to each other together.  Top level modules of course
>> would have their tests in the top level test directory as they are now.
>>
> 
> I really dislike having a test directory inside a python package. You
> have my -1 on that idea.

I'm also not in favor of this, especially since you still need a wrapper
in Lib/test to make them execute.

However, I can understand package maintainers wanting "everything in one
place".

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From martin at v.loewis.de  Sat Jun  7 09:07:21 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Jun 2008 09:07:21 +0200
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <4849D692.1050004@ronadam.com>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>	<alpine.DEB.1.00.0806062010470.6207@alice>	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>	<e04bdf310806061558ybf0b745ncd2728ad8240301e@mail.gmail.com>	<E8DFD9BD-777F-43FE-B52E-621121210E36@gmail.com>
	<4849C7AE.4070408@v.loewis.de> <4849D692.1050004@ronadam.com>
Message-ID: <484A33A9.1020909@v.loewis.de>

> Personally I'd like to see packages have their own test directory.

That's a good idea, and it is already implemented for several packages.

> This
> keeps things related to each other together.  Top level modules of
> course would have their tests in the top level test directory as they
> are now.

Indeed.

Martin

From martin at v.loewis.de  Sat Jun  7 09:16:23 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Jun 2008 09:16:23 +0200
Subject: [Python-Dev] Location of uname in os.py
In-Reply-To: <7934.12022.qm@web30904.mail.mud.yahoo.com>
References: <7934.12022.qm@web30904.mail.mud.yahoo.com>
Message-ID: <484A35C7.3070803@v.loewis.de>

> Can anyone give me an idea as to where the function uname is located in
> os? 

This question is out-of-scope for python-dev; please use python-list
instead when asking how something is currently implemented.
However, there is an easy answer, so I'll give it anyway: it's
implemented in Modules/posixmodule.c:posix_uname.

Finding out why it still shows up in os is left as an exercise for
the reader.

Regards,
Martin

From martin at v.loewis.de  Sat Jun  7 09:18:18 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Jun 2008 09:18:18 +0200
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <200806071324.21468.steve@pearwood.info>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>	<4849ACF8.20305@v.loewis.de>
	<200806071324.21468.steve@pearwood.info>
Message-ID: <484A363A.2000406@v.loewis.de>

> I think, and this is just my opinion, that if you are forced to do a 
> linear search through the entire test suite in order to find the file 
> you want, that perhaps the test suite needs a tad better organisation.

Perhaps. However, if the linear search is then replaced with a recursive
one, nothing is gained.

Regards,
Martin

From jjl at pobox.com  Sat Jun  7 14:45:01 2008
From: jjl at pobox.com (John J Lee)
Date: Sat, 7 Jun 2008 13:45:01 +0100 (BST)
Subject: [Python-Dev] Python bug day (was Re:  New Developer)
In-Reply-To: <e04bdf310806061510k78d76ef0u320ac9bb92c901e0@mail.gmail.com>
References: <mailman.3406.1212787831.1043.python-dev@python.org>
	<443348.68183.qm@web30903.mail.mud.yahoo.com>
	<e04bdf310806061510k78d76ef0u320ac9bb92c901e0@mail.gmail.com>
Message-ID: <alpine.DEB.1.00.0806071338110.6098@alice>

On Fri, 6 Jun 2008, Facundo Batista wrote:
[...]
> Next week we'll have a Python Bug Weekend [3], it's a good moment to gain speed.
[...]
> [3] http://wiki.python.org/moin/PythonBugDay

That page says the next bug day will be on Sat, June 21st-22nd 2008, which 
is in two weeks' time.  Has that plan changed?


John


From musiccomposition at gmail.com  Sat Jun  7 15:02:44 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sat, 7 Jun 2008 08:02:44 -0500
Subject: [Python-Dev] on Python's tests (and making them better)
In-Reply-To: <4849C618.8000209@v.loewis.de>
References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com>
	<alpine.DEB.1.00.0806062010470.6207@alice>
	<1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com>
	<4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com>
	<1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com>
	<4849C618.8000209@v.loewis.de>
Message-ID: <1afaf6160806070602h26fccefcv7e9aa5a9f659b7cd@mail.gmail.com>

On Fri, Jun 6, 2008 at 6:19 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> (just to test your own intuition: when reversed() got added, was
> that more of a language change than when sys.meta_path got
> added?)

I would say they were both core language changes.

Anyway, I will happily drop this part of this proposal, since it such
a small part.

>
> Regards,
> Martin
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From greg at krypto.org  Sat Jun  7 17:28:58 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 7 Jun 2008 08:28:58 -0700
Subject: [Python-Dev] wrt the beta deadline and freelist management
In-Reply-To: <48475050.9090602@bullseye.andymac.org>
References: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org>
	<48475050.9090602@bullseye.andymac.org>
Message-ID: <52dc1c820806070828x26e85a0bg6dad2c3188414315@mail.gmail.com>

On Wed, Jun 4, 2008 at 7:32 PM, Andrew MacIntyre <
andymac at bullseye.apana.org.au> wrote:

> There are 2 disparate approaches to clearing/compacting free lists for
> basic types:
> - APIs of the form Py<type>_ClearFreeList() called from gc.collect()
>  [class, frame, method, tuple & unicode types];
> - APIs of the form Py<type>_CompactFreeList() called from
>  sys._compact_freelists()  [int & float types];
>
> Both approaches are new for 2.6 & 3.0.
>
> The patch at http://bugs.python.org/issue2862 is geared towards bring
> the int/float free list management into line with the others.
>
> The patch at http://bugs.python.org/issue3029 adds free list management
> to the dict, list & set types.  The value of this is probably minor,
> and this patch is thus not significant in its own right other than for
> consistency.
>
> However Raymond's comment to issue 3029 that the management routines
> shouldn't be public APIs is IMO important (& I happen to agree).
>
> It would be nice to reduce the 2 approaches to one.
>
> I would rather the gc.collect() approach, as this seems to be more
> obvious to many users who have gone looking for free list management in
> prior versions, however my opinion isn't particularly valuable on this.
>
> Can this be resolved for 2.6/3.0?
>

I agree with the gc.collect approach taken in your issue2862 patch.  Barring
any objections, I suggest we accept it and will commit it in the next couple
days.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080607/a3aad932/attachment.htm>

From g.brandl at gmx.net  Sat Jun  7 20:16:16 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 07 Jun 2008 18:16:16 +0000
Subject: [Python-Dev] ABC issues
In-Reply-To: <ca471dc20805271026y6bbee46fxe6c63832e86707c6@mail.gmail.com>
References: <g1ev8d$ub0$1@ger.gmane.org>	<BF4854167FBB49D59B0D5679D0DCAD86@RaymondLaptop1>
	<ca471dc20805271026y6bbee46fxe6c63832e86707c6@mail.gmail.com>
Message-ID: <g2ec94$bsp$1@ger.gmane.org>

Guido van Rossum schrieb:
> On Mon, May 26, 2008 at 11:59 AM, Raymond Hettinger <python at rcn.com> wrote:
>>> * The 2.6-backported Mapping ABC has the 3.0 dict API,
>>>  that is, it uses keys() that returns a view etc.
>>
>> Curious to hear what Guido thinks about this one.
>> A nice use of the Mapping ABC is to be able to
>> get 3.0 behaviors.  I thought that was the whole
>> point of all these backports.  If the ABC gets altered,
>> then it just makes the 2-to-3 conversion harder.
> 
> It's wrong if the ABC doesn't describe the behavior of actual
> implementations; that is its primary purpose, the mixin class is a
> nice side benefit.
> 
> We could make the incompatible mixin classes available separately
> though, if you think they're useful.
> 
>>> * The 2.6 UserDict is not registered as a mapping.
>>
>> Since the API's are not currently the same, it makes
>> sense that UserDict is not registered.
>> If the Mapping ABC does get changed, only IterableUserDict
>> should get registered.  A regular UserDict does not comply.
> 
> Fair enough. I recomment to fix the Mapping ABC and register IterableUserDict.

As a followup:
- The Mapping fix is in http://bugs.python.org/issue3057.
- Registering IterableUserDict is now in SVN.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From pfreixes at gmail.com  Sat Jun  7 19:14:36 2008
From: pfreixes at gmail.com (Pau Freixes)
Date: Sat, 7 Jun 2008 19:14:36 +0200
Subject: [Python-Dev] Different cpu python code usage between embedded mode
	and standalone mode
Message-ID: <207312b70806071014yff950c3j8faa5577462e127d@mail.gmail.com>

Hi list,

First Hello to all, I have a serious problem for understand some results
when I'm comparing cpu usages between same python code in embedded mode and
standalone mode ( python name_script.py )

This last months I have been writing a  program in c like to mod_python for
embedding python language, it's a middleware for dispatch and execute python
batch programs into several nodes. Now I'm writing some python program for
test how scale this into several nodes and comparing with "standalone"
performance.

I found a very strange problem with one application named md5challenge, this
aplication try to calculate the max number md5 digest in several seconds,
md5challenge use a simple signal alarm for stop program when time has
passed. This is the code of python script

_nrdigest = 0
_const_b = 20
_f = None
_signal = False


def handler_alrm(signum, frame):
    global _signal
    global _nrdigest
    global _f


    _signal = True

def try_me():
    global _nrdigest
    global _f
    global _signal

    _f = open("/dev/urandom","r")
    while _signal is not True:
        buff = _f.read(_const_b)
        md5.md5(buff).hexdigest()
        _nrdigest = _nrdigest + 1

    if _f is not None :
        _f.close()

# Define entry point with one input variable
# req is a instance of Request object, usefull members of this object are:
# req.input is a dictionary with input.xml variables
# req.constants is a dictionary with constants defined into signature.xml
# req.output is void dictionary for full with output variables
# req.config is a dictionary with config values take from namespace
# req.apdn_pid is a pid of aplication


def main( req ):
    global _nrdigest


    signal.signal(signal.SIGALRM, handler_alrm)
    signal.alarm(req.input['time'])

    try_me()

    req.output['count'] = _nrdigest

    return req.OK


if __name__ == "__main__":

    # test code
    class test_req:
        pass

    req = test_req()
    req.input = { 'time' : 10 }
    req.output = { 'ret' : 0, 'count' : 0 }
    req.OK = 1

    main(req)

    print "Reached %d digests" % req.output['count']


When I try to run this program in standalone into my Pentium Dual Core
md5challenge reached 1.000.000 milion keys in 10 seconds but when i try to
run this code in embedded mode md5challenge reached about 200.000 more keys
!!! I repeat this test many times and  always  wins  embedded mode  !!!
What's happen ?

Also I tested to erase read dependencies from /dev/random, and calculate all
keys from same buffer. In this case embedded mode win always also, and the
difference are more bigger !!!

The alarm time expires in both case in 10 seconds.

Thks to all, can anybody help to me for understand this results ?


-- 
Pau Freixes
Linux GNU/User
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080607/716e7e87/attachment.htm>

From guido at python.org  Sat Jun  7 19:27:07 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Jun 2008 10:27:07 -0700
Subject: [Python-Dev] Different cpu python code usage between embedded
	mode and standalone mode
In-Reply-To: <207312b70806071014yff950c3j8faa5577462e127d@mail.gmail.com>
References: <207312b70806071014yff950c3j8faa5577462e127d@mail.gmail.com>
Message-ID: <ca471dc20806071027h676c7eadqb27299301a29232b@mail.gmail.com>

This is not an issue for python-dev, but I have to ask: what do you
mean by "embedded mode"?

On Sat, Jun 7, 2008 at 10:14 AM, Pau Freixes <pfreixes at gmail.com> wrote:
> Hi list,
>
> First Hello to all, I have a serious problem for understand some results
> when I'm comparing cpu usages between same python code in embedded mode and
> standalone mode ( python name_script.py )
>
> This last months I have been writing a  program in c like to mod_python for
> embedding python language, it's a middleware for dispatch and execute python
> batch programs into several nodes. Now I'm writing some python program for
> test how scale this into several nodes and comparing with "standalone"
> performance.
>
> I found a very strange problem with one application named md5challenge, this
> aplication try to calculate the max number md5 digest in several seconds,
> md5challenge use a simple signal alarm for stop program when time has
> passed. This is the code of python script
>
> _nrdigest = 0
> _const_b = 20
> _f = None
> _signal = False
>
>
> def handler_alrm(signum, frame):
>     global _signal
>     global _nrdigest
>     global _f
>
>
>     _signal = True
>
> def try_me():
>     global _nrdigest
>     global _f
>     global _signal
>
>     _f = open("/dev/urandom","r")
>     while _signal is not True:
>         buff = _f.read(_const_b)
>         md5.md5(buff).hexdigest()
>         _nrdigest = _nrdigest + 1
>
>     if _f is not None :
>         _f.close()
>
> # Define entry point with one input variable
> # req is a instance of Request object, usefull members of this object are:
> # req.input is a dictionary with input.xml variables
> # req.constants is a dictionary with constants defined into signature.xml
> # req.output is void dictionary for full with output variables
> # req.config is a dictionary with config values take from namespace
> # req.apdn_pid is a pid of aplication
>
>
> def main( req ):
>     global _nrdigest
>
>
>     signal.signal(signal.SIGALRM, handler_alrm)
>     signal.alarm(req.input['time'])
>
>     try_me()
>
>     req.output['count'] = _nrdigest
>
>     return req.OK
>
>
> if __name__ == "__main__":
>
>     # test code
>     class test_req:
>         pass
>
>     req = test_req()
>     req.input = { 'time' : 10 }
>     req.output = { 'ret' : 0, 'count' : 0 }
>     req.OK = 1
>
>     main(req)
>
>     print "Reached %d digests" % req.output['count']
>
>
> When I try to run this program in standalone into my Pentium Dual Core
> md5challenge reached 1.000.000 milion keys in 10 seconds but when i try to
> run this code in embedded mode md5challenge reached about 200.000 more keys
> !!! I repeat this test many times and  always  wins  embedded mode  !!!
> What's happen ?
>
> Also I tested to erase read dependencies from /dev/random, and calculate all
> keys from same buffer. In this case embedded mode win always also, and the
> difference are more bigger !!!
>
> The alarm time expires in both case in 10 seconds.
>
> Thks to all, can anybody help to me for understand this results ?
>
> --
> Pau Freixes
> Linux GNU/User
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Jun  7 19:30:16 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Jun 2008 10:30:16 -0700
Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC
In-Reply-To: <5d44f72f0806062025s17fc0e26yc61cbf17e8b13172@mail.gmail.com>
References: <FA99F71136404E2ABD7A0BCA1552258A@RaymondLaptop1>
	<4847E828.9060001@gmail.com>
	<545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1>
	<ca471dc20806061101j74712f6mc3fa8f622c7b6926@mail.gmail.com>
	<e8a0972d0806061140v21ba368es4f8db90e7d1cdae3@mail.gmail.com>
	<EA74414F0FEB42158756F1D9036FB925@RaymondLaptop1>
	<ca471dc20806061347n4da42726r988824abdf29d482@mail.gmail.com>
	<1A488C5E25A14DC5A0DAC585C073E8BC@RaymondLaptop1>
	<ca471dc20806061523s2ce6f465l83bdad1a29161501@mail.gmail.com>
	<5d44f72f0806062025s17fc0e26yc61cbf17e8b13172@mail.gmail.com>
Message-ID: <ca471dc20806071030m587ab20ew1d3c7efbcb400c62@mail.gmail.com>

I recommend switching back to __int__ and int; even in 2.5, these are
expected to return either an int or a long as required. There's no
need to mess with __long__ at all.

On Fri, Jun 6, 2008 at 8:25 PM, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> Well, it seems like Integral instances should be able to be passed to
> either int() or long(), so __long__ should probably stay. I have no
> idea why I didn't include __int__, but its absence was probably the
> only reason __index__ calls long() instead of int().
>
> On Fri, Jun 6, 2008 at 3:23 PM, Guido van Rossum <guido at python.org> wrote:
>> Both of these seem 2.6-specific quirks. Those lines wereJeffrey's;
>> maybe he remembers? I'm guessing that adding __long__ was done since
>> 2.6 supports it, and the removal of __int__ was an oversight.  I also
>> think that there's no reason to change __index__ to call long(); int()
>> will automatically return a long as needed. Maybe changing __long__
>> back to __int__ is also harmless.
>>
>> On Fri, Jun 6, 2008 at 2:13 PM, Raymond Hettinger <python at rcn.com> wrote:
>>> From: "Guido van Rossum" <guido at python.org>
>>>>
>>>> Make that int() instead of long() and I'm okay with it.
>>>
>>> Does anyone know why Integral says that __long__ is a required abstract
>>> method, but not __int__?
>>>
>>> Likewise, why is index() defined as long(self) instead of int(self)?
>>>
>>> There may be some design nuance that I'm not seeing.
>>>
>>>
>>> Raymond
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>>
>
>
>
> --
> Namast?,
> Jeffrey Yasskin
> http://jeffrey.yasskin.info/
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Sat Jun  7 22:13:48 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 07 Jun 2008 20:13:48 +0000
Subject: [Python-Dev] Deprecate parser's "ast" functions?
Message-ID: <g2ej5h$tnl$1@ger.gmane.org>

The parser module exports each function and type twice, once with "AST" in
the name, once with "ST".  Since "AST" now has a different meaning for
Python code compilation, I propose to deprecate the "ast" variants in 2.6
and remove them in Python 3000.

(Also, all keyword arguments are called "ast", that cannot change in 2.6
but should in 3.0.)

I'm at the moment changing the documentation of parser to only refer to
the "st" variants anymore.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From gnewsg at gmail.com  Sat Jun  7 22:11:34 2008
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Sat, 7 Jun 2008 13:11:34 -0700 (PDT)
Subject: [Python-Dev] Modules for 2.6 inclusion
In-Reply-To: <ac2200130806061440w2f196254q17005f387fea575c@mail.gmail.com>
References: <g2avui$jjo$1@ger.gmane.org>
	<ac2200130806060443k2d8fc4fbnf26f78d371890081@mail.gmail.com> 
	<4849AC6C.7020004@v.loewis.de>
	<ac2200130806061440w2f196254q17005f387fea575c@mail.gmail.com>
Message-ID: <905d05c9-eb24-467f-8a24-d9d0c7e3b439@c58g2000hsc.googlegroups.com>

On 6 Giu, 13:27, Georg Brandl <g.bra... at gmx.net> wrote:

> - setuptools
>    BDFL pronouncement for inclusion in 2.5:
>    http://mail.python.org/pipermail/python-dev/2006-April/063964.html

I'd like to see more interest about this issue since it's a real shame
that the current distutils is not even able to solve dependencies and
has such a poor integration with Pypi.
Having a *valid* distribution tool like setuptools integrated with
Python would be the first step to have one of the most important
things Python is currently missing finally available: a centralized
archive of softwares like Perl CPAN or Ruby Gems.
IMHO this should be put on top of the "TODO" list and I'm honestly
surprised it raised so little clamor so far.


--- Giampaolo
http://code.google.com/p/pyftpdlib

From g.brandl at gmx.net  Sun Jun  8 00:27:46 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 07 Jun 2008 22:27:46 +0000
Subject: [Python-Dev] Modules for 2.6 inclusion
In-Reply-To: <ac2200130806060443k2d8fc4fbnf26f78d371890081@mail.gmail.com>
References: <g2avui$jjo$1@ger.gmane.org>
	<ac2200130806060443k2d8fc4fbnf26f78d371890081@mail.gmail.com>
Message-ID: <g2er0m$htr$1@ger.gmane.org>

Guilherme Polo schrieb:

> I created an issue 1 week ago (http://bugs.python.org/issue2983)
> suggesting the addition of the ttk module to lib-tk, and to the new
> tkinter package. Is there any chance to this be accepted for Python
> 2.6 ?

This may be a good thing to have since it can show that Tkinter is not
as dead (and dead ugly) as many people think it is.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From armin.ronacher at active-4.com  Sat Jun  7 23:10:14 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sat, 7 Jun 2008 21:10:14 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?b?cGVwOGl0eSBfX2Z1dHVyZV9f?=
Message-ID: <loom.20080607T210748-495@post.gmane.org>

Hi,

That's just a flaming-sword thread but I want to mention it nonetheless :-)

Basically I propose getting rid of __future__._Feature.getMandatoryRelease()
in favour of __future__._Feature.mandatory.  That's backwards compatibile
and much more pythonic.  Getters/Setters are considered unpythonic and the
getting rid of all that Java-like naming sounds like a good idea to me :)

If threading/processing gets a pep8ified API with 2.6, why not __future__?

Proposed patch: http://paste.pocoo.org/show/64512/


Regards,
Armin


From pfreixes at gmail.com  Sun Jun  8 13:15:08 2008
From: pfreixes at gmail.com (Pau Freixes)
Date: Sun, 8 Jun 2008 13:15:08 +0200
Subject: [Python-Dev] GIL cpu usage problem, confirm me
Message-ID: <207312b70806080415n5a628179o157e755eb08ea82b@mail.gmail.com>

Hi List,

Surly this is a recurring theme into python dev world, but I need your help
for confirm if the follow image it's really

http://www.milnou.net/~pfreixes/img/cpu_usage_gil_problem.png

I'm writing a brief article for my blog and I need to make sure about the
current problem with GIL and multi core environments, this picture try to
explain with images the problem for scheduling multiple threads running
python code of same interpreter into multiple cpu cores. Can  anyone confirm
to me this picture ?

And if it's possible answer this two questions I will be happy :/

1) When this situation it's produced into one core environment whats happens
when thread library or os switch context into other python thread and this
don't have a GIL ?

2) Exist some PEP or plans for modify this and run multiple thread python
for same interpreter at current time ? for python 3000?

Thanks and excuse for this break.


-- 
Pau Freixes
Linux GNU/User
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080608/67676ef5/attachment.htm>

From musiccomposition at gmail.com  Sun Jun  8 15:37:20 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sun, 8 Jun 2008 08:37:20 -0500
Subject: [Python-Dev] GIL cpu usage problem, confirm me
In-Reply-To: <207312b70806080415n5a628179o157e755eb08ea82b@mail.gmail.com>
References: <207312b70806080415n5a628179o157e755eb08ea82b@mail.gmail.com>
Message-ID: <1afaf6160806080637v2816e03g1f7bf15ca2306f1c@mail.gmail.com>

On Sun, Jun 8, 2008 at 6:15 AM, Pau Freixes <pfreixes at gmail.com> wrote:
> Hi List,
>
> Surly this is a recurring theme into python dev world, but I need your help
> for confirm if the follow image it's really
>
> http://www.milnou.net/~pfreixes/img/cpu_usage_gil_problem.png
>
> I'm writing a brief article for my blog and I need to make sure about the
> current problem with GIL and multi core environments, this picture try to
> explain with images the problem for scheduling multiple threads running
> python code of same interpreter into multiple cpu cores. Can  anyone confirm
> to me this picture ?
>
> And if it's possible answer this two questions I will be happy :/

Next time, comp.lang.python would be the place to ask these questions.

>
> 1) When this situation it's produced into one core environment whats happens
> when thread library or os switch context into other python thread and this
> don't have a GIL ?

They run together.
>
> 2) Exist some PEP or plans for modify this and run multiple thread python
> for same interpreter at current time ? for python 3000?

Certainly not in core Python. Have a look
http://code.google.com/p/python-threadsafe/, though.




-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From exarkun at divmod.com  Sun Jun  8 15:49:33 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sun, 8 Jun 2008 09:49:33 -0400
Subject: [Python-Dev] GIL cpu usage problem, confirm me
In-Reply-To: <1afaf6160806080637v2816e03g1f7bf15ca2306f1c@mail.gmail.com>
Message-ID: <20080608134933.4714.378522420.divmod.quotient.6949@ohm>

On Sun, 8 Jun 2008 08:37:20 -0500, Benjamin Peterson <musiccomposition at gmail.com> wrote:
>
>Certainly not in core Python. Have a look
>http://code.google.com/p/python-threadsafe/, though.
>

http://code.google.com/p/python-safethread/

Jean-Paul

From curt at hagenlocher.org  Mon Jun  9 04:24:55 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Sun, 8 Jun 2008 19:24:55 -0700
Subject: [Python-Dev] Assignment to None
Message-ID: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>

My apologies if this is one of those "dead horse" issues.  The
following seems a little inconsistent to me:

>>> c = C()
>>> c.None
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: C instance has no attribute 'None'
>>> c.None = 'foo'
  File "<stdin>", line 1
SyntaxError: assignment to None
>>> setattr(c, 'None', 'foo')
>>> c.None
'foo'
>>>

So, it's okay to setattr the attribute name "None" but not okay to set
it directly?  Is this deliberate or is it an unintentional side effect
of parser changes to prevent assignment to None?

--
Curt Hagenlocher
curt at hagenlocher.org

From musiccomposition at gmail.com  Mon Jun  9 05:10:12 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sun, 8 Jun 2008 22:10:12 -0500
Subject: [Python-Dev] Assignment to None
In-Reply-To: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
Message-ID: <1afaf6160806082010w5ca671ctfa4c259cacb61351@mail.gmail.com>

On Sun, Jun 8, 2008 at 9:24 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
> My apologies if this is one of those "dead horse" issues.  The
> following seems a little inconsistent to me:
>
>>>> c = C()
>>>> c.None
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> AttributeError: C instance has no attribute 'None'
>>>> c.None = 'foo'
>  File "<stdin>", line 1
> SyntaxError: assignment to None
>>>> setattr(c, 'None', 'foo')
>>>> c.None
> 'foo'
>>>>
>
> So, it's okay to setattr the attribute name "None" but not okay to set
> it directly?  Is this deliberate or is it an unintentional side effect
> of parser changes to prevent assignment to None?

I believe that runtime checks like the one needed for setattr would be
too much of a performance hit to be practical. The syntax changes are
meant to avoid mistakes and confusion rather than completely prevent
something from ever happening.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From steve at pearwood.info  Mon Jun  9 05:48:44 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 9 Jun 2008 13:48:44 +1000
Subject: [Python-Dev] Assignment to None
In-Reply-To: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
Message-ID: <200806091348.44361.steve@pearwood.info>

On Mon, 9 Jun 2008 12:24:55 pm Curt Hagenlocher wrote:

> So, it's okay to setattr the attribute name "None" but not okay to 
> set it directly? ?

I suspect this is off-topic for python-dev, and would be better on 
comp.lang.python or similar, but for what it's worth, I consider having 
an attribute named 'None' bad practise, regardless of any runtime 
checks. But as part of Python's "we're all consenting adults here" 
philosophy, I wouldn't approve of expensive or extensive run-time 
checks specifically to prevent it. If you really have to name your 
attribute None, and are prepared to live with the consequences, then go 
ahead.

In a similar fashion:

>>> class Parrot(object):
...     pass
...
>>> p = Parrot()
>>> p.1 = 'spam'
  File "<stdin>", line 1
    p.1
      ^
SyntaxError: invalid syntax
>>> setattr(p, '1', 'spam')
>>> 


-- 
Steven

From martin at v.loewis.de  Mon Jun  9 07:19:44 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jun 2008 07:19:44 +0200
Subject: [Python-Dev] Assignment to None
In-Reply-To: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
Message-ID: <484CBD70.3000808@v.loewis.de>

> So, it's okay to setattr the attribute name "None" but not okay to set
> it directly?  Is this deliberate or is it an unintentional side effect
> of parser changes to prevent assignment to None?

It's deliberate. setattr(o, "foo bar", "baz") also works, even though
"foo bar" is not an identifier. setattr doesn't take the Python grammar
into account, but only the object's structure.

Regards,
Martin

From greg at krypto.org  Mon Jun  9 07:20:36 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 8 Jun 2008 22:20:36 -0700
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <48490121.5030701@egenix.com>
References: <48397ECC.9070805@cheimes.de> <483EF139.8000606@egenix.com>
	<483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com>
	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
	<4843E884.1060705@egenix.com>
	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>
	<ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>
	<52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com>
	<48490121.5030701@egenix.com>
Message-ID: <52dc1c820806082220n2f0832e1l7af4bc2fecf31987@mail.gmail.com>

On Fri, Jun 6, 2008 at 2:19 AM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 2008-06-03 01:29, Gregory P. Smith wrote:
>
>> On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum <guido at python.org>
>> wrote:
>>
>>  I will freely admit that I haven't followed this thread in any detail,
>>> but if it were up to me, I'd have the 2.6 internal code use PyString
>>>
>>
>> ...
>>
>> Should we read this as a BDFL pronouncement and make it so?
>>
>> All that would mean change wise is that trunk r63675 as well as possibly
>> r63672 and r63677 would need to be rolled back and this whole discussion
>> over if such a big change should have happened would turn into a moot
>> point.
>>
>
> I would certainly welcome reverting the change.
>
> All that's needed to support PyBytes API in 2.x is a set of #defines
> that map the new APIs to the PyString names. That's a clean and
> easily understandable solution.
>

Okay, I've reverted r63675 in trunk revision r64048.  That leaves all of the
python modules and internals using PyString_ api names instead of PyBytes_
api names as they were before.  PyBytes_ #define's exist for the appropriate
PyString methods incase anyone wants to use those.

Programmers interested in the code
> for a PyString API can then still look up the code in stringobject.c,
> e.g. to find out how a certain special case is handled or to check
> the ref counting - just like they did for years.


The files still exist with the new names.  bytesobject.c instead of
stringobject.c.  Those renames were done in the other CLs i mentioned which
have not yet been reverted.  The current state seems a bit odd because they
depend on the #defines to cause method definitions to be the PyString_ names
instead of the PyBytes_ names.


>
>
> Developer who want to start differentiating between mixed byte/text
> data and bytes-only can start using PyBytes for byte data.
>
>  I would also add macros that map the PyBytes_* APIs to PyString_*, but
>>> I would not start using these internally except in code newly written
>>> for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString
>>> for 8-bit strings containing text, and PyBytes for 8-bit strings
>>> containing binary data. For 8-bit strings that could contain either
>>> text or data, I'd use PyString, in the spirit of 2.x.
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080608/efa67dcf/attachment-0001.htm>

From musiccomposition at gmail.com  Mon Jun  9 01:38:05 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sun, 8 Jun 2008 18:38:05 -0500
Subject: [Python-Dev] Nab those release blockers!
Message-ID: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com>

As we approach the planned first betas for 2.6/3.0, my heart sinks
when I see the imposing list of 16 release blockers. [1] Luckily, most
of the issues have patches that just need *your* review. Others,
namely the Py3k exception issues, may require a little more
discussion, but I'm sure we can get it done.

Let's get this release on time! [clapping]

[1] http://bugs.python.org/issue?%40sort0=activity&%40sortdir0=on&%40sort1=&%40group0=&%40group1=&%40columns=title%2Cid%2Cactivity%2Cversions%2Cstatus&%40filter=priority%2Cstatus&priority=1&status=1&%40pagesize=50&%40startwith=0

-- 
Thanks
Benjamin Peterson
"There's no place like 127.0.0.1."

From fuzzyman at voidspace.org.uk  Mon Jun  9 10:28:03 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 09 Jun 2008 09:28:03 +0100
Subject: [Python-Dev] Assignment to None
In-Reply-To: <200806091348.44361.steve@pearwood.info>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<200806091348.44361.steve@pearwood.info>
Message-ID: <484CE993.5050506@voidspace.org.uk>

Steven D'Aprano wrote:
> On Mon, 9 Jun 2008 12:24:55 pm Curt Hagenlocher wrote:
>
>   
>> So, it's okay to setattr the attribute name "None" but not okay to 
>> set it directly?  
>>     
>
> I suspect this is off-topic for python-dev, and would be better on 
> comp.lang.python or similar, but for what it's worth, I consider having 
> an attribute named 'None' bad practise, regardless of any runtime 
> checks. But as part of Python's "we're all consenting adults here" 
> philosophy, I wouldn't approve of expensive or extensive run-time 
> checks specifically to prevent it. If you really have to name your 
> attribute None, and are prepared to live with the consequences, then go 
> ahead.
>
>   

FWIW Curt is asking because he is on the team responsible for 
implementing IronPython.

In .NET you have several enumerations with members called None, for 
example 'Alignment.None'. It would be inconvenient for IronPython users 
if it enforced this particular syntax rule of Python's; that not only is 
assignment to None disallowed but that members named None are invalid 
syntax.

The question is, what is the specification for Python. Is it that 
assignment to None is disallowed and the naming of members called None 
being invalid syntax is merely an artefact of the implementation of 
this, or does Python require this...

Michael Foord

> In a similar fashion:
>
>   
>>>> class Parrot(object):
>>>>         
> ...     pass
> ...
>   
>>>> p = Parrot()
>>>> p.1 = 'spam'
>>>>         
>   File "<stdin>", line 1
>     p.1
>       ^
> SyntaxError: invalid syntax
>   
>>>> setattr(p, '1', 'spam')
>>>>
>>>>         
>
>
>   


From mal at egenix.com  Mon Jun  9 10:44:38 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 09 Jun 2008 10:44:38 +0200
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <52dc1c820806082220n2f0832e1l7af4bc2fecf31987@mail.gmail.com>
References: <48397ECC.9070805@cheimes.de>
	<483EF139.8000606@egenix.com>	<483F34C3.3050402@gmail.com>
	<483FBCB4.5020007@egenix.com>	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>	<4843E884.1060705@egenix.com>	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>	<ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>	<52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com>	<48490121.5030701@egenix.com>
	<52dc1c820806082220n2f0832e1l7af4bc2fecf31987@mail.gmail.com>
Message-ID: <484CED76.3070703@egenix.com>

On 2008-06-09 07:20, Gregory P. Smith wrote:
> On Fri, Jun 6, 2008 at 2:19 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> 
>> On 2008-06-03 01:29, Gregory P. Smith wrote:
>>
>>> On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum <guido at python.org>
>>> wrote:
>>>
>>>  I will freely admit that I haven't followed this thread in any detail,
>>>> but if it were up to me, I'd have the 2.6 internal code use PyString
>>>>
>>> ...
>>>
>>> Should we read this as a BDFL pronouncement and make it so?
>>>
>>> All that would mean change wise is that trunk r63675 as well as possibly
>>> r63672 and r63677 would need to be rolled back and this whole discussion
>>> over if such a big change should have happened would turn into a moot
>>> point.
>>>
>> I would certainly welcome reverting the change.
>>
>> All that's needed to support PyBytes API in 2.x is a set of #defines
>> that map the new APIs to the PyString names. That's a clean and
>> easily understandable solution.
>>
> 
> Okay, I've reverted r63675 in trunk revision r64048.  That leaves all of the
> python modules and internals using PyString_ api names instead of PyBytes_
> api names as they were before.  PyBytes_ #define's exist for the appropriate
> PyString methods incase anyone wants to use those.

Thanks.

> Programmers interested in the code
>> for a PyString API can then still look up the code in stringobject.c,
>> e.g. to find out how a certain special case is handled or to check
>> the ref counting - just like they did for years.
> 
> 
> The files still exist with the new names.  bytesobject.c instead of
> stringobject.c.  Those renames were done in the other CLs i mentioned which
> have not yet been reverted.  The current state seems a bit odd because they
> depend on the #defines to cause method definitions to be the PyString_ names
> instead of the PyBytes_ names.

Please restore the original state, ie. PyString APIs live in
stringobject.h and stringobject.c. bytesobject.h should then have
the #defines for PyBytes APIs, pointing them to the PyString
names (basically what's currently in stringobject.h).

>> Developer who want to start differentiating between mixed byte/text
>> data and bytes-only can start using PyBytes for byte data.
>>
>>  I would also add macros that map the PyBytes_* APIs to PyString_*, but
>>>> I would not start using these internally except in code newly written
>>>> for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString
>>>> for 8-bit strings containing text, and PyBytes for 8-bit strings
>>>> containing binary data. For 8-bit strings that could contain either
>>>> text or data, I'd use PyString, in the spirit of 2.x.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 09 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            27 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From jnoller at gmail.com  Mon Jun  9 12:33:33 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 9 Jun 2008 06:33:33 -0400
Subject: [Python-Dev] Nab those release blockers!
In-Reply-To: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com>
References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com>
Message-ID: <788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com>

For the multiprocessing module all I have left is the docs in rest  
format - given I'm a rest newbie, it's taking longer than I thought

On Jun 8, 2008, at 7:38 PM, "Benjamin Peterson" <musiccomposition at gmail.com 
 > wrote:

> As we approach the planned first betas for 2.6/3.0, my heart sinks
> when I see the imposing list of 16 release blockers. [1] Luckily, most
> of the issues have patches that just need *your* review. Others,
> namely the Py3k exception issues, may require a little more
> discussion, but I'm sure we can get it done.
>
> Let's get this release on time! [clapping]
>
> [1] http://bugs.python.org/issue?%40sort0=activity&%40sortdir0=on&%40sort1=&%40group0=&%40group1=&%40columns=title%2Cid%2Cactivity%2Cversions%2Cstatus&%40filter=priority%2Cstatus&priority=1&status=1&%40pagesize=50&%40startwith=0
>
> -- 
> Thanks
> Benjamin Peterson
> "There's no place like 127.0.0.1."
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com

From ncoghlan at gmail.com  Mon Jun  9 13:28:57 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 09 Jun 2008 21:28:57 +1000
Subject: [Python-Dev] Nab those release blockers!
In-Reply-To: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com>
References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com>
Message-ID: <484D13F9.30105@gmail.com>

Benjamin Peterson wrote:
> As we approach the planned first betas for 2.6/3.0, my heart sinks
> when I see the imposing list of 16 release blockers. [1] Luckily, most
> of the issues have patches that just need *your* review. Others,
> namely the Py3k exception issues, may require a little more
> discussion, but I'm sure we can get it done.

For issue 643841 (providing a ProxyMixin class in a new typetools 
module), aside from the minor tweak mentioned in the last couple of 
comments, I'm basically looking for a yea or nay from Barry or Guido. I 
think it's the best answer we're going to find to the problem of how to 
write type-agnostic delegation classes in a Python world without classic 
classes.

Cheers,
Nick.

http://bugs.python.org/issue643841


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From musiccomposition at gmail.com  Mon Jun  9 15:37:42 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 9 Jun 2008 08:37:42 -0500
Subject: [Python-Dev] Nab those release blockers!
In-Reply-To: <788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com>
References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com>
	<788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com>
Message-ID: <1afaf6160806090637w77786a23q9cac4ee7e12277cc@mail.gmail.com>

On Mon, Jun 9, 2008 at 5:33 AM, Jesse Noller <jnoller at gmail.com> wrote:
> For the multiprocessing module all I have left is the docs in rest format -
> given I'm a rest newbie, it's taking longer than I thought

If you want to email what you have done to docs at python.org, we'd love to help!


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From tom at vector-seven.com  Mon Jun  9 15:43:27 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Mon, 09 Jun 2008 23:43:27 +1000
Subject: [Python-Dev] Tuple pack/unpack and the definition of AST Assign
	nodes
Message-ID: <484D337F.8090201@vector-seven.com>

In porting one of the old peephole optimizations to the new AST compiler 
I noticed something weird going on with the following code:

a, b, c = 1, 2, 3

Now, as you would expect this gets parsed into an Assign node. That 
Assign node looks like the following:

Assign.targets = [Tuple(Name(a), Name(b), Name(c))]
Assign.value = Tuple(1, 2, 3)

What's weird here is that Assign.targets is an asdl_seq ... why are we 
wrapping the names in a Tuple() node? Shouldn't it look something more 
like this:

Assign.targets = [Name(a), Name(b), Name(c)]

I understand that parsing the testlist might yield a tuple and it was 
thus easier to just use the tuple rather than unpack it into an asdl_seq 
... but if this was the intention, then why is Assign.targets an expr* 
rather than a plain old expr?

Cheers,
Tom

From ncoghlan at gmail.com  Mon Jun  9 16:04:05 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Jun 2008 00:04:05 +1000
Subject: [Python-Dev] Tuple pack/unpack and the definition of AST Assign
 nodes
In-Reply-To: <484D337F.8090201@vector-seven.com>
References: <484D337F.8090201@vector-seven.com>
Message-ID: <484D3855.703@gmail.com>

Thomas Lee wrote:
> In porting one of the old peephole optimizations to the new AST compiler 
> I noticed something weird going on with the following code:
> 
> a, b, c = 1, 2, 3
> 
> Now, as you would expect this gets parsed into an Assign node. That 
> Assign node looks like the following:
> 
> Assign.targets = [Tuple(Name(a), Name(b), Name(c))]
> Assign.value = Tuple(1, 2, 3)
> 
> What's weird here is that Assign.targets is an asdl_seq ... why are we 
> wrapping the names in a Tuple() node? Shouldn't it look something more 
> like this:
> 
> Assign.targets = [Name(a), Name(b), Name(c)]
> 
> I understand that parsing the testlist might yield a tuple and it was 
> thus easier to just use the tuple rather than unpack it into an asdl_seq 
> ... but if this was the intention, then why is Assign.targets an expr* 
> rather than a plain old expr?

I haven't looked at that code recently, but I believe the ADSL sequence 
in the assignment node is for statements where there are actually 
multiple assignment targets, such as:

 >>> p = x, y = 1, 2
 >>> p, x, y
((1, 2), 1, 2)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From tom at vector-seven.com  Mon Jun  9 16:21:14 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Tue, 10 Jun 2008 00:21:14 +1000
Subject: [Python-Dev] Tuple pack/unpack and the definition of AST Assign
 nodes
In-Reply-To: <484D3855.703@gmail.com>
References: <484D337F.8090201@vector-seven.com> <484D3855.703@gmail.com>
Message-ID: <484D3C5A.6020808@vector-seven.com>

Nick Coghlan wrote:
>
> I haven't looked at that code recently, but I believe the ADSL 
> sequence in the assignment node is for statements where there are 
> actually multiple assignment targets, such as:
>
> >>> p = x, y = 1, 2
> >>> p, x, y
> ((1, 2), 1, 2)
>
> Cheers,
> Nick.
>
Ah I see. A quick test verifies exactly this:

    >>> import _ast
    >>> ast = compile("p = x, y = 1, 2", "<string>", "exec", 
_ast.PyCF_ONLY_AST)
    >>> ast.body[0]
    <_ast.Assign object at 0xb7d0122c>
    >>> ast.body[0].targets
    [<_ast.Name object at 0xb7d0124c>, <_ast.Tuple object at 0xb7d0128c>]
    >>> ast.body[0].value
    <_ast.Tuple object at 0xb7d0132c>
    >>>

I thought this would have been implemented as nested Assign nodes, but 
I'm obviously wrong. :) Thanks for the clarification.

Cheers,
T

From aleaxit at gmail.com  Mon Jun  9 17:12:42 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Mon, 9 Jun 2008 08:12:42 -0700
Subject: [Python-Dev] Assignment to None
In-Reply-To: <484CE993.5050506@voidspace.org.uk>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<200806091348.44361.steve@pearwood.info>
	<484CE993.5050506@voidspace.org.uk>
Message-ID: <e8a0972d0806090812w5e4d8bf2saeff4b963183080d@mail.gmail.com>

The problem is more general: what if a member  (of some external
object we're proxying one way or another) is named print (in Python <
3), or class, or...?  To allow foo.print or bar.class would require
pretty big changes to Python's parser -- I have vague memories that
the issue was discussed ages ago (possibly in conjunction with some
early release of Jython) but never went anywhere much (including
proposals to automatically append an underscore to such IDs in the
proxying layer, etc etc).  Maybe None in particular is enough of a
special case (if it just happens to be hugely often used in dotNET
libraries)?

Alex

On Mon, Jun 9, 2008 at 1:28 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> Steven D'Aprano wrote:
>>
>> On Mon, 9 Jun 2008 12:24:55 pm Curt Hagenlocher wrote:
>>
>>
>>>
>>> So, it's okay to setattr the attribute name "None" but not okay to set it
>>> directly?
>>
>> I suspect this is off-topic for python-dev, and would be better on
>> comp.lang.python or similar, but for what it's worth, I consider having an
>> attribute named 'None' bad practise, regardless of any runtime checks. But
>> as part of Python's "we're all consenting adults here" philosophy, I
>> wouldn't approve of expensive or extensive run-time checks specifically to
>> prevent it. If you really have to name your attribute None, and are prepared
>> to live with the consequences, then go ahead.
>>
>>
>
> FWIW Curt is asking because he is on the team responsible for implementing
> IronPython.
>
> In .NET you have several enumerations with members called None, for example
> 'Alignment.None'. It would be inconvenient for IronPython users if it
> enforced this particular syntax rule of Python's; that not only is
> assignment to None disallowed but that members named None are invalid
> syntax.
>
> The question is, what is the specification for Python. Is it that assignment
> to None is disallowed and the naming of members called None being invalid
> syntax is merely an artefact of the implementation of this, or does Python
> require this...
>
> Michael Foord
>
>> In a similar fashion:
>>
>>
>>>>>
>>>>> class Parrot(object):
>>>>>
>>
>> ...     pass
>> ...
>>
>>>>>
>>>>> p = Parrot()
>>>>> p.1 = 'spam'
>>>>>
>>
>>  File "<stdin>", line 1
>>    p.1
>>      ^
>> SyntaxError: invalid syntax
>>
>>>>>
>>>>> setattr(p, '1', 'spam')
>>>>>
>>>>>
>>
>>
>>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
>

From fuzzyman at voidspace.org.uk  Mon Jun  9 17:46:20 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 09 Jun 2008 16:46:20 +0100
Subject: [Python-Dev] Assignment to None
In-Reply-To: <e8a0972d0806090812w5e4d8bf2saeff4b963183080d@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	
	<200806091348.44361.steve@pearwood.info>	
	<484CE993.5050506@voidspace.org.uk>
	<e8a0972d0806090812w5e4d8bf2saeff4b963183080d@mail.gmail.com>
Message-ID: <484D504C.7060406@voidspace.org.uk>

Alex Martelli wrote:
> The problem is more general: what if a member  (of some external
> object we're proxying one way or another) is named print (in Python <
> 3), or class, or...?  To allow foo.print or bar.class would require
> pretty big changes to Python's parser -- I have vague memories that
> the issue was discussed ages ago (possibly in conjunction with some
> early release of Jython) but never went anywhere much (including
> proposals to automatically append an underscore to such IDs in the
> proxying layer, etc etc).  Maybe None in particular is enough of a
> special case (if it just happens to be hugely often used in dotNET
> libraries)?
>   

'None' as a member does occur particularly frequently in the .NET world.

A halfway house might be to state (something like):

    Python as a language disallows you from having names the same as 
keywords or 'None'. An implementation restriction specific to CPython is 
that the same restriction also applies to member names. Alternative 
implementations are free to not implement this restriction, with the 
caveat that code using reserved member names directly will be invalid 
syntax for CPython.

Although this appears to raise the spectre of code written for Jython or 
IronPython not being compatible with CPython, in reality if they are 
using Java or .NET objects then they are already incompatible with 
CPython (IronPython doesn't work by proxying .NET objects by the way - 
you work directly with them but the DLR 'catches' attribute look ups etc 
to add Python methods to basic types and do 'other magic' like wrapping 
Python functions as delegates).

This at least enshrines the current IronPython behaviour with the veneer 
of respectability.

Michael Foord

> Alex
>
> On Mon, Jun 9, 2008 at 1:28 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>   
>> Steven D'Aprano wrote:
>>     
>>> On Mon, 9 Jun 2008 12:24:55 pm Curt Hagenlocher wrote:
>>>
>>>
>>>       
>>>> So, it's okay to setattr the attribute name "None" but not okay to set it
>>>> directly?
>>>>         
>>> I suspect this is off-topic for python-dev, and would be better on
>>> comp.lang.python or similar, but for what it's worth, I consider having an
>>> attribute named 'None' bad practise, regardless of any runtime checks. But
>>> as part of Python's "we're all consenting adults here" philosophy, I
>>> wouldn't approve of expensive or extensive run-time checks specifically to
>>> prevent it. If you really have to name your attribute None, and are prepared
>>> to live with the consequences, then go ahead.
>>>
>>>
>>>       
>> FWIW Curt is asking because he is on the team responsible for implementing
>> IronPython.
>>
>> In .NET you have several enumerations with members called None, for example
>> 'Alignment.None'. It would be inconvenient for IronPython users if it
>> enforced this particular syntax rule of Python's; that not only is
>> assignment to None disallowed but that members named None are invalid
>> syntax.
>>
>> The question is, what is the specification for Python. Is it that assignment
>> to None is disallowed and the naming of members called None being invalid
>> syntax is merely an artefact of the implementation of this, or does Python
>> require this...
>>
>> Michael Foord
>>
>>     
>>> In a similar fashion:
>>>
>>>
>>>       
>>>>>> class Parrot(object):
>>>>>>
>>>>>>             
>>> ...     pass
>>> ...
>>>
>>>       
>>>>>> p = Parrot()
>>>>>> p.1 = 'spam'
>>>>>>
>>>>>>             
>>>  File "<stdin>", line 1
>>>    p.1
>>>      ^
>>> SyntaxError: invalid syntax
>>>
>>>       
>>>>>> setattr(p, '1', 'spam')
>>>>>>
>>>>>>
>>>>>>             
>>>
>>>       
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
>>
>>     


From tonynelson at georgeanelson.com  Mon Jun  9 18:46:31 2008
From: tonynelson at georgeanelson.com (Tony Nelson)
Date: Mon, 9 Jun 2008 12:46:31 -0400
Subject: [Python-Dev] Assignment to None
In-Reply-To: <484D504C.7060406@voidspace.org.uk>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	
	<200806091348.44361.steve@pearwood.info>	
	<484CE993.5050506@voidspace.org.uk>
	<e8a0972d0806090812w5e4d8bf2saeff4b963183080d@mail.gmail.com>
	<484D504C.7060406@voidspace.org.uk>
Message-ID: <p04330107c4730af274e7@[192.168.123.162]>

At 4:46 PM +0100 6/9/08, Michael Foord wrote:
>Alex Martelli wrote:
>> The problem is more general: what if a member  (of some external
>> object we're proxying one way or another) is named print (in Python <
>> 3), or class, or...?  To allow foo.print or bar.class would require
>> pretty big changes to Python's parser -- I have vague memories that
>> the issue was discussed ages ago (possibly in conjunction with some
>> early release of Jython) but never went anywhere much (including
>> proposals to automatically append an underscore to such IDs in the
>> proxying layer, etc etc).  Maybe None in particular is enough of a
>> special case (if it just happens to be hugely often used in dotNET
>> libraries)?
>>
>
>'None' as a member does occur particularly frequently in the .NET world.
>
>A halfway house might be to state (something like):
>
>    Python as a language disallows you from having names the same as
>keywords or 'None'. An implementation restriction specific to CPython is
>that the same restriction also applies to member names. Alternative
>implementations are free to not implement this restriction, with the
>caveat that code using reserved member names directly will be invalid
>syntax for CPython.
 ...

Or perhaps CPython should just stop trying to detect this at compile time.
Note that while assignment to ".None" is not allowed, setattr(foo, "None",
1) then referencing ".None" is allowed.

>>> f.None = 1
SyntaxError: assignment to None
>>> f.None
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'Foo' object has no attribute 'None'
>>> setattr(f, 'None', 1)
> f.None
1
>>>
-- 
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson at georgeanelson.com>
      '                              <http://www.georgeanelson.com/>

From tom at vector-seven.com  Mon Jun  9 19:02:00 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Tue, 10 Jun 2008 03:02:00 +1000
Subject: [Python-Dev] Assignment to None
In-Reply-To: <p04330107c4730af274e7@[192.168.123.162]>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>		<200806091348.44361.steve@pearwood.info>		<484CE993.5050506@voidspace.org.uk>	<e8a0972d0806090812w5e4d8bf2saeff4b963183080d@mail.gmail.com>	<484D504C.7060406@voidspace.org.uk>
	<p04330107c4730af274e7@[192.168.123.162]>
Message-ID: <484D6208.40503@vector-seven.com>

Tony Nelson wrote:
> At 4:46 PM +0100 6/9/08, Michael Foord wrote:
>   
>> Alex Martelli wrote:
>>     
>>> The problem is more general: what if a member  (of some external
>>> object we're proxying one way or another) is named print (in Python <
>>> 3), or class, or...?  To allow foo.print or bar.class would require
>>> pretty big changes to Python's parser -- I have vague memories that
>>> the issue was discussed ages ago (possibly in conjunction with some
>>> early release of Jython) but never went anywhere much (including
>>> proposals to automatically append an underscore to such IDs in the
>>> proxying layer, etc etc).  Maybe None in particular is enough of a
>>> special case (if it just happens to be hugely often used in dotNET
>>> libraries)?
>>>
>>>       
>> 'None' as a member does occur particularly frequently in the .NET world.
>>
>> A halfway house might be to state (something like):
>>
>>    Python as a language disallows you from having names the same as
>> keywords or 'None'. An implementation restriction specific to CPython is
>> that the same restriction also applies to member names. Alternative
>> implementations are free to not implement this restriction, with the
>> caveat that code using reserved member names directly will be invalid
>> syntax for CPython.
>>     
>  ...
>
> Or perhaps CPython should just stop trying to detect this at compile time.
> Note that while assignment to ".None" is not allowed, setattr(foo, "None",
> 1) then referencing ".None" is allowed.
>
>   
I'm +0 on this at the moment, but I can understand the desire for it. 
Maybe we should stop trying to check for this assignment on attributes?

Currently there are separate checks for assignment to None: one for the 
"foo.None = ..." form, another for the "None = ..." form. Removing the 
check for the former looks like it would be a one-liner.

Cheers,
T

From tnelson at onresolve.com  Mon Jun  9 19:50:01 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Mon, 9 Jun 2008 18:50:01 +0100
Subject: [Python-Dev] Obtaining short file path
In-Reply-To: <2EDCAD28478FC346B0C7A59FA49448B507002C96@MSX102V1.corp.int>
References: <2EDCAD28478FC346B0C7A59FA49448B507002C96@MSX102V1.corp.int>
Message-ID: <6167796BFEB5D0438720AC212E89A6B007872732@exchange.onresolve.com>

If you want a short path name, you should use win32api.GetShortPathName().  Attempting to compute it yourself isn?t as straight forward as you think.

From: python-dev-bounces+tnelson=onresolve.com at python.org [mailto:python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Hartwell Bryan
Sent: 27 May 2008 08:00
To: python-dev at python.org
Subject: [Python-Dev] Obtaining short file path


Hi,

Purpose: obtaining the system (?short?) path from a full path

Background: File dialogs (visual studio) return a full path (e.g. f=?C:\this path has spaces\thisfilenameislongerthan8char.txt?). If this value is provided to Python, it will not recongize this as a file. In fact os.path.isfile(f) doesn?t return false, it crashes. Likewise, when calling executables (from Python) with files as arguments a short path is required. VB FileSystemObject has the ShortPath method, while os.path and path (www.jorendorff.com) modules do not (at least as far as my googling could determine). Why bother creating a COM interface when you?re just going to pass as shell run-time arguments all the values the server is better at computing?

System: Python 2.3; Windows XP

Sample Code:

import win32com.client

import time

import os,sys

import os.path

#-------------------------------------------------------------

def shortpath(x):

  z=''

  for y in x.split('\\'):

    if len(y.split('.')[0])>8:

      if ('.' in y):

        z=z+'\\'+y.split('.')[0][:6].upper()+'~1'+'.'+y.split('.')[1]

      else:

        z=z+'\\'+y[:6].upper()+'~1'

    else:

      z=z+'\\'+y

  return z[1:]

#-------------------------------------------------------------

xlApp = win32com.client.Dispatch("Excel.Application")

xlBook = xlApp.ActiveWorkbook

savFile = str(sys.argv[1])

rawFile = str(xlBook.Sheets("Timestamp").TextBox2)

#print os.path.isfile(savFile)

r=shortpath(rawFile)

print r

try:

  print os.path.isfile(r)

except:

  print 'something rude'

time.sleep(7)

Notes: This code does not account for peer paths or files that share the first 8 characters (and file extension). I?m also aware that this is not the normal means for submitting a ?patch?, but in my job function I don?t see myself regularly participating in python development (and I?m probably not savvy enough) so the effort wasn?t worth it. However I still thought others might benefit from what seems to be (to me) a fundamental path function. Do with it, or ignore it, as you please.

Cheers,

Bryan Hartwell

________________________________

This message is intended only for the use of the intended recipients, and it may be privileged and confidential. If you are not the intended recipient, you are hereby notified that any review, retransmission, conversion to hard copy, copying, circulation or other use of this message is strictly prohibited. If you are not the intended recipient, please notify me immediately by return e-mail, and delete this message from your system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080609/5786306b/attachment.htm>

From gisle at activestate.com  Mon Jun  9 19:18:10 2008
From: gisle at activestate.com (Gisle Aas)
Date: Mon, 9 Jun 2008 19:18:10 +0200
Subject: [Python-Dev] Assignment to None
In-Reply-To: <e8a0972d0806090812w5e4d8bf2saeff4b963183080d@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<200806091348.44361.steve@pearwood.info>
	<484CE993.5050506@voidspace.org.uk>
	<e8a0972d0806090812w5e4d8bf2saeff4b963183080d@mail.gmail.com>
Message-ID: <EBF7DA6F-1678-4317-9C87-09D1318A5F77@activestate.com>

On Jun 9, 2008, at 17:12, Alex Martelli wrote:

> The problem is more general: what if a member  (of some external
> object we're proxying one way or another) is named print (in Python <
> 3), or class, or...?  To allow foo.print or bar.class would require
> pretty big changes to Python's parser

I simple solution to this would be to introduce syntax that allow you  
to "quote" identifiers that would otherwise be lexed as keywords.   
For instance C# uses a prefix '@' for such quoting, so you could  
write foo. at print, bar. at class and baz. at None if you insist on using  
such identifiers.  Too ugly for Python I guess.

Regards,
Gisle


From tjreedy at udel.edu  Mon Jun  9 21:22:39 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 9 Jun 2008 15:22:39 -0400
Subject: [Python-Dev] Assignment to None
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk><e8a0972d0806090812w5e4d8bf2saeff4b963183080d@mail.gmail.com><484D504C.7060406@voidspace.org.uk>
	<p04330107c4730af274e7@[192.168.123.162]>
Message-ID: <g2jvu0$5uq$1@ger.gmane.org>


"Tony Nelson" <tonynelson at georgeanelson.com> wrote in message 
news:p04330107c4730af274e7@[192.168.123.162]...
| At 4:46 PM +0100 6/9/08, Michael Foord wrote:
| Or perhaps CPython should just stop trying to detect this at compile 
time.
| Note that while assignment to ".None" is not allowed, setattr(foo, 
"None",
| 1) then referencing ".None" is allowed.
|
| >>> f.None = 1
| SyntaxError: assignment to None
| >>> f.None
| Traceback (most recent call last):
|  File "<stdin>", line 1, in ?
| AttributeError: 'Foo' object has no attribute 'None'
| >>> setattr(f, 'None', 1)
| > f.None
| 1

I was a little surprised by this.  ISTM that f.None should consistently 
work or not work both for getting and setting.  The last result could be 
considered a bug rusulting from getting disabling depending on None not 
being set, but having a backdoor that does allow it to be set.

I recently read someone (Guido?) saying that a language should start 
'strict' because it is much easier to allow something new than disallow 
something old.  Allowing 'None' and possibly other keywords as attributes 
will not break code (that does not depend on the exceptions).   But I 
presume there was some thought that the restriction might prevent buggy 
code.

At the global level, the subversion does not work:
>>> globals()['None'] = 'ha'
>>> None
>>> dir()
['None', '__builtins__', '__doc__', '__name__', '__package__']
>>> globals()['None']
'ha'

tjr




From martin at v.loewis.de  Mon Jun  9 22:36:05 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jun 2008 22:36:05 +0200
Subject: [Python-Dev] Assignment to None
In-Reply-To: <g2jvu0$5uq$1@ger.gmane.org>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk><e8a0972d0806090812w5e4d8bf2saeff4b963183080d@mail.gmail.com><484D504C.7060406@voidspace.org.uk>	<p04330107c4730af274e7@[192.168.123.162]>
	<g2jvu0$5uq$1@ger.gmane.org>
Message-ID: <484D9435.5020907@v.loewis.de>

> At the global level, the subversion does not work:

I think you are misinterpreting what you are seeing. When you refer to
the global identifier None, the compiler just knows that it must be
the NoneType singleton, and returns it as a constant, without doing any
name lookup. So it isn't that assignment of the global None fails to
be performed; instead, the compiler just ignores any global with the
name None that might have been set.

The compiler doesn't do the same thing for attributes, and rightfully
so: it would surely be surprising if foo.None would yield the NoneType
singleton for any arbitrary foo. Instead, the compiler generates a
regular attribute, which then regularly finds the attribute in the
object's dictionary if it is there.

> I was a little surprised by this.  ISTM that f.None should
> consistently work or not work both for getting and setting.

That will only happen when (if) None becomes a keywords: then
foo.None will be a syntax error. You will still be able to put
attributes named 'None' (or 'import' or 'if') into any object
using getattr/setattr, or by direct access to __dict__.

Regards,
Martin

From martin at v.loewis.de  Mon Jun  9 23:15:05 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jun 2008 23:15:05 +0200
Subject: [Python-Dev] Assignment to None
In-Reply-To: <484CE993.5050506@voidspace.org.uk>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>
	<484CE993.5050506@voidspace.org.uk>
Message-ID: <484D9D59.903@v.loewis.de>

> The question is, what is the specification for Python.

Now, that's a more interesting question than the question originally
asked (which I interpreted as "why does it work the way it works").

The only indication in the specification of that feature I could find
was:

http://docs.python.org/dev/library/constants.html

"Changed in version 2.4: Assignments to None are illegal and raise a
SyntaxError."

Now, given that this talks about the built-in namespace, this *doesn't*
specify that foo.None=1 should also raise a syntax error.

So the implementation apparently deviates from the specification.

In Python 3, None, True, and False are keywords, so clearly, the
intended semantics is also the implemented one (and the language
description for 2.x needs to be updated/clarified).

Regards,
Martin


From josiah.carlson at gmail.com  Tue Jun 10 03:42:32 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Mon, 9 Jun 2008 18:42:32 -0700
Subject: [Python-Dev] asyncore patch
Message-ID: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com>

As we approach the 2.6 beta date, and after getting my updated public
key pushed to the python.org servers, I would really like to get the
asyncore/asynchat patch (with documentation) committed.  Previously,
we were waiting on documentation, which the last patch had, but which
was > 80 columns.  That is now fixed.  Neal had promised to commit if
I got everything in order, but now that I have access, I can commit
it.

Would it be ok if I committed the changes?  Neal, do you want to
commit the changes if I post an updated patch with a blurb for the
NEWS file?

What can I do to get these patch in and included with the beta?

 - Josiah

From musiccomposition at gmail.com  Tue Jun 10 04:19:01 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 9 Jun 2008 21:19:01 -0500
Subject: [Python-Dev] asyncore patch
In-Reply-To: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com>
References: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com>
Message-ID: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com>

On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson <josiah.carlson at gmail.com> wrote:
>
> Would it be ok if I committed the changes?  Neal, do you want to
> commit the changes if I post an updated patch with a blurb for the
> NEWS file?

You are the asyncore maintainer, correct? I believe it's pretty much
up to you, then. :)



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From josiah.carlson at gmail.com  Tue Jun 10 07:01:59 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Mon, 9 Jun 2008 22:01:59 -0700
Subject: [Python-Dev] asyncore patch
In-Reply-To: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com>
References: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com>
	<1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com>
Message-ID: <e6511dbf0806092201i5a983f1fu558e7465739635da@mail.gmail.com>

On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson <josiah.carlson at gmail.com> wrote:
>>
>> Would it be ok if I committed the changes?  Neal, do you want to
>> commit the changes if I post an updated patch with a blurb for the
>> NEWS file?
>
> You are the asyncore maintainer, correct? I believe it's pretty much
> up to you, then. :)

Indeed, but I didn't want to step on anyone's toes.

It's committed in revision 64062 for anyone who cares.

 - Josiah

From tjreedy at udel.edu  Tue Jun 10 08:11:12 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 10 Jun 2008 02:11:12 -0400
Subject: [Python-Dev] Assignment to None
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk><e8a0972d0806090812w5e4d8bf2saeff4b963183080d@mail.gmail.com><484D504C.7060406@voidspace.org.uk>	<p04330107c4730af274e7@[192.168.123.162]><g2jvu0$5uq$1@ger.gmane.org>
	<484D9435.5020907@v.loewis.de>
Message-ID: <g2l5u0$507$1@ger.gmane.org>


""Martin v. L?wis"" <martin at v.loewis.de> wrote in message 
news:484D9435.5020907 at v.loewis.de...
|> At the global level, the subversion does not work:
|
| I think you are misinterpreting what you are seeing.

No, you didn't understand the code I posted which explicitely demonstrated 
the same point you repeated.  Again:

>>> globals()['None'] = 'ha'
>>> None
>>> dir()
['None', '__builtins__', '__doc__', '__name__', '__package__']
>>> globals()['None']
'ha'

The point of the the last two lines is that the assignment *was* made but 
was ignored in line 2 [because, obviously, the interpreter bypasses the 
normal lookup procedure].




From ncoghlan at gmail.com  Tue Jun 10 13:04:32 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Jun 2008 21:04:32 +1000
Subject: [Python-Dev] Tuple pack/unpack and the definition of AST Assign
 nodes
In-Reply-To: <484D3C5A.6020808@vector-seven.com>
References: <484D337F.8090201@vector-seven.com> <484D3855.703@gmail.com>
	<484D3C5A.6020808@vector-seven.com>
Message-ID: <484E5FC0.80900@gmail.com>

Thomas Lee wrote:
> Nick Coghlan wrote:
>>
>> I haven't looked at that code recently, but I believe the ADSL 
>> sequence in the assignment node is for statements where there are 
>> actually multiple assignment targets, such as:
>>
>> >>> p = x, y = 1, 2
>> >>> p, x, y
>> ((1, 2), 1, 2)
>>
>> Cheers,
>> Nick.
>>
> Ah I see. A quick test verifies exactly this:
> 
>    >>> import _ast
>    >>> ast = compile("p = x, y = 1, 2", "<string>", "exec", 
> _ast.PyCF_ONLY_AST)
>    >>> ast.body[0]
>    <_ast.Assign object at 0xb7d0122c>
>    >>> ast.body[0].targets
>    [<_ast.Name object at 0xb7d0124c>, <_ast.Tuple object at 0xb7d0128c>]
>    >>> ast.body[0].value
>    <_ast.Tuple object at 0xb7d0132c>
>    >>>
> 
> I thought this would have been implemented as nested Assign nodes, but 
> I'm obviously wrong. :) Thanks for the clarification.

It's one of the key differences between Python's "assignment as a 
statement" and C-style "assignment as an expression" - nested Assignment 
nodes are meaningful with the latter approach, but nonsensical with the 
way Python defines assignment (the rightmost expression is evaluated 
once and then assigned to each of the target expressions from left to 
right).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Tue Jun 10 13:12:11 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Jun 2008 21:12:11 +1000
Subject: [Python-Dev] asyncore patch
In-Reply-To: <e6511dbf0806092201i5a983f1fu558e7465739635da@mail.gmail.com>
References: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com>	<1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com>
	<e6511dbf0806092201i5a983f1fu558e7465739635da@mail.gmail.com>
Message-ID: <484E618B.60502@gmail.com>

Josiah Carlson wrote:
> On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson <josiah.carlson at gmail.com> wrote:
>>> Would it be ok if I committed the changes?  Neal, do you want to
>>> commit the changes if I post an updated patch with a blurb for the
>>> NEWS file?
>> You are the asyncore maintainer, correct? I believe it's pretty much
>> up to you, then. :)
> 
> Indeed, but I didn't want to step on anyone's toes.
> 
> It's committed in revision 64062 for anyone who cares.

I'm sure asyncore users will appreciate the work put into this, even if 
there's something of a shortage of such people on this list :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From g.brandl at gmx.net  Tue Jun 10 15:08:37 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 10 Jun 2008 15:08:37 +0200
Subject: [Python-Dev] asyncore patch
In-Reply-To: <484E618B.60502@gmail.com>
References: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com>	<1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com>	<e6511dbf0806092201i5a983f1fu558e7465739635da@mail.gmail.com>
	<484E618B.60502@gmail.com>
Message-ID: <g2lucm$lss$1@ger.gmane.org>

Nick Coghlan schrieb:
> Josiah Carlson wrote:
>> On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson
>> <musiccomposition at gmail.com> wrote:
>>> On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson <josiah.carlson at gmail.com> wrote:
>>>> Would it be ok if I committed the changes?  Neal, do you want to
>>>> commit the changes if I post an updated patch with a blurb for the
>>>> NEWS file?
>>> You are the asyncore maintainer, correct? I believe it's pretty much
>>> up to you, then. :)
>> 
>> Indeed, but I didn't want to step on anyone's toes.
>> 
>> It's committed in revision 64062 for anyone who cares.
> 
> I'm sure asyncore users will appreciate the work put into this, even if 
> there's something of a shortage of such people on this list :)

I don't want to be picky, but it seems the commit broke the tests:

test test_asyncore failed -- Traceback (most recent call last):
   File "/tmp/python-test/local/lib/python2.6/test/test_asyncore.py", line 387, 
in test_recv
     self.assertEqual(w.fd, fd)
AssertionError: 6 != 3

Looks like a minor oversight.

Georg


From barry at python.org  Tue Jun 10 15:10:26 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 10 Jun 2008 09:10:26 -0400
Subject: [Python-Dev] asyncore patch
In-Reply-To: <g2lucm$lss$1@ger.gmane.org>
References: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com>	<1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com>	<e6511dbf0806092201i5a983f1fu558e7465739635da@mail.gmail.com>
	<484E618B.60502@gmail.com> <g2lucm$lss$1@ger.gmane.org>
Message-ID: <3555721E-D086-49E8-8E1B-F79D1967E7A1@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 10, 2008, at 9:08 AM, Georg Brandl wrote:

> Nick Coghlan schrieb:
>> Josiah Carlson wrote:
>>> On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson
>>> <musiccomposition at gmail.com> wrote:
>>>> On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson <josiah.carlson at gmail.com 
>>>> > wrote:
>>>>> Would it be ok if I committed the changes?  Neal, do you want to
>>>>> commit the changes if I post an updated patch with a blurb for the
>>>>> NEWS file?
>>>> You are the asyncore maintainer, correct? I believe it's pretty  
>>>> much
>>>> up to you, then. :)
>>> Indeed, but I didn't want to step on anyone's toes.
>>> It's committed in revision 64062 for anyone who cares.
>> I'm sure asyncore users will appreciate the work put into this,  
>> even if there's something of a shortage of such people on this  
>> list :)
>
> I don't want to be picky, but it seems the commit broke the tests:
>
> test test_asyncore failed -- Traceback (most recent call last):
>  File "/tmp/python-test/local/lib/python2.6/test/test_asyncore.py",  
> line 387, in test_recv
>    self.assertEqual(w.fd, fd)
> AssertionError: 6 != 3
>
> Looks like a minor oversight.

This needs to be fixed or backed out if we're going to make beta  
tomorrow.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSE59Q3EjvBPtnXfVAQLVsQQAsbk6VvlabljNtDX63e9DvBnt8Gh15l0W
/yw+lRp9BEThLwitp1ZllFbZElPHsyKaFJn16yOhrysjp/Hf1zWJYgYcHee1m/P4
o4eJMgYlkgIymolJFiviirPj/4YPC8XYkGVcPs1so7XvwVfRvn/u+uhmf1y/+Uwo
ql6fkLjXVMo=
=iLdD
-----END PGP SIGNATURE-----

From musiccomposition at gmail.com  Tue Jun 10 15:12:33 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 10 Jun 2008 08:12:33 -0500
Subject: [Python-Dev] asyncore patch
In-Reply-To: <3555721E-D086-49E8-8E1B-F79D1967E7A1@python.org>
References: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com>
	<1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com>
	<e6511dbf0806092201i5a983f1fu558e7465739635da@mail.gmail.com>
	<484E618B.60502@gmail.com> <g2lucm$lss$1@ger.gmane.org>
	<3555721E-D086-49E8-8E1B-F79D1967E7A1@python.org>
Message-ID: <1afaf6160806100612l50581aefx736d3e72bafcbeab@mail.gmail.com>

On Tue, Jun 10, 2008 at 8:10 AM, Barry Warsaw <barry at python.org> wrote:
>> I don't want to be picky, but it seems the commit broke the tests:
>>
>> test test_asyncore failed -- Traceback (most recent call last):
>>  File "/tmp/python-test/local/lib/python2.6/test/test_asyncore.py", line
>> 387, in test_recv
>>   self.assertEqual(w.fd, fd)
>> AssertionError: 6 != 3
>>
>> Looks like a minor oversight.
>
> This needs to be fixed or backed out if we're going to make beta tomorrow.

Oh, I think there are more pressing issues than this, but FWIW I made
a issue for it: 3074


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From josiah.carlson at gmail.com  Tue Jun 10 16:56:31 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Tue, 10 Jun 2008 07:56:31 -0700
Subject: [Python-Dev] asyncore patch
In-Reply-To: <1afaf6160806100612l50581aefx736d3e72bafcbeab@mail.gmail.com>
References: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com>
	<1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com>
	<e6511dbf0806092201i5a983f1fu558e7465739635da@mail.gmail.com>
	<484E618B.60502@gmail.com> <g2lucm$lss$1@ger.gmane.org>
	<3555721E-D086-49E8-8E1B-F79D1967E7A1@python.org>
	<1afaf6160806100612l50581aefx736d3e72bafcbeab@mail.gmail.com>
Message-ID: <e6511dbf0806100756y5264db0u874ba999e039d22@mail.gmail.com>

I'm working on it now.  I'll do my best to have a fix by the time I go
to work this morning.

 - Josiah

On Tue, Jun 10, 2008 at 6:12 AM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Tue, Jun 10, 2008 at 8:10 AM, Barry Warsaw <barry at python.org> wrote:
>>> I don't want to be picky, but it seems the commit broke the tests:
>>>
>>> test test_asyncore failed -- Traceback (most recent call last):
>>>  File "/tmp/python-test/local/lib/python2.6/test/test_asyncore.py", line
>>> 387, in test_recv
>>>   self.assertEqual(w.fd, fd)
>>> AssertionError: 6 != 3
>>>
>>> Looks like a minor oversight.
>>
>> This needs to be fixed or backed out if we're going to make beta tomorrow.
>
> Oh, I think there are more pressing issues than this, but FWIW I made
> a issue for it: 3074
>
>
> --
> Cheers,
> Benjamin Peterson
> "There's no place like 127.0.0.1."
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/josiah.carlson%40gmail.com
>

From gnewsg at gmail.com  Tue Jun 10 17:26:59 2008
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Tue, 10 Jun 2008 08:26:59 -0700 (PDT)
Subject: [Python-Dev] asyncore patch
In-Reply-To: <e6511dbf0806092201i5a983f1fu558e7465739635da@mail.gmail.com>
References: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com> 
	<1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> 
	<e6511dbf0806092201i5a983f1fu558e7465739635da@mail.gmail.com>
Message-ID: <ae51a594-6dcd-4eff-9f01-e8ad20958103@c65g2000hsa.googlegroups.com>

On 10 Giu, 07:01, "Josiah Carlson" <josiah.carl... at gmail.com> wrote:
> On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson
>
> <musiccomposit... at gmail.com> wrote:
> > On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson <josiah.carl... at gmail.com> wrote:
>
> >> Would it be ok if I committed the changes? ?Neal, do you want to
> >> commit the changes if I post an updated patch with a blurb for the
> >> NEWS file?
>
> > You are the asyncore maintainer, correct? I believe it's pretty much
> > up to you, then. :)
>
> Indeed, but I didn't want to step on anyone's toes.
>
> It's committed in revision 64062 for anyone who cares.
>
> ?- Josiah


I've started to test the new code by using the pyftpdlib test suite.
On Windows all tests passed but on Linux I get some "EBADF Bad file
descriptor" errors occurring when using poll() instead of select().
I'll try to look into them today and open a report if necessary.
In the meanwhile I noticed some minor bugs in asyncore.py. Take a look
at the patch below:


Index: Lib/asyncore.py
===================================================================
--- Lib/asyncore.py	(revisione 64069)
+++ Lib/asyncore.py	(copia locale)
@@ -228,7 +228,7 @@
             # passed be connected.
             try:
                 self.addr = sock.getpeername()
-            except socket.error:
+            except socket.error, err:
                 if err[0] == ENOTCONN:
                     # To handle the case where we got an unconnected
                     # socket.
@@ -424,7 +424,7 @@
             #check for errors
             err = self.socket.getsockopt(socket.SOL_SOCKET,
socket.SO_ERROR)
             if err != 0:
-                raise socket.error(err, strerror(err))
+                raise socket.error(err, _strerror(err))

             self.handle_connect_event()
         self.handle_write()


--- Giampaolo
http://code.google.com/p/pyftpdlib/

From tom at vector-seven.com  Tue Jun 10 17:52:07 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Wed, 11 Jun 2008 01:52:07 +1000
Subject: [Python-Dev] Assignment to None
In-Reply-To: <484D9D59.903@v.loewis.de>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk>
	<484D9D59.903@v.loewis.de>
Message-ID: <484EA327.2050704@vector-seven.com>

Martin v. L?wis wrote:
>> The question is, what is the specification for Python.
>>     
>
> Now, that's a more interesting question than the question originally
> asked (which I interpreted as "why does it work the way it works").
>
> The only indication in the specification of that feature I could find
> was:
>
> http://docs.python.org/dev/library/constants.html
>
> "Changed in version 2.4: Assignments to None are illegal and raise a
> SyntaxError."
>
> Now, given that this talks about the built-in namespace, this *doesn't*
> specify that foo.None=1 should also raise a syntax error.
>
> So the implementation apparently deviates from the specification.
>
> In Python 3, None, True, and False are keywords, so clearly, the
> intended semantics is also the implemented one (and the language
> description for 2.x needs to be updated/clarified).
>
>   
Interestingly enough, the semantics of True, False and None are 
different from one another in 2.6:

True = "blah" and False = 6 are perfectly legal in Python <=2.6.

Funny, I just ran into this. I was trying to figure out why the AST 
optimization code was breaking test_xmlrpc ... turns out xmlrpclib 
defines xmlrpclib.True and xmlrpclib.False and the optimizer was trying 
to resolve them as constants while compiling the module. Ouch.

What happened in 3k? Were the constants in xmlrpclib renamed/removed?

Cheers,
T


From josiah.carlson at gmail.com  Tue Jun 10 17:58:33 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Tue, 10 Jun 2008 08:58:33 -0700
Subject: [Python-Dev] asyncore patch
In-Reply-To: <ae51a594-6dcd-4eff-9f01-e8ad20958103@c65g2000hsa.googlegroups.com>
References: <e6511dbf0806091842j54af4032ib85eafd23b88e96b@mail.gmail.com>
	<1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com>
	<e6511dbf0806092201i5a983f1fu558e7465739635da@mail.gmail.com>
	<ae51a594-6dcd-4eff-9f01-e8ad20958103@c65g2000hsa.googlegroups.com>
Message-ID: <e6511dbf0806100858xa05901fre18c9030adac4774@mail.gmail.com>

On Tue, Jun 10, 2008 at 8:26 AM, Giampaolo Rodola' <gnewsg at gmail.com> wrote:
> On 10 Giu, 07:01, "Josiah Carlson" <josiah.carl... at gmail.com> wrote:
>> On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson
>>
>> <musiccomposit... at gmail.com> wrote:
>> > On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson <josiah.carl... at gmail.com> wrote:
>>
>> >> Would it be ok if I committed the changes?  Neal, do you want to
>> >> commit the changes if I post an updated patch with a blurb for the
>> >> NEWS file?
>>
>> > You are the asyncore maintainer, correct? I believe it's pretty much
>> > up to you, then. :)
>>
>> Indeed, but I didn't want to step on anyone's toes.
>>
>> It's committed in revision 64062 for anyone who cares.
>>
>>  - Josiah
>
>
> I've started to test the new code by using the pyftpdlib test suite.
> On Windows all tests passed but on Linux I get some "EBADF Bad file
> descriptor" errors occurring when using poll() instead of select().
> I'll try to look into them today and open a report if necessary.
> In the meanwhile I noticed some minor bugs in asyncore.py. Take a look
> at the patch below:
>
>
> Index: Lib/asyncore.py
> ===================================================================
> --- Lib/asyncore.py     (revisione 64069)
> +++ Lib/asyncore.py     (copia locale)
> @@ -228,7 +228,7 @@
>             # passed be connected.
>             try:
>                 self.addr = sock.getpeername()
> -            except socket.error:
> +            except socket.error, err:
>                 if err[0] == ENOTCONN:
>                     # To handle the case where we got an unconnected
>                     # socket.
> @@ -424,7 +424,7 @@
>             #check for errors
>             err = self.socket.getsockopt(socket.SOL_SOCKET,
> socket.SO_ERROR)
>             if err != 0:
> -                raise socket.error(err, strerror(err))
> +                raise socket.error(err, _strerror(err))
>
>             self.handle_connect_event()
>         self.handle_write()

I have fixed the testcase (it was failing due to the new semantics of
the filewrapper automatically duping the file handle), as well as the
two NameErrors that Giampaolo pointed out.  See revision 64080.

 - Josiah

From exarkun at divmod.com  Tue Jun 10 18:04:03 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 10 Jun 2008 12:04:03 -0400
Subject: [Python-Dev] segfault in struct module
In-Reply-To: 0
Message-ID: <20080610160403.4714.104530664.divmod.quotient.7701@ohm>


    exarkun at charm:~$ ~/Projects/python/trunk/python
    Python 2.6a3+ (trunk:63964, Jun  5 2008, 16:49:12)
    [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import struct
    >>> struct.pack("357913941c", 'a')
    Segmentation fault
    exarkun at charm:~$

The unit test for exactly this case was deleted in r60892.  I would like to
suggest that just deleting unit tests isn't a very good idea.

Jean-Paul

From hodgestar+pythondev at gmail.com  Tue Jun 10 18:31:13 2008
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Tue, 10 Jun 2008 18:31:13 +0200
Subject: [Python-Dev] Exception.__unicode__ and tp_unicode
Message-ID: <fb73205e0806100931y68160721w2a134e1e8204993b@mail.gmail.com>

Originally Python exceptions had no __unicode__ method. In Python 2.5
__unicode__ was added. This led to "unicode(Exception)" failing and so
the addition of __unicode__ was reverted [1].

This leaves Python 2.6 in a position where calls to
"unicode(Exception(u'\xe1'))" fail as they are equivalent to
"uncode(str(Exception(u'\xe1'))" which cannot convert the non-ASCII
character to ASCII (or other default encoding) [2].

>From here there are 3 options:

 1) Leave things as they are.
 2) Add back __unicode__ and have "unicode(Exception)" fail.
 3) Add a tp_unicode slot to Python objects and have everything work
(at the cost of adding the slot).

Each option has its draw backs.

Ideally I'd like to see 3) implemented (there are already two
volunteers for and some initial stabs at implementing it) but a change
to Object is going to need an okay from someone quite high up. Also,
if you know of any code this would break, now is the time to let me
know.

If we can't have 3) I'd like to see us fall back to option 2). Passing
unicode exceptions back is useful in a number of common situations
(non-English exception messages, database errors, pretty much anywhere
that something goes wrong while dealing with potentially non-ASCII
text) and encoding to some specific format is usually not an option
since there is no way to know where the exception will eventually be
caught. Also, unicode(ClassA) already fails for any class that
implements __unicode__ so even without this effecting Exception it's
already not safe to do u"%s" % SomeClass. Also, there is a readily
available work around by doing u"%s" % str(SomeClass).

I'm opposed to 1) because a full work around means doing something like:

  def unicode_exception(e):
    if len(e.args) == 0:
      return u""
    elif len(e.args) == 1:
      return unicode(e.args[0])
    else:
      return unicode(e.args)

and then using unicode_exception(...) instead of unicode(...) whenever
one needs to get a unicode value for an exception.

The issue doesn't affect Python 3.0 where unicode(...) is replaced by str(...).

[1] http://bugs.python.org/issue1551432
[2] http://bugs.python.org/issue2517

Schiavo
Simon

From g.brandl at gmx.net  Tue Jun 10 18:39:22 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 10 Jun 2008 18:39:22 +0200
Subject: [Python-Dev] Assignment to None
In-Reply-To: <484EA327.2050704@vector-seven.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk>	<484D9D59.903@v.loewis.de>
	<484EA327.2050704@vector-seven.com>
Message-ID: <g2manq$62l$1@ger.gmane.org>

Thomas Lee schrieb:

> What happened in 3k? Were the constants in xmlrpclib renamed/removed?

They were removed, as there is no way they can be accessed as attributes of a
module now.

Georg


From dje at google.com  Tue Jun 10 19:13:46 2008
From: dje at google.com (Doug Evans)
Date: Tue, 10 Jun 2008 10:13:46 -0700 (PDT)
Subject: [Python-Dev] bogus comment in Python.h
Message-ID: <20080610171346.52EE01C7641@localhost>

I spent a bit of time trying to figure out what's going on here
(was getting errors regarding missing uintptr_t while trying to compile
an external module with Python 2.4).
pyport.h now includes stdint.h but can we fix up this in Python.h?

/* For uintptr_t, intptr_t */
#ifdef HAVE_STDDEF_H
#include <stddef.h>
#endif

I'm guessing removing the inclusion of stddef.h will break something.
It's a bit of a wart (or not) that Python.h includes stdlib.h, stddef.h, et.al.
but pyport.h includes stdint.h (why not just include them all in one place?).

Anyways, to save some confusion for someone trying to track down
a problem in the future, could the above comment be removed?

--- Python.h	(revision 64082)
+++ Python.h	(working copy)
@@ -43,8 +43,6 @@
 #ifdef HAVE_UNISTD_H
 #include <unistd.h>
 #endif
-
-/* For uintptr_t, intptr_t */
 #ifdef HAVE_STDDEF_H
 #include <stddef.h>
 #endif

[I'd suggest rewording it except that I'm not sure exactly what stddef.h
is used for.  Presumably it's size_t but I didn't want to guess.]

From jnoller at gmail.com  Tue Jun 10 21:11:55 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 10 Jun 2008 15:11:55 -0400
Subject: [Python-Dev] PEP 371 trunk implementation
Message-ID: <4222a8490806101211m5999112ct4a2adb9f92a57789@mail.gmail.com>

I've uploaded the first "rolled up" patch against trunk for PEP 371's
implementation. This includes docs, tests, module, the works.

The only outstanding issues with this are:

* Had to comment out several unit tests due to threading._Event and
multiprocessing Event API differences.
* One test failing consistently on OS/X due to a change that came with
rolling into trunk, I'm working on this
* Bug http://bugs.python.org/issue1683 needs to have the patch
reviewed and applied to trunk.
* Full doc-content review

The patch is both on the codereview system and the bug tracker:

http://bugs.python.org/issue3050
http://codereview.appspot.com/2061

-jesse

From barry at python.org  Wed Jun 11 01:23:17 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 10 Jun 2008 19:23:17 -0400
Subject: [Python-Dev] Nab those release blockers!
In-Reply-To: <788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com>
References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com>
	<788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com>
Message-ID: <D44CB955-EBE2-4E31-9D28-35420D81DFF3@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 9, 2008, at 6:33 AM, Jesse Noller wrote:

> For the multiprocessing module all I have left is the docs in rest  
> format - given I'm a rest newbie, it's taking longer than I thought

It's okay if docs don't make it into the beta.  That's the kind of  
thing that can (but still, must) be fixed up before the final release.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSE8M5XEjvBPtnXfVAQIBcQP/S5BMgNS/sOXajnU7eLfXGM39/PwStL8b
HOv/MZiHtjmRx5Rtro7lDnjaf9u4Ae+V90a5Pt+aAIANSLenzl1GVcJuuQNP+baa
trrTvvBGD3C78au76yOIKRZ0xfvv+rxQXo3mzQ4Yf3GiyOaDdhHMiar6qHlQhesK
1j6rghX/J0Q=
=r31d
-----END PGP SIGNATURE-----

From musiccomposition at gmail.com  Wed Jun 11 01:26:25 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 10 Jun 2008 18:26:25 -0500
Subject: [Python-Dev] Nab those release blockers!
In-Reply-To: <D44CB955-EBE2-4E31-9D28-35420D81DFF3@python.org>
References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com>
	<788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com>
	<D44CB955-EBE2-4E31-9D28-35420D81DFF3@python.org>
Message-ID: <1afaf6160806101626y11c5bb4btfbf053f6ee66fdfe@mail.gmail.com>

On Tue, Jun 10, 2008 at 6:23 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jun 9, 2008, at 6:33 AM, Jesse Noller wrote:
>
>> For the multiprocessing module all I have left is the docs in rest format
>> - given I'm a rest newbie, it's taking longer than I thought
>
> It's okay if docs don't make it into the beta.  That's the kind of thing
> that can (but still, must) be fixed up before the final release.

Luckily, it's done now, and I think I will be able to check in
multiprocessing tonight or tomorrow.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From jnoller at gmail.com  Wed Jun 11 01:44:39 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 10 Jun 2008 19:44:39 -0400
Subject: [Python-Dev] Nab those release blockers!
In-Reply-To: <1afaf6160806101626y11c5bb4btfbf053f6ee66fdfe@mail.gmail.com>
References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com>
	<788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com>
	<D44CB955-EBE2-4E31-9D28-35420D81DFF3@python.org>
	<1afaf6160806101626y11c5bb4btfbf053f6ee66fdfe@mail.gmail.com>
Message-ID: <CDBCCCB7-7AD4-41D4-86A9-549DA386775C@gmail.com>

I'll try to roll an updated patch with the review info tonight for you

On Jun 10, 2008, at 7:26 PM, "Benjamin Peterson" <musiccomposition at gmail.com 
 > wrote:

> On Tue, Jun 10, 2008 at 6:23 PM, Barry Warsaw <barry at python.org>  
> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On Jun 9, 2008, at 6:33 AM, Jesse Noller wrote:
>>
>>> For the multiprocessing module all I have left is the docs in rest  
>>> format
>>> - given I'm a rest newbie, it's taking longer than I thought
>>
>> It's okay if docs don't make it into the beta.  That's the kind of  
>> thing
>> that can (but still, must) be fixed up before the final release.
>
> Luckily, it's done now, and I think I will be able to check in
> multiprocessing tonight or tomorrow.
>
>
> -- 
> Cheers,
> Benjamin Peterson
> "There's no place like 127.0.0.1."

From guido at python.org  Wed Jun 11 02:07:28 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jun 2008 17:07:28 -0700
Subject: [Python-Dev] Have been sick, am behind on mail,
	let me know if there's anything urgent for me
Message-ID: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>

All,

I've been sick since Saturday and still don't feel up to much. I've
collected a severe email backlog going back to June 6th. If there's
anything someone really needs me to look at ASAP (e.g. a BDFL decision
affecting the impending beta release) please send me an email (a
followup to this thread is fine) and I'll try to look at it soon.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fijall at gmail.com  Wed Jun 11 03:10:17 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 11 Jun 2008 03:10:17 +0200
Subject: [Python-Dev] bug or a feature?
Message-ID: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>

What do you think about this code:

class A:
   locals()[42] = 98

Seems people rely on it working. Do we consider it part of python
language? (Note that you cannot do the same with getattr/setattr which
checks if argument is a string)

Cheers,
fijal

From musiccomposition at gmail.com  Wed Jun 11 03:14:39 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 10 Jun 2008 20:14:39 -0500
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
Message-ID: <1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com>

On Tue, Jun 10, 2008 at 8:10 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> What do you think about this code:
>
> class A:
>   locals()[42] = 98
>
> Seems people rely on it working. Do we consider it part of python
> language? (Note that you cannot do the same with getattr/setattr which
> checks if argument is a string)

Seems like a bug to me, but I don't think there is much we can do
about it short of making locals a custom dict which rejects none
string keys.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From steve at holdenweb.com  Wed Jun 11 03:30:56 2008
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 10 Jun 2008 21:30:56 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com>
Message-ID: <484F2AD0.5010106@holdenweb.com>

Benjamin Peterson wrote:
> On Tue, Jun 10, 2008 at 8:10 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> What do you think about this code:
>>
>> class A:
>>   locals()[42] = 98
>>
>> Seems people rely on it working. Do we consider it part of python
>> language? (Note that you cannot do the same with getattr/setattr which
>> checks if argument is a string)
> 
> Seems like a bug to me, but I don't think there is much we can do
> about it short of making locals a custom dict which rejects none
> string keys.
> 
> 
Given that the documentation has for a long time said:

"""locals( )

Update and return a dictionary representing the current local symbol 
table. Warning: The contents of this dictionary should not be modified; 
changes may not affect the values of local variables used by the 
interpreter. """

it would be hard for the people who have relied on this behavior to 
complain if it stopped working. Basically retaining this behavior is 
pandering to people who have ignored the language specification!

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From scott+python-dev at scottdial.com  Wed Jun 11 03:36:27 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 10 Jun 2008 21:36:27 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
Message-ID: <484F2C1B.2090404@scottdial.com>

Maciej Fijalkowski wrote:
> What do you think about this code:
> 
> class A:
>    locals()[42] = 98
> 
> Seems people rely on it working.

I apologize for my ignorance, but who? Could you please cite something 
reputable that relies on this detail?

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From fijall at gmail.com  Wed Jun 11 03:37:45 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 11 Jun 2008 03:37:45 +0200
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <484F2C1B.2090404@scottdial.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
Message-ID: <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>

On Wed, Jun 11, 2008 at 3:36 AM, Scott Dial
<scott+python-dev at scottdial.com> wrote:
> Maciej Fijalkowski wrote:
>>
>> What do you think about this code:
>>
>> class A:
>>   locals()[42] = 98
>>
>> Seems people rely on it working.
>
> I apologize for my ignorance, but who? Could you please cite something
> reputable that relies on this detail?
>

It's in tests of sqlalchemy. My question is among the lines "should I
bug sqlalchemy guys to remove this, or should I change pypy to accept
this".

From musiccomposition at gmail.com  Wed Jun 11 03:40:16 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 10 Jun 2008 20:40:16 -0500
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
Message-ID: <1afaf6160806101840j56928e7fiad88dd73f26668c4@mail.gmail.com>

On Tue, Jun 10, 2008 at 8:37 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Wed, Jun 11, 2008 at 3:36 AM, Scott Dial
>>
>> I apologize for my ignorance, but who? Could you please cite something
>> reputable that relies on this detail?
>>
>
> It's in tests of sqlalchemy. My question is among the lines "should I
> bug sqlalchemy guys to remove this, or should I change pypy to accept
> this".

If you have that power, I recommend you tell them to get rid of it. :)


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From steve at holdenweb.com  Wed Jun 11 03:30:56 2008
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 10 Jun 2008 21:30:56 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com>
Message-ID: <484F2AD0.5010106@holdenweb.com>

Benjamin Peterson wrote:
> On Tue, Jun 10, 2008 at 8:10 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> What do you think about this code:
>>
>> class A:
>>   locals()[42] = 98
>>
>> Seems people rely on it working. Do we consider it part of python
>> language? (Note that you cannot do the same with getattr/setattr which
>> checks if argument is a string)
> 
> Seems like a bug to me, but I don't think there is much we can do
> about it short of making locals a custom dict which rejects none
> string keys.
> 
> 
Given that the documentation has for a long time said:

"""locals( )

Update and return a dictionary representing the current local symbol 
table. Warning: The contents of this dictionary should not be modified; 
changes may not affect the values of local variables used by the 
interpreter. """

it would be hard for the people who have relied on this behavior to 
complain if it stopped working. Basically retaining this behavior is 
pandering to people who have ignored the language specification!

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/



From python at rcn.com  Wed Jun 11 04:18:28 2008
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Jun 2008 19:18:28 -0700
Subject: [Python-Dev] bug or a feature?
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com>
Message-ID: <E969B4CC03EE4B11AD8FD873CB35B2E1@RaymondLaptop1>

>> What do you think about this code:
>>
>> class A:
>>   locals()[42] = 98
>>
>> Seems people rely on it working. Do we consider it part of python
>> language? (Note that you cannot do the same with getattr/setattr which
>> checks if argument is a string)
> 
> Seems like a bug to me, but I don't think there is much we can do
> about it short of making locals a custom dict which rejects none
> string keys.

I see no reason to invent a new custom dict to prevent people from
doing something they find to be useful.  If you can't segfault with it,
who cares.

Also, adding another custom type just makes it more difficult to remember
what is returned by locals() or globals().


Raymond



From musiccomposition at gmail.com  Wed Jun 11 04:20:32 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 10 Jun 2008 21:20:32 -0500
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <E969B4CC03EE4B11AD8FD873CB35B2E1@RaymondLaptop1>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com>
	<E969B4CC03EE4B11AD8FD873CB35B2E1@RaymondLaptop1>
Message-ID: <1afaf6160806101920l21ede0d4rdf785088e7076917@mail.gmail.com>

On Tue, Jun 10, 2008 at 9:18 PM, Raymond Hettinger <python at rcn.com> wrote:
> I see no reason to invent a new custom dict to prevent people from
> doing something they find to be useful.  If you can't segfault with it,
> who cares.

Don't worry; I'm not suggesting it.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From guido at python.org  Wed Jun 11 05:12:48 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jun 2008 20:12:48 -0700
Subject: [Python-Dev] pep8ity __future__
In-Reply-To: <loom.20080607T210748-495@post.gmane.org>
References: <loom.20080607T210748-495@post.gmane.org>
Message-ID: <ca471dc20806102012l409a3b61yb1073a7144976b29@mail.gmail.com>

I don't see it the same way; this is a terribly unimportant API, so
let's not mess with it. threading.py is worth fixing (a) because it's
so popular, and (b) because some folks insisted that the new
multiprocessing module have an API that is as similar as possible to
threading. IOW The general moratorium on pep8ifying remains; we made a
specific exception for threading.py for very specific reasons.

--Guido

On Sat, Jun 7, 2008 at 2:10 PM, Armin Ronacher
<armin.ronacher at active-4.com> wrote:
> Hi,
>
> That's just a flaming-sword thread but I want to mention it nonetheless :-)
>
> Basically I propose getting rid of __future__._Feature.getMandatoryRelease()
> in favour of __future__._Feature.mandatory.  That's backwards compatibile
> and much more pythonic.  Getters/Setters are considered unpythonic and the
> getting rid of all that Java-like naming sounds like a good idea to me :)
>
> If threading/processing gets a pep8ified API with 2.6, why not __future__?
>
> Proposed patch: http://paste.pocoo.org/show/64512/
>
>
> Regards,
> Armin
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at krypto.org  Wed Jun 11 05:42:57 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 10 Jun 2008 20:42:57 -0700
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <484CED76.3070703@egenix.com>
References: <48397ECC.9070805@cheimes.de> <483FBCB4.5020007@egenix.com>
	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>
	<4843E884.1060705@egenix.com>
	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>
	<ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>
	<52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com>
	<48490121.5030701@egenix.com>
	<52dc1c820806082220n2f0832e1l7af4bc2fecf31987@mail.gmail.com>
	<484CED76.3070703@egenix.com>
Message-ID: <52dc1c820806102042x19c935cev61739772fddb2173@mail.gmail.com>

On Mon, Jun 9, 2008 at 1:44 AM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 2008-06-09 07:20, Gregory P. Smith wrote:
>
>> On Fri, Jun 6, 2008 at 2:19 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>
>>  On 2008-06-03 01:29, Gregory P. Smith wrote:
>>>
>>>  On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum <guido at python.org>
>>>> wrote:
>>>>
>>>>  I will freely admit that I haven't followed this thread in any detail,
>>>>
>>>>> but if it were up to me, I'd have the 2.6 internal code use PyString
>>>>>
>>>>>  ...
>>>>
>>>> Should we read this as a BDFL pronouncement and make it so?
>>>>
>>>> All that would mean change wise is that trunk r63675 as well as possibly
>>>> r63672 and r63677 would need to be rolled back and this whole discussion
>>>> over if such a big change should have happened would turn into a moot
>>>> point.
>>>>
>>>>  I would certainly welcome reverting the change.
>>>
>>> All that's needed to support PyBytes API in 2.x is a set of #defines
>>> that map the new APIs to the PyString names. That's a clean and
>>> easily understandable solution.
>>>
>>>
>> Okay, I've reverted r63675 in trunk revision r64048.  That leaves all of
>> the
>> python modules and internals using PyString_ api names instead of PyBytes_
>> api names as they were before.  PyBytes_ #define's exist for the
>> appropriate
>> PyString methods incase anyone wants to use those.
>>
>
> Thanks.
>
>  Programmers interested in the code
>>
>>> for a PyString API can then still look up the code in stringobject.c,
>>> e.g. to find out how a certain special case is handled or to check
>>> the ref counting - just like they did for years.
>>>
>>
>>
>> The files still exist with the new names.  bytesobject.c instead of
>> stringobject.c.  Those renames were done in the other CLs i mentioned
>> which
>> have not yet been reverted.  The current state seems a bit odd because
>> they
>> depend on the #defines to cause method definitions to be the PyString_
>> names
>> instead of the PyBytes_ names.
>>
>
> Please restore the original state, ie. PyString APIs live in
> stringobject.h and stringobject.c. bytesobject.h should then have
> the #defines for PyBytes APIs, pointing them to the PyString
> names (basically what's currently in stringobject.h).
>

all done as of 64105
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080610/4cba5859/attachment.htm>

From greg.ewing at canterbury.ac.nz  Wed Jun 11 06:02:13 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 11 Jun 2008 16:02:13 +1200
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
Message-ID: <484F4E45.9060004@canterbury.ac.nz>

Maciej Fijalkowski wrote:
> What do you think about this code:
> 
> class A:
>    locals()[42] = 98
> 
> Seems people rely on it working. Do we consider it part of python
> language?

Modifying the dict returned by locals() is documented
as NOT being guaranteed to work, isn't it?

-- 
Greg

From stefan_ml at behnel.de  Wed Jun 11 07:15:30 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 11 Jun 2008 07:15:30 +0200
Subject: [Python-Dev] Have been sick, am behind on mail,
 let me know if there's anything urgent for me
In-Reply-To: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
Message-ID: <g2nn1i$413$1@ger.gmane.org>

Hi Guido,

glad to hear you're recovering.

Guido van Rossum wrote:
> anything someone really needs me to look at ASAP (e.g. a BDFL decision
> affecting the impending beta release) please send me an email

This seems to require a BDFL decision:

http://bugs.python.org/issue2997

Executive Summary: PyNumberMethods has been changed on py3k back in 2006 with
the nb_divide removal, so it's now incompatible with Py2. But there are three
more unused struct members *behind* that field that can be removed for beta1,
but have not been removed yet. Should they be removed for cleanliness (patch
in the issue) or should nb_divide and nb_inplace_divide instead be restored
(rev 43285) to restore backwards compatibility?

Thanks,
Stefan


From greg at krypto.org  Wed Jun 11 08:31:23 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 10 Jun 2008 23:31:23 -0700
Subject: [Python-Dev] segfault in struct module
In-Reply-To: <20080610160403.4714.104530664.divmod.quotient.7701@ohm>
References: <20080610160403.4714.104530664.divmod.quotient.7701@ohm>
Message-ID: <52dc1c820806102331t7a07e9d4v76f9a3734e953d32@mail.gmail.com>

On Tue, Jun 10, 2008 at 9:04 AM, Jean-Paul Calderone <exarkun at divmod.com>
wrote:

>
>   exarkun at charm:~$ ~/Projects/python/trunk/python
>   Python 2.6a3+ (trunk:63964, Jun  5 2008, 16:49:12)
>   [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
>   Type "help", "copyright", "credits" or "license" for more information.
>   >>> import struct
>   >>> struct.pack("357913941c", 'a')
>   Segmentation fault
>   exarkun at charm:~$
>
> The unit test for exactly this case was deleted in r60892.  I would like to
> suggest that just deleting unit tests isn't a very good idea.
>
> Jean-Paul
>

That deletion was apparently in the release25-maint branch where this bug
has been fixed in order to avoid the test case from allocating 8gb on 64bit
hosts where it wouldn't need to raise a MemoryError (
http://bugs.python.org/issue2137).  However it looks like it was never
committed to trunk.  (release25-maint r60793 has a lot of fixes)

I'm looking into merging the fixes into trunk.

Martin, since you committed 60793 in Feb, any others like this that need
merging from release25-maint to trunk off the top of your head?

-Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080610/cd798152/attachment.htm>

From g.brandl at gmx.net  Wed Jun 11 08:32:14 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 11 Jun 2008 08:32:14 +0200
Subject: [Python-Dev] multiprocessing problems
Message-ID: <g2nrhg$f8i$1@ger.gmane.org>

Currently, multiprocessing cannot be imported:

 >>> import multiprocessing
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/home/gbr/devel/python/Lib/multiprocessing/__init__.py", line 63, in 
<module>
import _multiprocessing
AttributeError: 'module' object has no attribute 'BufferTooShort'

The test suite passes (at least for some buildbots) because it imports
_multiprocessing first, which then in its init function imports multiprocessing
to get the BufferTooShort exception.

Since BufferTooShort and other exceptions inheriting from ProcessError are
simple derived exceptions, why aren't they created in the C module in the
first place?

Georg


From martin at v.loewis.de  Wed Jun 11 10:19:06 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Jun 2008 10:19:06 +0200
Subject: [Python-Dev] segfault in struct module
In-Reply-To: <52dc1c820806102331t7a07e9d4v76f9a3734e953d32@mail.gmail.com>
References: <20080610160403.4714.104530664.divmod.quotient.7701@ohm>
	<52dc1c820806102331t7a07e9d4v76f9a3734e953d32@mail.gmail.com>
Message-ID: <484F8A7A.7060407@v.loewis.de>

> Martin, since you committed 60793 in Feb, any others like this that need
> merging from release25-maint to trunk off the top of your head?

That's the entire chunk of "Google bug fixes", and yes, all of it needs
to be ported to 2.6 still.

I'll look into it, unless you volunteer :-) (it doesn't need to be done
before the beta, as it's bug fixes only).

Regards,
Martin

From mal at egenix.com  Wed Jun 11 10:52:06 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 11 Jun 2008 10:52:06 +0200
Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0
In-Reply-To: <52dc1c820806102042x19c935cev61739772fddb2173@mail.gmail.com>
References: <48397ECC.9070805@cheimes.de>
	<483FBCB4.5020007@egenix.com>	<52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com>	<4843E884.1060705@egenix.com>	<52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com>	<ca471dc20806021609v258d7e8bk66b259694dd469b5@mail.gmail.com>	<52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com>	<48490121.5030701@egenix.com>	<52dc1c820806082220n2f0832e1l7af4bc2fecf31987@mail.gmail.com>	<484CED76.3070703@egenix.com>
	<52dc1c820806102042x19c935cev61739772fddb2173@mail.gmail.com>
Message-ID: <484F9236.4020101@egenix.com>

On 2008-06-11 05:42, Gregory P. Smith wrote:
> On Mon, Jun 9, 2008 at 1:44 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> 
>> On 2008-06-09 07:20, Gregory P. Smith wrote:
>>
>>> On Fri, Jun 6, 2008 at 2:19 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>>
>>>  On 2008-06-03 01:29, Gregory P. Smith wrote:
>>>>  On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum <guido at python.org>
>>>>> wrote:
>>>>>
>>>>>  I will freely admit that I haven't followed this thread in any detail,
>>>>>
>>>>>> but if it were up to me, I'd have the 2.6 internal code use PyString
>>>>>>
>>>>>>  ...
>>>>> Should we read this as a BDFL pronouncement and make it so?
>>>>>
>>>>> All that would mean change wise is that trunk r63675 as well as possibly
>>>>> r63672 and r63677 would need to be rolled back and this whole discussion
>>>>> over if such a big change should have happened would turn into a moot
>>>>> point.
>>>>>
>>>>>  I would certainly welcome reverting the change.
>>>> All that's needed to support PyBytes API in 2.x is a set of #defines
>>>> that map the new APIs to the PyString names. That's a clean and
>>>> easily understandable solution.
>>>>
>>>>
>>> Okay, I've reverted r63675 in trunk revision r64048.  That leaves all of
>>> the
>>> python modules and internals using PyString_ api names instead of PyBytes_
>>> api names as they were before.  PyBytes_ #define's exist for the
>>> appropriate
>>> PyString methods incase anyone wants to use those.
>>>
>> Thanks.
>>
>>  Programmers interested in the code
>>>> for a PyString API can then still look up the code in stringobject.c,
>>>> e.g. to find out how a certain special case is handled or to check
>>>> the ref counting - just like they did for years.
>>>>
>>>
>>> The files still exist with the new names.  bytesobject.c instead of
>>> stringobject.c.  Those renames were done in the other CLs i mentioned
>>> which
>>> have not yet been reverted.  The current state seems a bit odd because
>>> they
>>> depend on the #defines to cause method definitions to be the PyString_
>>> names
>>> instead of the PyBytes_ names.
>>>
>> Please restore the original state, ie. PyString APIs live in
>> stringobject.h and stringobject.c. bytesobject.h should then have
>> the #defines for PyBytes APIs, pointing them to the PyString
>> names (basically what's currently in stringobject.h).
>>
> 
> all done as of 64105

Thank you !

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 11 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            25 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From ncoghlan at gmail.com  Wed Jun 11 11:21:12 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Jun 2008 19:21:12 +1000
Subject: [Python-Dev] Assignment to None
In-Reply-To: <484EA327.2050704@vector-seven.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk>	<484D9D59.903@v.loewis.de>
	<484EA327.2050704@vector-seven.com>
Message-ID: <484F9908.9020009@gmail.com>

Thomas Lee wrote:
> Martin v. L?wis wrote:
>> In Python 3, None, True, and False are keywords, so clearly, the
>> intended semantics is also the implemented one (and the language
>> description for 2.x needs to be updated/clarified).
>>
>>   
> Interestingly enough, the semantics of True, False and None are 
> different from one another in 2.6:
> 
> True = "blah" and False = 6 are perfectly legal in Python <=2.6.

True and False didn't get the same treatment as None because we didn't 
want to break the follow compatibility workaround for Python version 
prior to 2.2.2:

try:
   True
except NameError:
   True, False = 1, 0

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Wed Jun 11 11:34:35 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Jun 2008 19:34:35 +1000
Subject: [Python-Dev] Exception.__unicode__ and tp_unicode
In-Reply-To: <fb73205e0806100931y68160721w2a134e1e8204993b@mail.gmail.com>
References: <fb73205e0806100931y68160721w2a134e1e8204993b@mail.gmail.com>
Message-ID: <484F9C2B.1070805@gmail.com>

Simon Cross wrote:
> Originally Python exceptions had no __unicode__ method. In Python 2.5
> __unicode__ was added. This led to "unicode(Exception)" failing and so
> the addition of __unicode__ was reverted [1].
> 
> This leaves Python 2.6 in a position where calls to
> "unicode(Exception(u'\xe1'))" fail as they are equivalent to
> "uncode(str(Exception(u'\xe1'))" which cannot convert the non-ASCII
> character to ASCII (or other default encoding) [2].
> 
>>From here there are 3 options:
> 
>  1) Leave things as they are.
>  2) Add back __unicode__ and have "unicode(Exception)" fail.
>  3) Add a tp_unicode slot to Python objects and have everything work
> (at the cost of adding the slot).

4) Fix PyObject_Unicode to not retrieve __unicode__ from new-style 
instances, and instead only look for the method on their types (similar 
to the way PyObject_Format looks up the __format__ method).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Wed Jun 11 11:40:57 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Jun 2008 19:40:57 +1000
Subject: [Python-Dev] Have been sick, am behind on mail,
 let me know if there's anything urgent for me
In-Reply-To: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
Message-ID: <484F9DA9.3030606@gmail.com>

Guido van Rossum wrote:
> All,
> 
> I've been sick since Saturday and still don't feel up to much. I've
> collected a severe email backlog going back to June 6th. If there's
> anything someone really needs me to look at ASAP (e.g. a BDFL decision
> affecting the impending beta release) please send me an email (a
> followup to this thread is fine) and I'll try to look at it soon.
> 

As it involves adding a new standard library module, my proposal [1] to 
add a typetools.ProxyMixin class to 2.6/3.0 to make it practical to 
write type-agnostic interface adapter classes as new-style classes needs 
a pronouncement from yourself or Barry as to whether or not it can go in.

I think it's an important tool to add since 3.0 no longer has classic 
classes around to handle the task, and I'm suggesting a new module for 
it because none of the existing standard library modules seemed like a 
good fit for the functionality.

Cheers,
Nick.

[1] http://bugs.python.org/issue643841

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Wed Jun 11 11:50:59 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Jun 2008 19:50:59 +1000
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <484F4E45.9060004@canterbury.ac.nz>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F4E45.9060004@canterbury.ac.nz>
Message-ID: <484FA003.5020703@gmail.com>

Greg Ewing wrote:
> Maciej Fijalkowski wrote:
>> What do you think about this code:
>>
>> class A:
>>    locals()[42] = 98
>>
>> Seems people rely on it working. Do we consider it part of python
>> language?
> 
> Modifying the dict returned by locals() is documented
> as NOT being guaranteed to work, isn't it?
> 
Yep - it just happens to work for class and module level namespaces in 
CPython.

Implementations are also permitted to restrict namespace dictionaries to 
only accept string keys (I believe Jython does this for performance 
reasons - CPython just optimised the hell out of normal dictionaries 
that happen to only contain string keys instead).

So I don't see any reason for Maciej to mess with PyPy to support code 
which deliberately makes use of formally undefined behaviour.

Cheers,
Nick.



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Wed Jun 11 11:54:43 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Jun 2008 19:54:43 +1000
Subject: [Python-Dev] pep8ity __future__
In-Reply-To: <ca471dc20806102012l409a3b61yb1073a7144976b29@mail.gmail.com>
References: <loom.20080607T210748-495@post.gmane.org>
	<ca471dc20806102012l409a3b61yb1073a7144976b29@mail.gmail.com>
Message-ID: <484FA0E3.709@gmail.com>

Guido van Rossum wrote:
> I don't see it the same way; this is a terribly unimportant API, so
> let's not mess with it. threading.py is worth fixing (a) because it's
> so popular, and (b) because some folks insisted that the new
> multiprocessing module have an API that is as similar as possible to
> threading. IOW The general moratorium on pep8ifying remains; we made a
> specific exception for threading.py for very specific reasons.

We're also only *adding* the PEP 8 compliant API to threading, and 
leaving the old API in-place indefinitely in the 2.x series.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From barry at python.org  Wed Jun 11 13:35:25 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jun 2008 07:35:25 -0400
Subject: [Python-Dev] Betas today - I hope
Message-ID: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

So I had planned to do a bunch of work last night looking at the  
release blocker issues, but nature intervened.  A bunch of severe  
thunderstorms knock out my 'net access until this morning.

I'll try to find some time during the day to look at the RB issues.   
Hopefully we can get Guido to look at them too and Pronounce on some  
of them.  Guido please start with:

http://bugs.python.org/issue643841

My plan is to begin building the betas tonight, at around 9 or 10pm  
EDT (0100 to 0200 UTC Thursday).  If a showstopper comes up before  
then, I'll email the list.  If you think we really aren't ready for  
beta, then I would still like to get a release out today.  In that  
case, we'll call it alpha and delay the betas.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSE+4fXEjvBPtnXfVAQJ4AgP+MD0o3Mw5borZRnQffomXfsAFOMLD0PDr
fS5PwhxkVL/qJU7ZM7v9l8j5walI7Boj1aH7w8UQdV0KpEc0ruqWhsPFCOny3L0W
WgHMtyD0Wc+GAf6ckpOxxWI4Blg52MVzkhplKHaRJ5c4lNriBKt1o9xeupElWtHq
awcHgApvTms=
=iNK1
-----END PGP SIGNATURE-----

From hodgestar+pythondev at gmail.com  Wed Jun 11 13:56:31 2008
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Wed, 11 Jun 2008 13:56:31 +0200
Subject: [Python-Dev] Exception.__unicode__ and tp_unicode
In-Reply-To: <484F9C2B.1070805@gmail.com>
References: <fb73205e0806100931y68160721w2a134e1e8204993b@mail.gmail.com>
	<484F9C2B.1070805@gmail.com>
Message-ID: <fb73205e0806110456n39617ae9m441888c51ad2ccc6@mail.gmail.com>

On Wed, Jun 11, 2008 at 11:34 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 4) Fix PyObject_Unicode to not retrieve __unicode__ from new-style
> instances, and instead only look for the method on their types (similar to
> the way PyObject_Format looks up the __format__ method).

Thanks for the suggestion Nick. I've added a patch which implements it
to the bug.

Schiavo
Simon

From barry at python.org  Wed Jun 11 14:04:43 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jun 2008 08:04:43 -0400
Subject: [Python-Dev] Nab those release blockers!
In-Reply-To: <484D13F9.30105@gmail.com>
References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com>
	<484D13F9.30105@gmail.com>
Message-ID: <0E8A956A-07AA-4B3C-A6AE-5E0E04912B5A@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 9, 2008, at 7:28 AM, Nick Coghlan wrote:

> Benjamin Peterson wrote:
>> As we approach the planned first betas for 2.6/3.0, my heart sinks
>> when I see the imposing list of 16 release blockers. [1] Luckily,  
>> most
>> of the issues have patches that just need *your* review. Others,
>> namely the Py3k exception issues, may require a little more
>> discussion, but I'm sure we can get it done.
>
> For issue 643841 (providing a ProxyMixin class in a new typetools  
> module), aside from the minor tweak mentioned in the last couple of  
> comments, I'm basically looking for a yea or nay from Barry or  
> Guido. I think it's the best answer we're going to find to the  
> problem of how to write type-agnostic delegation classes in a Python  
> world without classic classes.

I've updated the issue with my own opinion.  Let's see what Guido  
thinks.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSE+/W3EjvBPtnXfVAQIP0QP7BNuejYBEZcp9n/iF8KicfKIJ9L39pNDA
QWSW72ApVnwaLEebl1QPEfF5LBzjVSYqWu5RcuxYnFro61FIi6QSX91LN+fq+o0a
YuPTa3P9VAIbrfpZQVT9MsLNTqcxtsrA6YtGyv5K48oPrkKCm/9/8+7EFOXeRmXQ
gXBv7K+3ZT4=
=qsrW
-----END PGP SIGNATURE-----

From barry at python.org  Wed Jun 11 14:06:55 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jun 2008 08:06:55 -0400
Subject: [Python-Dev] Have been sick, am behind on mail,
	let me know if there's anything urgent for me
In-Reply-To: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
Message-ID: <7F90796A-E7D2-4C32-A03A-794EFCE94C14@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 10, 2008, at 8:07 PM, Guido van Rossum wrote:

> I've been sick since Saturday and still don't feel up to much. I've
> collected a severe email backlog going back to June 6th. If there's
> anything someone really needs me to look at ASAP (e.g. a BDFL decision
> affecting the impending beta release) please send me an email (a
> followup to this thread is fine) and I'll try to look at it soon.

Hi Guido, sorry you've been sick!  If you're up to it, please  
pronounce on this issue:

http://bugs.python.org/issue643841

It's a long bug thread, but needs a decision.

Thanks,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSE+/4nEjvBPtnXfVAQJDzQP9FtWLT3+MIBoeRLRqFtXEZxphi+IDU7wp
Jtw82cFgKz/wPPxyyGEtnLSmt6duj3MTTftuVsTKS1plqsd6puC2S1VbEOvS9otp
xQ0E1mn/xb4B/BvH9n3xkS9FTiYlePkmU61IdAdmSAQ3gtFlwDPB8o5k30RuvhLl
S+Ai+mBqPq4=
=TUyr
-----END PGP SIGNATURE-----

From musiccomposition at gmail.com  Wed Jun 11 14:15:45 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 11 Jun 2008 07:15:45 -0500
Subject: [Python-Dev] Have been sick, am behind on mail,
	let me know if there's anything urgent for me
In-Reply-To: <1afaf6160806102017u21d508d3u4e1bfbf3ff155ca2@mail.gmail.com>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
	<1afaf6160806101714j3fa18965h44cd2f3092ba3f3c@mail.gmail.com>
	<ca471dc20806101844t187a383bl4413168a792a676a@mail.gmail.com>
	<1afaf6160806101852v1a53f4f0yebd32f68150fe726@mail.gmail.com>
	<ca471dc20806101900q3152b0eav46475f2ff61b5014@mail.gmail.com>
	<1afaf6160806101919p774e1dbfm43762e3d09bceff7@mail.gmail.com>
	<ca471dc20806102008s7eee4bc0t869338f870495b43@mail.gmail.com>
	<1afaf6160806102017u21d508d3u4e1bfbf3ff155ca2@mail.gmail.com>
Message-ID: <1afaf6160806110515w4db61294tf49ceda4c54401f6@mail.gmail.com>

[just ccing python-dev]

On Tue, Jun 10, 2008 at 10:17 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Tue, Jun 10, 2008 at 10:08 PM, Guido van Rossum <guido at python.org> wrote:
>> On Tue, Jun 10, 2008 at 7:19 PM, Benjamin Peterson
>> <musiccomposition at gmail.com> wrote:
>>>
>>> I think that we should accept Antoine's patch and begin the twilight
>>> years of sys.exc_info in favor of passing the exception instances
>>> around. This makes for more explicit and less magical code. I don't
>>> think there's any sys.exc_info case that can't be rewritten without
>>> it.
>>
>> OK, assuming it works and doesn't break any unittests (or fixes the
>> ones it expects to break), and has unittests for the new behavior, I'd
>> say go for it.
>
> Excellent!
>
>>
>>> I think the implicit chaining is assuming a little too much
>>> about the needs of the program.
>>
>> That's why it's on a separate attribute. It can be handy to use when
>> you need to debug an exception that happens in an exception handle.
>> Sometimes it just helps to know why the handler was being invoked in
>> the first place, other times you really want to know the original
>> exception because that's the problem you're trying to track down. But
>> I believe this is where Collin ran into a brick wall. I still think it
>> could be implemented post beta 1.
>
> Ok. I will make an issue for it in the morning.
>
> Thanks for your pronouncements.
>
>
>
> --
> Cheers,
> Benjamin Peterson
> "There's no place like 127.0.0.1."
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From musiccomposition at gmail.com  Wed Jun 11 14:23:20 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 11 Jun 2008 07:23:20 -0500
Subject: [Python-Dev] multiprocessing problems
In-Reply-To: <g2nrhg$f8i$1@ger.gmane.org>
References: <g2nrhg$f8i$1@ger.gmane.org>
Message-ID: <1afaf6160806110523r707ffbeejf00ecef9aff3e615@mail.gmail.com>

On Wed, Jun 11, 2008 at 1:32 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Currently, multiprocessing cannot be imported:
>
>>>> import multiprocessing
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>  File "/home/gbr/devel/python/Lib/multiprocessing/__init__.py", line 63, in
> <module>
> import _multiprocessing
> AttributeError: 'module' object has no attribute 'BufferTooShort'
>
> The test suite passes (at least for some buildbots) because it imports
> _multiprocessing first, which then in its init function imports
> multiprocessing
> to get the BufferTooShort exception.

I fixed it by moving the _multiprocessing import below the exception
definitions.

>
> Since BufferTooShort and other exceptions inheriting from ProcessError are
> simple derived exceptions, why aren't they created in the C module in the
> first place?

They should eventually go to C, but I'll worry about it after the betas. :)


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From jnoller at gmail.com  Wed Jun 11 14:28:03 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 11 Jun 2008 08:28:03 -0400
Subject: [Python-Dev] multiprocessing problems
In-Reply-To: <1afaf6160806110523r707ffbeejf00ecef9aff3e615@mail.gmail.com>
References: <g2nrhg$f8i$1@ger.gmane.org>
	<1afaf6160806110523r707ffbeejf00ecef9aff3e615@mail.gmail.com>
Message-ID: <4222a8490806110528x19e50295h2a9d6f92b7d5a337@mail.gmail.com>

On Wed, Jun 11, 2008 at 8:23 AM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Wed, Jun 11, 2008 at 1:32 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Currently, multiprocessing cannot be imported:
>>
>>>>> import multiprocessing
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>>  File "/home/gbr/devel/python/Lib/multiprocessing/__init__.py", line 63, in
>> <module>
>> import _multiprocessing
>> AttributeError: 'module' object has no attribute 'BufferTooShort'
>>
>> The test suite passes (at least for some buildbots) because it imports
>> _multiprocessing first, which then in its init function imports
>> multiprocessing
>> to get the BufferTooShort exception.
>
> I fixed it by moving the _multiprocessing import below the exception
> definitions.
>
>>
>> Since BufferTooShort and other exceptions inheriting from ProcessError are
>> simple derived exceptions, why aren't they created in the C module in the
>> first place?
>
> They should eventually go to C, but I'll worry about it after the betas. :)
>

Thanks Ben, you beat me to it. I'll add it to the tracker.

From ncoghlan at gmail.com  Wed Jun 11 14:59:09 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Jun 2008 22:59:09 +1000
Subject: [Python-Dev] Have been sick, am behind on mail,
 let me know if there's anything urgent for me
In-Reply-To: <7F90796A-E7D2-4C32-A03A-794EFCE94C14@python.org>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
	<7F90796A-E7D2-4C32-A03A-794EFCE94C14@python.org>
Message-ID: <484FCC1D.1010605@gmail.com>

Barry Warsaw wrote:
> On Jun 10, 2008, at 8:07 PM, Guido van Rossum wrote:
> 
>> I've been sick since Saturday and still don't feel up to much. I've
>> collected a severe email backlog going back to June 6th. If there's
>> anything someone really needs me to look at ASAP (e.g. a BDFL decision
>> affecting the impending beta release) please send me an email (a
>> followup to this thread is fine) and I'll try to look at it soon.
> 
> Hi Guido, sorry you've been sick!  If you're up to it, please pronounce 
> on this issue:
> 
> http://bugs.python.org/issue643841
> 
> It's a long bug thread, but needs a decision.

I just added a couple of messages at the end that recap the more recent 
discussions (the early responses to the issue are *really* old...)

Cheers,
Nick.

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mal at egenix.com  Wed Jun 11 16:32:09 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 11 Jun 2008 16:32:09 +0200
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
Message-ID: <484FE1E9.2080904@egenix.com>

On 2008-06-11 13:35, Barry Warsaw wrote:
> So I had planned to do a bunch of work last night looking at the release 
> blocker issues, but nature intervened.  A bunch of severe thunderstorms 
> knock out my 'net access until this morning.
> 
> I'll try to find some time during the day to look at the RB issues.  
> Hopefully we can get Guido to look at them too and Pronounce on some of 
> them.  Guido please start with:
> 
> http://bugs.python.org/issue643841
> 
> My plan is to begin building the betas tonight, at around 9 or 10pm EDT 
> (0100 to 0200 UTC Thursday).  If a showstopper comes up before then, 
> I'll email the list.  If you think we really aren't ready for beta, then 
> I would still like to get a release out today.  In that case, we'll call 
> it alpha and delay the betas.

There are two things I'd like to get in to 3.0:

  * .transform()/.untransform() methods (this is mostly done, just need
    to add the methods to PyUnicode, PyBytes and PyByteArray)

  * cleanup of the PyUnicode_AsString() and PyUnicode_AsStringAndSize()
    C APIs (these APIs don't fit into the naming scheme used in the
    Unicode API and have a few other issues as well, see issue 2799;
    at the very least they should be made interpreter internal, ie.
    rename them to _PyUnicode_AsString() and _PyUnicode_AsStringAndSize()
    to prevent their use in extensions)

I did not have time in the last few days to work on these and won't
in the next few days either. Next week looks much better.

If it's ok to make the above changes after the release (whatever you
call it ;-), that would be great.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 11 2008)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            25 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From ironfroggy at socialserve.com  Wed Jun 11 16:06:35 2008
From: ironfroggy at socialserve.com (Calvin Spealman)
Date: Wed, 11 Jun 2008 10:06:35 -0400
Subject: [Python-Dev] update_wrapper should preserve staticmethod behavior
Message-ID: <DF383E5E-53EA-4FB5-A539-6E67ECBC4FF4@socialserve.com>

I'd like to make a claim about the following example, that  
update_wrapper should be improved to preserve the behavior of known,  
built-in decorators. In this case, I'm talking about staticmethod.  
The order I list here feels natural, but it obviously doesn't work.  
The only reason it doesn't seems to be that it is trying to decorate  
the descriptor, not the function itself. This is expected, but it  
could certainly be smart enough to detect a descriptor and attempt to  
get the actual callable underneath, could it not? It would not work  
for all cases, of course.

 >>> def d(f):
...     def nf(*a, **kw):
...             print "decorated function called"
...             return f(*a, **kwargs)
...     functools.update_wrapper(nf, f)
...     return nf
...
 >>> class A(object):
...     @d
...     @staticmethod
...     def a(self):
...             print "a"
...
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "<stdin>", line 3, in A
   File "<stdin>", line 5, in d
   File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
python2.5/functools.py", line 33, in update_wrapper
     setattr(wrapper, attr, getattr(wrapped, attr))
AttributeError: 'staticmethod' object has no attribute '__module__'

From walter at livinglogic.de  Wed Jun 11 17:15:56 2008
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed, 11 Jun 2008 17:15:56 +0200
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <484FE1E9.2080904@egenix.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
	<484FE1E9.2080904@egenix.com>
Message-ID: <484FEC2C.6070409@livinglogic.de>

M.-A. Lemburg wrote:
> On 2008-06-11 13:35, Barry Warsaw wrote:
>> So I had planned to do a bunch of work last night looking at the 
>> release blocker issues, but nature intervened.  A bunch of severe 
>> thunderstorms knock out my 'net access until this morning.
>>
>> I'll try to find some time during the day to look at the RB issues.  
>> Hopefully we can get Guido to look at them too and Pronounce on some 
>> of them.  Guido please start with:
>>
>> http://bugs.python.org/issue643841
>>
>> My plan is to begin building the betas tonight, at around 9 or 10pm 
>> EDT (0100 to 0200 UTC Thursday).  If a showstopper comes up before 
>> then, I'll email the list.  If you think we really aren't ready for 
>> beta, then I would still like to get a release out today.  In that 
>> case, we'll call it alpha and delay the betas.
> 
> There are two things I'd like to get in to 3.0:
> 
>  * .transform()/.untransform() methods (this is mostly done, just need
>    to add the methods to PyUnicode, PyBytes and PyByteArray)

What would these methods do? Use the codec machinery without any type 
checks?

I think for transformations we don't need the full codec machinery:

We probably don't need extensible error handling.

There are transformation that are not invertible, so it doesn't make 
sense to have both operations in one object. If the operation *is* 
invertible, two tranformers can be used.

Do we really need a registry that maps function named to functions?

A simple API might look like this:

class TransformInfo:
    # stateless transformer
    def transform(self, input):

    # return stateful incremental transformer
    def incrementaltransformer(self):

    # wrap stream in a transforming stream
    def streamtransformer(self, stream):

incrementaltransformer() would return an object that has one method:

    def transform(self, input, final=False);

> [...]

Servus,
    Walter

From g.brandl at gmx.net  Wed Jun 11 17:48:08 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 11 Jun 2008 17:48:08 +0200
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <484FE1E9.2080904@egenix.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
	<484FE1E9.2080904@egenix.com>
Message-ID: <g2os3q$15t$1@ger.gmane.org>

M.-A. Lemburg schrieb:
> On 2008-06-11 13:35, Barry Warsaw wrote:
>> So I had planned to do a bunch of work last night looking at the release 
>> blocker issues, but nature intervened.  A bunch of severe thunderstorms 
>> knock out my 'net access until this morning.
>> 
>> I'll try to find some time during the day to look at the RB issues.  
>> Hopefully we can get Guido to look at them too and Pronounce on some of 
>> them.  Guido please start with:
>> 
>> http://bugs.python.org/issue643841
>> 
>> My plan is to begin building the betas tonight, at around 9 or 10pm EDT 
>> (0100 to 0200 UTC Thursday).  If a showstopper comes up before then, 
>> I'll email the list.  If you think we really aren't ready for beta, then 
>> I would still like to get a release out today.  In that case, we'll call 
>> it alpha and delay the betas.
> 
> There are two things I'd like to get in to 3.0:

Another thing that should get in is the PEP 3138 implementation, that is,
the ascii() builtin and repr() changes.

Georg


From pje at telecommunity.com  Wed Jun 11 18:15:25 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 11 Jun 2008 12:15:25 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com
 >
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
Message-ID: <20080611161455.CBD703A405F@sparrow.telecommunity.com>

At 03:37 AM 6/11/2008 +0200, Maciej Fijalkowski wrote:
>On Wed, Jun 11, 2008 at 3:36 AM, Scott Dial
><scott+python-dev at scottdial.com> wrote:
> > Maciej Fijalkowski wrote:
> >>
> >> What do you think about this code:
> >>
> >> class A:
> >>   locals()[42] = 98
> >>
> >> Seems people rely on it working.
> >
> > I apologize for my ignorance, but who? Could you please cite something
> > reputable that relies on this detail?
> >
>
>It's in tests of sqlalchemy. My question is among the lines "should I
>bug sqlalchemy guys to remove this, or should I change pypy to accept
>this".

That test is there to ensure that it interoperates with code using 
the AddOns library from the Cheeseshop; SQLAlchemy is not the source 
of the usage.


From greg at krypto.org  Wed Jun 11 18:55:26 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Wed, 11 Jun 2008 09:55:26 -0700
Subject: [Python-Dev] segfault in struct module
In-Reply-To: <484F8A7A.7060407@v.loewis.de>
References: <20080610160403.4714.104530664.divmod.quotient.7701@ohm>
	<52dc1c820806102331t7a07e9d4v76f9a3734e953d32@mail.gmail.com>
	<484F8A7A.7060407@v.loewis.de>
Message-ID: <52dc1c820806110955p200b9153x601045f9ae01dd71@mail.gmail.com>

On Wed, Jun 11, 2008 at 1:19 AM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> > Martin, since you committed 60793 in Feb, any others like this that need
> > merging from release25-maint to trunk off the top of your head?
>
> That's the entire chunk of "Google bug fixes", and yes, all of it needs
> to be ported to 2.6 still.
>
> I'll look into it, unless you volunteer :-) (it doesn't need to be done
> before the beta, as it's bug fixes only).
>
> Regards,
> Martin
>

No problem, I already did it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080611/492ad379/attachment.htm>

From mal at egenix.com  Wed Jun 11 18:57:28 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 11 Jun 2008 18:57:28 +0200
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <484FEC2C.6070409@livinglogic.de>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<484FE1E9.2080904@egenix.com>
	<484FEC2C.6070409@livinglogic.de>
Message-ID: <485003F8.40803@egenix.com>

On 2008-06-11 17:15, Walter D?rwald wrote:
> M.-A. Lemburg wrote:
>> On 2008-06-11 13:35, Barry Warsaw wrote:
>>> So I had planned to do a bunch of work last night looking at the 
>>> release blocker issues, but nature intervened.  A bunch of severe 
>>> thunderstorms knock out my 'net access until this morning.
>>>
>>> I'll try to find some time during the day to look at the RB issues.  
>>> Hopefully we can get Guido to look at them too and Pronounce on some 
>>> of them.  Guido please start with:
>>>
>>> http://bugs.python.org/issue643841
>>>
>>> My plan is to begin building the betas tonight, at around 9 or 10pm 
>>> EDT (0100 to 0200 UTC Thursday).  If a showstopper comes up before 
>>> then, I'll email the list.  If you think we really aren't ready for 
>>> beta, then I would still like to get a release out today.  In that 
>>> case, we'll call it alpha and delay the betas.
>>
>> There are two things I'd like to get in to 3.0:
>>
>>  * .transform()/.untransform() methods (this is mostly done, just need
>>    to add the methods to PyUnicode, PyBytes and PyByteArray)
> 
> What would these methods do? Use the codec machinery without any type 
> checks?

As discussed in another thread some weeks ago:

.transform() and .untransform() use the codecs to apply same-type
conversions. They do apply type checks to make sure that the
codec does indeed return the same type.

E.g. text.transform('xml-escape') or data.transform('base64').

> I think for transformations we don't need the full codec machinery:
 > ...

No need to invent another wheel :-) The codecs already exist for
Py2.x and can be used by the .encode()/.decode() methods in Py2.x
(where no type checks occur).

In Py3.x, .encode()/.decode() only allow conversions of the type
unicode <-> bytes. .transform()/.untransform() add conversions
of the type unicode <-> unicode or bytes <-> bytes.

All other conversions in Py3.x have to go through codecs.encode() and
codecs.decode() which are the generic codec access functions from
the codec registry.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 11 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            25 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From guido at python.org  Wed Jun 11 19:02:55 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Jun 2008 10:02:55 -0700
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
Message-ID: <ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>

On Wed, Jun 11, 2008 at 4:35 AM, Barry Warsaw <barry at python.org> wrote:
> So I had planned to do a bunch of work last night looking at the release
> blocker issues, but nature intervened.  A bunch of severe thunderstorms
> knock out my 'net access until this morning.
>
> I'll try to find some time during the day to look at the RB issues.
>  Hopefully we can get Guido to look at them too and Pronounce on some of
> them.  Guido please start with:
>
> http://bugs.python.org/issue643841

I've added a comment. Let me know if anything I said is unclear.

> My plan is to begin building the betas tonight, at around 9 or 10pm EDT
> (0100 to 0200 UTC Thursday).  If a showstopper comes up before then, I'll
> email the list.  If you think we really aren't ready for beta, then I would
> still like to get a release out today.  In that case, we'll call it alpha
> and delay the betas.

I'd rather call it beta even if certain things are still known to
change in the next release. Beta means we'll have a much easier time
pushing back on random other feature change proposals.

> - -Barry

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jun 11 19:15:04 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Jun 2008 10:15:04 -0700
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <484FE1E9.2080904@egenix.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
	<484FE1E9.2080904@egenix.com>
Message-ID: <ca471dc20806111015g2cb4c023g6d5cdee1ec2ec3b8@mail.gmail.com>

On Wed, Jun 11, 2008 at 7:32 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> There are two things I'd like to get in to 3.0:
>
>  * .transform()/.untransform() methods (this is mostly done, just need
>   to add the methods to PyUnicode, PyBytes and PyByteArray)

I'm +0 on this. It is very minor syntactic sugar that typically saves
you only one import. I expect the argument will nearly always be a
literal, e.g. data.transform('gzip') rather than a variable like
data.transform(compression_method). But it *is* convenient and can
make code more readable, e.g.

  if compressed:
     data = data.untransform('gzip')

Nobody will have to guess what that does. (IMO the confusion about
which direction the transformation goes is theoretical. except perhaps
in the case of rot13. :-)

>  * cleanup of the PyUnicode_AsString() and PyUnicode_AsStringAndSize()
>   C APIs (these APIs don't fit into the naming scheme used in the
>   Unicode API and have a few other issues as well, see issue 2799;
>   at the very least they should be made interpreter internal, ie.
>   rename them to _PyUnicode_AsString() and _PyUnicode_AsStringAndSize()
>   to prevent their use in extensions)

I'm okay with this too.

> I did not have time in the last few days to work on these and won't
> in the next few days either. Next week looks much better.
>
> If it's ok to make the above changes after the release (whatever you
> call it ;-), that would be great.

That's up to the release manager, but IMO we could have a small list
of *specific* things that are not yet implemented in beta 1 but that
we know will be in beta 2. This is IMO better than just calling it
another alpha, because that keeps the floodgates for more feature
change requests open for another month.

--Guido

> Thanks,
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jun 11 2008)
>>>>
>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>
> ________________________________________________________________________
> 2008-07-07: EuroPython 2008, Vilnius, Lithuania            25 days to go
>
> :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jun 11 19:27:20 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Jun 2008 10:27:20 -0700
Subject: [Python-Dev] Have been sick, am behind on mail,
	let me know if there's anything urgent for me
In-Reply-To: <g2nn1i$413$1@ger.gmane.org>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
	<g2nn1i$413$1@ger.gmane.org>
Message-ID: <ca471dc20806111027m23798254x9690d3fa8146a9e7@mail.gmail.com>

On Tue, Jun 10, 2008 at 10:15 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> This seems to require a BDFL decision:
>
> http://bugs.python.org/issue2997
>
> Executive Summary: PyNumberMethods has been changed on py3k back in 2006 with
> the nb_divide removal, so it's now incompatible with Py2. But there are three
> more unused struct members *behind* that field that can be removed for beta1,
> but have not been removed yet. Should they be removed for cleanliness (patch
> in the issue) or should nb_divide and nb_inplace_divide instead be restored
> (rev 43285) to restore backwards compatibility?

I've added a recommendation to apply this patch to the bug.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Jun 11 19:29:42 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Jun 2008 10:29:42 -0700
Subject: [Python-Dev] Have been sick, am behind on mail,
	let me know if there's anything urgent for me
In-Reply-To: <484FCC1D.1010605@gmail.com>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
	<7F90796A-E7D2-4C32-A03A-794EFCE94C14@python.org>
	<484FCC1D.1010605@gmail.com>
Message-ID: <ca471dc20806111029r202959ecy6b7b69208b1d3528@mail.gmail.com>

On Wed, Jun 11, 2008 at 5:59 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> http://bugs.python.org/issue643841
>>
>> It's a long bug thread, but needs a decision.
>
> I just added a couple of messages at the end that recap the more recent
> discussions (the early responses to the issue are *really* old...)

I've added my opinion to the bug: I am in favor of documenting the
current behavior (making it mandatory for all implementations). I
would like to hold back on adding a proxy mixin class until we've
found some actual use for it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Wed Jun 11 19:37:01 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jun 2008 13:37:01 -0400
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
	<ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>
Message-ID: <F3F1C7BD-8D54-4320-BCD9-3608F4DDFB85@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 11, 2008, at 1:02 PM, Guido van Rossum wrote:

> On Wed, Jun 11, 2008 at 4:35 AM, Barry Warsaw <barry at python.org>  
> wrote:
>> So I had planned to do a bunch of work last night looking at the  
>> release
>> blocker issues, but nature intervened.  A bunch of severe  
>> thunderstorms
>> knock out my 'net access until this morning.
>>
>> I'll try to find some time during the day to look at the RB issues.
>> Hopefully we can get Guido to look at them too and Pronounce on  
>> some of
>> them.  Guido please start with:
>>
>> http://bugs.python.org/issue643841
>
> I've added a comment. Let me know if anything I said is unclear.

Nope, it was perfect, thanks.

>> My plan is to begin building the betas tonight, at around 9 or 10pm  
>> EDT
>> (0100 to 0200 UTC Thursday).  If a showstopper comes up before  
>> then, I'll
>> email the list.  If you think we really aren't ready for beta, then  
>> I would
>> still like to get a release out today.  In that case, we'll call it  
>> alpha
>> and delay the betas.
>
> I'd rather call it beta even if certain things are still known to
> change in the next release. Beta means we'll have a much easier time
> pushing back on random other feature change proposals.

Sounds good.  I'm going to go through the other release critical  
issues and will follow up on this thread if there are any other  
questions.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFANPXEjvBPtnXfVAQIMfgQAovATNsivTKAMIUPsue+Tq5OwVlE8zxIf
KqmNfXDdyAWhWjnWrao0V73A2v6TKvgL2H6SON6UT1oHvRus1ahLWXQWpTUIdn4L
jiYfeek14XoVim8mRz7L8mhxThADPMj3YkhWK448QyZbMkColMqUIVCgvfaQYSxM
2+3zOWQJ/AQ=
=eqAr
-----END PGP SIGNATURE-----

From guido at python.org  Wed Jun 11 19:39:26 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Jun 2008 10:39:26 -0700
Subject: [Python-Dev] update_wrapper should preserve staticmethod
	behavior
In-Reply-To: <DF383E5E-53EA-4FB5-A539-6E67ECBC4FF4@socialserve.com>
References: <DF383E5E-53EA-4FB5-A539-6E67ECBC4FF4@socialserve.com>
Message-ID: <ca471dc20806111039q43b58511nf72741e2a7d112b9@mail.gmail.com>

Please submit a fix to the issue tracker at bugs.python.org if you
care about this.

On Wed, Jun 11, 2008 at 7:06 AM, Calvin Spealman
<ironfroggy at socialserve.com> wrote:
> I'd like to make a claim about the following example, that update_wrapper
> should be improved to preserve the behavior of known, built-in decorators.
> In this case, I'm talking about staticmethod. The order I list here feels
> natural, but it obviously doesn't work. The only reason it doesn't seems to
> be that it is trying to decorate the descriptor, not the function itself.
> This is expected, but it could certainly be smart enough to detect a
> descriptor and attempt to get the actual callable underneath, could it not?
> It would not work for all cases, of course.
>
>>>> def d(f):
> ...     def nf(*a, **kw):
> ...             print "decorated function called"
> ...             return f(*a, **kwargs)
> ...     functools.update_wrapper(nf, f)
> ...     return nf
> ...
>>>> class A(object):
> ...     @d
> ...     @staticmethod
> ...     def a(self):
> ...             print "a"
> ...
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>  File "<stdin>", line 3, in A
>  File "<stdin>", line 5, in d
>  File
> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/functools.py",
> line 33, in update_wrapper
>    setattr(wrapper, attr, getattr(wrapped, attr))
> AttributeError: 'staticmethod' object has no attribute '__module__'
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From solipsis at pitrou.net  Wed Jun 11 19:48:54 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Jun 2008 17:48:54 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?update=5Fwrapper_should_preserve_staticmet?=
	=?utf-8?q?hod_behavior?=
References: <DF383E5E-53EA-4FB5-A539-6E67ECBC4FF4@socialserve.com>
Message-ID: <loom.20080611T174537-409@post.gmane.org>

Calvin Spealman <ironfroggy <at> socialserve.com> writes:
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "<stdin>", line 3, in A
>    File "<stdin>", line 5, in d
>    File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
> python2.5/functools.py", line 33, in update_wrapper
>      setattr(wrapper, attr, getattr(wrapped, attr))
> AttributeError: 'staticmethod' object has no attribute '__module__'

Well, if staticmethod doesn't mirror the original function's __module__
attribute, I'd say staticmethod is the culprit.

Since Python grew the update_wrapper function, it seems reasonable to ask
that all decorators (or decorator-alikes) provided with Python call
update_wrapper. Of course staticmethod is written in C, so is there a
C function somewhere providing the same functionality as update_wrapper does?




From barry at python.org  Wed Jun 11 19:54:49 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jun 2008 13:54:49 -0400
Subject: [Python-Dev] Have been sick, am behind on mail,
	let me know if there's anything urgent for me
In-Reply-To: <g2nn1i$413$1@ger.gmane.org>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
	<g2nn1i$413$1@ger.gmane.org>
Message-ID: <F905426C-0722-4237-832A-3E09FD247B21@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 11, 2008, at 1:15 AM, Stefan Behnel wrote:

> Guido van Rossum wrote:
>> anything someone really needs me to look at ASAP (e.g. a BDFL  
>> decision
>> affecting the impending beta release) please send me an email
>
> This seems to require a BDFL decision:
>
> http://bugs.python.org/issue2997

Guido's approved the patch.  Please go ahead and apply it.  If no one  
gets to it before tonight, I'll put it in beta 1 if it applies cleanly.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFARanEjvBPtnXfVAQJM/AP+I7gb3VM90c6iCAkvx8EijC2LmrIPtmN2
kSMtMLlyObEZLdGGiPOQdafwx+SeuKxwY2d/RF1VvM2K6fWyWjbw0wt2ZMLxs1UB
AVz4PmHSeS23jao2io75wBx4iUTmccte0dDBx6JJYojq6PZeMe4W5lJHuzuyrHWQ
i4EnPJHiqag=
=HaZV
-----END PGP SIGNATURE-----

From musiccomposition at gmail.com  Wed Jun 11 19:58:25 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 11 Jun 2008 12:58:25 -0500
Subject: [Python-Dev] Have been sick, am behind on mail,
	let me know if there's anything urgent for me
In-Reply-To: <F905426C-0722-4237-832A-3E09FD247B21@python.org>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
	<g2nn1i$413$1@ger.gmane.org>
	<F905426C-0722-4237-832A-3E09FD247B21@python.org>
Message-ID: <1afaf6160806111058w2be5e0bbh60321954f752c420@mail.gmail.com>

On Wed, Jun 11, 2008 at 12:54 PM, Barry Warsaw <barry at python.org> wrote:
>>
>> http://bugs.python.org/issue2997
>
> Guido's approved the patch.  Please go ahead and apply it.  If no one gets
> to it before tonight, I'll put it in beta 1 if it applies cleanly.

I'm going to handle it.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From barry at python.org  Wed Jun 11 20:00:42 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jun 2008 14:00:42 -0400
Subject: [Python-Dev] Have been sick, am behind on mail,
	let me know if there's anything urgent for me
In-Reply-To: <1afaf6160806111058w2be5e0bbh60321954f752c420@mail.gmail.com>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
	<g2nn1i$413$1@ger.gmane.org>
	<F905426C-0722-4237-832A-3E09FD247B21@python.org>
	<1afaf6160806111058w2be5e0bbh60321954f752c420@mail.gmail.com>
Message-ID: <B75E20FF-D279-4D56-9FC7-8E234BD4C7F0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 11, 2008, at 1:58 PM, Benjamin Peterson wrote:

> On Wed, Jun 11, 2008 at 12:54 PM, Barry Warsaw <barry at python.org>  
> wrote:
>>>
>>> http://bugs.python.org/issue2997
>>
>> Guido's approved the patch.  Please go ahead and apply it.  If no  
>> one gets
>> to it before tonight, I'll put it in beta 1 if it applies cleanly.
>
> I'm going to handle it.

Thanks!  You're awesome. :)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFASynEjvBPtnXfVAQK9YgQAuVbLsvZd+O2RGac/Tqr0DHcCPHm7lOL9
QyqUMzideytmuGhw4IjaIIKhP2G1Dg7iskuCEqkosm5niRKz4vw6/R53kMC5xPRQ
Adm+KfCxohXzJoN7+ON89wnM5AqfT1AxeYlhGkKGvxRfjTydIXMTTTSJLC4ztyT6
BWIG49zqEJY=
=kLlY
-----END PGP SIGNATURE-----

From greg at krypto.org  Wed Jun 11 20:02:53 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Wed, 11 Jun 2008 11:02:53 -0700
Subject: [Python-Dev] bogus comment in Python.h
In-Reply-To: <20080610171346.52EE01C7641@localhost>
References: <20080610171346.52EE01C7641@localhost>
Message-ID: <52dc1c820806111102k1db831b7n19037e4f2050cf91@mail.gmail.com>

On Tue, Jun 10, 2008 at 10:13 AM, Doug Evans <dje at google.com> wrote:

> I spent a bit of time trying to figure out what's going on here
> (was getting errors regarding missing uintptr_t while trying to compile
> an external module with Python 2.4).
> pyport.h now includes stdint.h but can we fix up this in Python.h?
>
> /* For uintptr_t, intptr_t */
> #ifdef HAVE_STDDEF_H
> #include <stddef.h>
> #endif
>
> I'm guessing removing the inclusion of stddef.h will break something.
> It's a bit of a wart (or not) that Python.h includes stdlib.h, stddef.h,
> et.al.
> but pyport.h includes stdint.h (why not just include them all in one
> place?).
>
> Anyways, to save some confusion for someone trying to track down
> a problem in the future, could the above comment be removed?


Okay.  I changed it to /* For size_t? */ in trunk (2.6).


> --- Python.h    (revision 64082)
> +++ Python.h    (working copy)
> @@ -43,8 +43,6 @@
>  #ifdef HAVE_UNISTD_H
>  #include <unistd.h>
>  #endif
> -
> -/* For uintptr_t, intptr_t */
>  #ifdef HAVE_STDDEF_H
>  #include <stddef.h>
>  #endif
>
> [I'd suggest rewording it except that I'm not sure exactly what stddef.h
> is used for.  Presumably it's size_t but I didn't want to guess.]
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080611/e23f0903/attachment-0001.htm>

From python at rcn.com  Wed Jun 11 20:03:17 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 11 Jun 2008 11:03:17 -0700
Subject: [Python-Dev] PEP 8 and optional underscores
Message-ID: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>

"Function names should be lowercase, with words separated by underscores as necessary to improve readability." -- PEP 8

If I'm reading this correctly, then underscores are not required  everywhere.  Can some of these be shortened?

    function:: active_count()
    method:: Thread.get_name()
    method:: Thread.is_alive()
    method:: Thread.is_daemon()
    method:: Thread.set_daemon(daemonic)

In some cases, the mental pronounciation changes and affects my perception of meaning.  For example, Thread.setName or 
Thread.setname both feel like a setter to me, but Thread.set_name causes a mental pause and a momentary double-take (is it the name 
of a set?).

A few months ago, I think there was a PEP 8 discussion rejecting suggestions to make underscores required everywhere (leading to 
getattr-->get_attr, iteritems-->iter_items,  staticmethod->static_method, setdefault->set_default, popitem->pop_item, 
splitlines->split_lines etc.)

Perhaps underscores should only be used when the contracted form lacks clarity.

Food for thought,


Raymond 


From musiccomposition at gmail.com  Wed Jun 11 20:21:13 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 11 Jun 2008 13:21:13 -0500
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
Message-ID: <1afaf6160806111121p37fd0071ne17cef0013a3412@mail.gmail.com>

On Wed, Jun 11, 2008 at 1:03 PM, Raymond Hettinger <python at rcn.com> wrote:
> "Function names should be lowercase, with words separated by underscores as
> necessary to improve readability." -- PEP 8
>
> If I'm reading this correctly, then underscores are not required
>  everywhere.  Can some of these be shortened?
>
>   function:: active_count()
>   method:: Thread.get_name()
>   method:: Thread.is_alive()
>   method:: Thread.is_daemon()
>   method:: Thread.set_daemon(daemonic)
>
> In some cases, the mental pronounciation changes and affects my perception
> of meaning.  For example, Thread.setName or Thread.setname both feel like a
> setter to me, but Thread.set_name causes a mental pause and a momentary
> double-take (is it the name of a set?).

Actually, in this case, I think the Pythonic thing to do would be to
use properties.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From theller at ctypes.org  Wed Jun 11 20:36:50 2008
From: theller at ctypes.org (Thomas Heller)
Date: Wed, 11 Jun 2008 20:36:50 +0200
Subject: [Python-Dev] rest markup in ctypes docs
Message-ID: <g2p602$54n$1@ger.gmane.org>

There are a few cases where the ctypes docs are rendered incorrectly:

http://docs.python.org/dev/library/ctypes.html#function-prototypes

This looks as if 'prototype' would be a symbol exposed by ctypes; it is
not - it is used as a placeholder for the object returned by calls to 
the ctypes.CFUNCTYPE, ctypes.WINFUNCTYPE, and ctypes.PYFUNCTYPE functions above.

The rest markup is like this:


.. function:: prototype(address)
   :noindex:

   Returns a foreign function at the specified address.

Some lines below, it looks like ctypes would export '1', '2', and '4'.
Here is the markup:


.. function:: prototype(address)
   :noindex:

   Returns a foreign function at the specified address.


How can this be fixed?

-- 
Thanks,
Thomas


From theller at ctypes.org  Wed Jun 11 20:40:23 2008
From: theller at ctypes.org (Thomas Heller)
Date: Wed, 11 Jun 2008 20:40:23 +0200
Subject: [Python-Dev] rest markup in ctypes docs
In-Reply-To: <g2p602$54n$1@ger.gmane.org>
References: <g2p602$54n$1@ger.gmane.org>
Message-ID: <g2p66m$54n$2@ger.gmane.org>

Thomas Heller schrieb:
> There are a few cases where the ctypes docs are rendered incorrectly:
> 
> http://docs.python.org/dev/library/ctypes.html#function-prototypes
> 
> This looks as if 'prototype' would be a symbol exposed by ctypes; it is
> not - it is used as a placeholder for the object returned by calls to 
> the ctypes.CFUNCTYPE, ctypes.WINFUNCTYPE, and ctypes.PYFUNCTYPE functions above.
> 
> The rest markup is like this:
> 
> 
> .. function:: prototype(address)
>    :noindex:
> 
>    Returns a foreign function at the specified address.
> 
> Some lines below, it looks like ctypes would export '1', '2', and '4'.
> Here is the markup:
> 
Sorry, I meant this:

.. data:: 1
   :noindex:

   Specifies an input parameter to the function.


> 
> How can this be fixed?
> 


-- 
Thanks,
Thomas


From ironfroggy at socialserve.com  Wed Jun 11 20:56:25 2008
From: ironfroggy at socialserve.com (Calvin Spealman)
Date: Wed, 11 Jun 2008 14:56:25 -0400
Subject: [Python-Dev] update_wrapper should preserve staticmethod
	behavior
In-Reply-To: <loom.20080611T174537-409@post.gmane.org>
References: <DF383E5E-53EA-4FB5-A539-6E67ECBC4FF4@socialserve.com>
	<loom.20080611T174537-409@post.gmane.org>
Message-ID: <58D5E407-8BCC-45C3-BFE8-96F0728DC412@socialserve.com>

staticmethod doesn't wrap anything, it just creates a descriptor on  
the class with a __get__ that returns the original, untouched  
callable. Doesn't even care _what_ the thing you use it on is  
(function, other callable, or something else entirely.)

This actually shouldn't be attempted on non-staticmethod descriptors,  
after thinking about it. Can't be sure that desc.__get__(cls) is  
usable to wrap when, at the end, you will be doing some_instance.a  
and now had the wrong __get__() signature used. Oh, no!

class A(object):
	@d
	@some_decorator_returns_a_descriptor
	def a():
		pass

What should probably happen here is that d needs to see its  
decorating a descriptor and itself return a descriptor to pass along  
the right behavior. So, when you do A().a() you should have d.__get__ 
(cls, inst) calling some_decorator_returns_a_descriptor.__get__(cls,  
inst) and acting as if that was the thing it decorated.

Of course, this would have the probably unexpected behavior of  
decorating such things at binding time (ie, when a classmethod is  
bound) rather than after definition. Not good. They could be cached  
and this used to implement new functionality that the decorator can  
be applied to the class method once for each class its bound to  
(interesting? useful?), but I can't think of a justification myself.

Unless any of this other behavior could be justified, I'll provide an  
update_wrapper() patch to at least become staticmethod smart.

On Jun 11, 2008, at 1:48 PM, Antoine Pitrou wrote:

> Calvin Spealman <ironfroggy <at> socialserve.com> writes:
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>>    File "<stdin>", line 3, in A
>>    File "<stdin>", line 5, in d
>>    File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
>> python2.5/functools.py", line 33, in update_wrapper
>>      setattr(wrapper, attr, getattr(wrapped, attr))
>> AttributeError: 'staticmethod' object has no attribute '__module__'
>
> Well, if staticmethod doesn't mirror the original function's  
> __module__
> attribute, I'd say staticmethod is the culprit.
>
> Since Python grew the update_wrapper function, it seems reasonable  
> to ask
> that all decorators (or decorator-alikes) provided with Python call
> update_wrapper. Of course staticmethod is written in C, so is there a
> C function somewhere providing the same functionality as  
> update_wrapper does?
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ 
> ironfroggy%40socialserve.com


From solipsis at pitrou.net  Wed Jun 11 21:22:49 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Jun 2008 19:22:49 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?update=5Fwrapper_should_preserve_staticmet?=
	=?utf-8?q?hod=09behavior?=
References: <DF383E5E-53EA-4FB5-A539-6E67ECBC4FF4@socialserve.com>
	<loom.20080611T174537-409@post.gmane.org>
	<58D5E407-8BCC-45C3-BFE8-96F0728DC412@socialserve.com>
Message-ID: <loom.20080611T191528-172@post.gmane.org>

Calvin Spealman <ironfroggy <at> socialserve.com> writes:
> 
> staticmethod doesn't wrap anything, it just creates a descriptor on  
> the class with a __get__ that returns the original, untouched  
> callable. Doesn't even care _what_ the thing you use it on is  
> (function, other callable, or something else entirely.)

FWIW, I still disagree. Technically, it might not "wrap" anything (in the sense
that it isn't defined as a function returning another function - which is a
narrow definition of a wrapper by the way), but semantically it does. To the
non-expert programmer, it is a decorator like any other one. The fact that it is
implemented differently from other decorators is not an excuse for it to follow
different rules...

Unless, of course, there is a good *semantic* reason for staticmethod not to
mirror the __module__ attribute.

(by the way, does the same apply to classmethod as well?)

Regards

Antoine.



From scott+python-dev at scottdial.com  Wed Jun 11 21:23:38 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 11 Jun 2008 15:23:38 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <20080611161455.CBD703A405F@sparrow.telecommunity.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>	<484F2C1B.2090404@scottdial.com>	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
Message-ID: <4850263A.3040805@scottdial.com>

Phillip J. Eby wrote:
> That test is there to ensure that it interoperates with code using the 
> AddOns library from the Cheeseshop; SQLAlchemy is not the source of the 
> usage.

Now that's interesting. The AddOns library uses class objects as keys in 
the __dict__, but that doesn't says anything about the usage of 
locals(). At no point in the AddOns library is locals() abused like 
this, so even if one asserts that assignment to the dict returned by 
locals() is a bug, the underlying behavior of interest is whether 
__dict__ is allowed to have non-string keys.

 >>> from peak.util.addons import AddOn
 >>> class C: pass
 >>> class A(AddOn): pass
 >>> spam = C()
 >>> print spam.__dict__
{}
 >>> A(spam)
 >>> print spam.__dict__
{<class 'A'>: <A object at ...>}

If non-string keys are not allowed in __dict__, then the AddOns library 
should be changed to add another dict to the object of interest to track 
these AddOn instances.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From g.brandl at gmx.net  Wed Jun 11 21:25:16 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 11 Jun 2008 21:25:16 +0200
Subject: [Python-Dev] rest markup in ctypes docs
In-Reply-To: <g2p66m$54n$2@ger.gmane.org>
References: <g2p602$54n$1@ger.gmane.org> <g2p66m$54n$2@ger.gmane.org>
Message-ID: <g2p8qv$evc$1@ger.gmane.org>

Thomas Heller schrieb:
> Thomas Heller schrieb:
>> There are a few cases where the ctypes docs are rendered incorrectly:
>> 
>> http://docs.python.org/dev/library/ctypes.html#function-prototypes
>> 
>> This looks as if 'prototype' would be a symbol exposed by ctypes; it is
>> not - it is used as a placeholder for the object returned by calls to 
>> the ctypes.CFUNCTYPE, ctypes.WINFUNCTYPE, and ctypes.PYFUNCTYPE functions above.
>> 
>> The rest markup is like this:
>> 
>> 
>> .. function:: prototype(address)
>>    :noindex:
>> 
>>    Returns a foreign function at the specified address.

You can use

.. function:: prototype(address)
    :noindex:
    :module:

here. (The :module: option gives the function's module name which is empty
in this case.)

>> Some lines below, it looks like ctypes would export '1', '2', and '4'.
>> Here is the markup:
>> 
> Sorry, I meant this:
> 
> ... data:: 1
>    :noindex:
> 
>    Specifies an input parameter to the function.

A simple definition list or table should suffice here.

Georg


From alexandre at peadrop.com  Wed Jun 11 22:54:48 2008
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Wed, 11 Jun 2008 16:54:48 -0400
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
Message-ID: <acd65fa20806111354h3788345xcea09b5d30cc8b0e@mail.gmail.com>

On Wed, Jun 11, 2008 at 7:35 AM, Barry Warsaw <barry at python.org> wrote:
> My plan is to begin building the betas tonight, at around 9 or 10pm EDT
> (0100 to 0200 UTC Thursday).  If a showstopper comes up before then, I'll
> email the list.  If you think we really aren't ready for beta, then I would
> still like to get a release out today.  In that case, we'll call it alpha
> and delay the betas.

I have two release blockers pending review:

  http://bugs.python.org/issue2918
  http://bugs.python.org/issue2917

I believe both patches are ready to be committed to the py3k branch.
However, I would certainly like that someone would review the patches
(or at least test them).

Right now, I am currently looking at fixing issue 2919
(http://bugs.python.org/issue2919). The profile and the cProfile
module differ much more than I originally expected.  So, I won't be
able to get these two for the beta.

I have also been looking at http://bugs.python.org/issue2874, in which
Benjamin Peterson proposed an simple solution to fix it. Although I
haven't tried his approach, I think I could get this one done for
today.

Finally, I would like to commit the patch in
http://bugs.python.org/issue2523 which fix the quadratic behavior in
BufferedReader.read(). It would also be nice to have someone else
experienced with the io module to review the patch.

Cheers,
-- Alexandre

From curt at hagenlocher.org  Wed Jun 11 23:27:51 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Wed, 11 Jun 2008 14:27:51 -0700
Subject: [Python-Dev] Assignment to None
In-Reply-To: <d2155e360806111426u16747c12r27b4239b5dfa481b@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<484CBD70.3000808@v.loewis.de>
	<d2155e360806111426u16747c12r27b4239b5dfa481b@mail.gmail.com>
Message-ID: <d2155e360806111427v234f1ad5yd9870ace7fc7b9b9@mail.gmail.com>

On Sun, Jun 8, 2008 at 10:19 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> > So, it's okay to setattr the attribute name "None" but not okay to set
> > it directly?  Is this deliberate or is it an unintentional side effect
> > of parser changes to prevent assignment to None?
>
> It's deliberate. setattr(o, "foo bar", "baz") also works, even though
> "foo bar" is not an identifier. setattr doesn't take the Python grammar
> into account, but only the object's structure.

Thanks for this example; I found it useful.


As Michael says, my primary interest in asking this question is
because I'm working on IronPython.  Compatibility with CPython is
extremely important for us, so I need to understand exactly what
behavior is mandated. Here's a recap of the ground this thread has
covered:

1. In all versions, direct access to members whose names are reserved
keywords (such as "print") is not allowed.
2. In Python 2.5 and 2.6, "True", "False" and "None" are not keywords,
but direct assignment to a member named "None" is specifically
prevented by the parser.  Members named "None" may, however, be read
directly if they are present.  There is no special handling for "True"
or "False".
3. In Python 3.0, "True", "False" and "None" are keywords.

This eventually leads to a problem for us in interoperability with
other CLR code. One example, as Michael points out, is that "None" is
a relatively common member of enumeration types. Now for IronPython
2.0, we're targeting compatibility with CPython 2.5.  If we duplicate
2.5's behavior and allow direct reads but not direct writes for a
member named None, we'd be okay for the enumeration example.
Enumerated values aren't writable anyway.

However, this sets us up for a problem with some hypothetical future
version of IronPython that targets 3.0 compabililty.  At that point,
we'll face the unpleasant task of having to choose between compability
and interoperability.  We haven't really had to do this before now
because the convention in CLR-based code is typically that
publicly-exposed symbols start with a capital letter -- and all of the
existing reserved words in Python are lower-case.  It's likely to be a
much bigger issue with Jython, given the Java convention of having
lower-cased method names.

If I recall correctly, Jython handles this by appending a trailing
underscore to the imported name and there's no reason why we couldn't
do something similar.

--
Curt Hagenlocher
curt at hagenlocher.org

From guido at python.org  Wed Jun 11 23:39:21 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Jun 2008 14:39:21 -0700
Subject: [Python-Dev] update_wrapper should preserve staticmethod
	behavior
In-Reply-To: <loom.20080611T191528-172@post.gmane.org>
References: <DF383E5E-53EA-4FB5-A539-6E67ECBC4FF4@socialserve.com>
	<loom.20080611T174537-409@post.gmane.org>
	<58D5E407-8BCC-45C3-BFE8-96F0728DC412@socialserve.com>
	<loom.20080611T191528-172@post.gmane.org>
Message-ID: <ca471dc20806111439ka9db04fl84b6cd4b6b99d98a@mail.gmail.com>

On Wed, Jun 11, 2008 at 12:22 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Calvin Spealman <ironfroggy <at> socialserve.com> writes:
>>
>> staticmethod doesn't wrap anything, it just creates a descriptor on
>> the class with a __get__ that returns the original, untouched
>> callable. Doesn't even care _what_ the thing you use it on is
>> (function, other callable, or something else entirely.)
>
> FWIW, I still disagree. Technically, it might not "wrap" anything (in the sense
> that it isn't defined as a function returning another function - which is a
> narrow definition of a wrapper by the way), but semantically it does. To the
> non-expert programmer, it is a decorator like any other one. The fact that it is
> implemented differently from other decorators is not an excuse for it to follow
> different rules...
>
> Unless, of course, there is a good *semantic* reason for staticmethod not to
> mirror the __module__ attribute.
>
> (by the way, does the same apply to classmethod as well?)

Remember (a) these are implemented in C, and (b) they were created in
Python 2.2, before we even had decorators (first introduced in 2.4).
I'm fine with making them more transparent and conformant to emerging
protocols, but they will always be a little different, due to being
implemented as objects rather than functional wrappers. The latter
approach *can* be used for decorator implementations to, it just is
done rarely.

I see this as something that can be done post beta 1. It has to be
done while carefully remaining backwards compatible though.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Wed Jun 11 23:42:59 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jun 2008 17:42:59 -0400
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <acd65fa20806111354h3788345xcea09b5d30cc8b0e@mail.gmail.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
	<acd65fa20806111354h3788345xcea09b5d30cc8b0e@mail.gmail.com>
Message-ID: <45AC8DA6-353F-41F5-997C-15E419D1D1F8@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 11, 2008, at 4:54 PM, Alexandre Vassalotti wrote:

> On Wed, Jun 11, 2008 at 7:35 AM, Barry Warsaw <barry at python.org>  
> wrote:
>> My plan is to begin building the betas tonight, at around 9 or 10pm  
>> EDT
>> (0100 to 0200 UTC Thursday).  If a showstopper comes up before  
>> then, I'll
>> email the list.  If you think we really aren't ready for beta, then  
>> I would
>> still like to get a release out today.  In that case, we'll call it  
>> alpha
>> and delay the betas.
>
> I have two release blockers pending review:
>
>  http://bugs.python.org/issue2918
>  http://bugs.python.org/issue2917
>
> I believe both patches are ready to be committed to the py3k branch.
> However, I would certainly like that someone would review the patches
> (or at least test them).

In IRC, Benjamin agreed to review these and commit them if they look  
okay.  I'm still 3-4 hours away from making the betas, so there's time  
to do this and still see how the buildbots react.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFBG43EjvBPtnXfVAQJSYAP9FwvFppbeIQX8IhCUtXAXf5jSoGkpTKHQ
FT5PBhI/WmM1HTwtx3i06EA/3P+rB2yIVOJhCuK1ORoLvAZO1C8gKUp/8yvgH2KD
0OtlVYgfs4iXwgXe36dI9uBt9AIaohHNnuEgzqzH/IXIcdZ21bp3WKJTbvSgruFX
q0K3SkO9ano=
=TI1T
-----END PGP SIGNATURE-----

From barry at python.org  Thu Jun 12 00:13:49 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jun 2008 18:13:49 -0400
Subject: [Python-Dev] Potential beta showstopper (was Re:  Have been sick,
	am behind on mail, let me know if there's anything urgent for me)
In-Reply-To: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
Message-ID: <0BFACB06-37F1-447A-8828-2B41DB7C98D1@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Okay, we have a potential showstopper for the betas today.  All the  
3.0 buildbots are bloody red:

http://www.python.org/dev/buildbot/stable/

Running the tests locally on OS X 10.5 and Ubuntu 8.04 confirm that  
the tests hang after test_xmlrpc.  The tests are uninterruptible.

I hereby give permission to any dev with commit privileges to checkin  
a fix, or back out the offending revision.  I will be off-line for  
about 3 more hours and if this can't be fixed by then, we will have to  
postpone the releases.

Once I'm back, I'll be on #python-dev @ freenode.

Thanks to Benjamin and Alexandre for helping me get through the  
current crop of release critical issues today.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFBOHXEjvBPtnXfVAQK6zQP+Nnjt6Q97heAuquZgKEfMxXBL/QzodDgB
Th58+w04h0gTVW39Y9NEeKynVPrjsbjxKdrbdZ0i+1pEbqJMXyV65jFmxqb+mYdd
194s5hm+U0KEk7h+64aEkmvFPAgeZ2TGBrqCu5CWDNbWmqerPWuNDzqPxQrWd4Mt
wcaic8LFaaU=
=4MTf
-----END PGP SIGNATURE-----

From musiccomposition at gmail.com  Thu Jun 12 00:18:26 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 11 Jun 2008 17:18:26 -0500
Subject: [Python-Dev] [Python-3000] Potential beta showstopper (was Re:
	Have been sick, am behind on mail,
	let me know if there's anything urgent for me)
In-Reply-To: <0BFACB06-37F1-447A-8828-2B41DB7C98D1@python.org>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
	<0BFACB06-37F1-447A-8828-2B41DB7C98D1@python.org>
Message-ID: <1afaf6160806111518t3655df6bl80a6b4dc589d08c3@mail.gmail.com>

On Wed, Jun 11, 2008 at 5:13 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Okay, we have a potential showstopper for the betas today.  All the 3.0
> buildbots are bloody red:
>
> http://www.python.org/dev/buildbot/stable/
>
> Running the tests locally on OS X 10.5 and Ubuntu 8.04 confirm that the
> tests hang after test_xmlrpc.  The tests are uninterruptible.
>
> I hereby give permission to any dev with commit privileges to checkin a fix,
> or back out the offending revision.  I will be off-line for about 3 more
> hours and if this can't be fixed by then, we will have to postpone the
> releases.

Already done.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From tjreedy at udel.edu  Thu Jun 12 02:32:45 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 11 Jun 2008 20:32:45 -0400
Subject: [Python-Dev] bug or a feature?
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>	<484F2C1B.2090404@scottdial.com>	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com><20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com>
Message-ID: <g2pqrc$pel$1@ger.gmane.org>


"Scott Dial" <scott+python-dev at scottdial.com> wrote in message 
news:4850263A.3040805 at scottdial.com...
|| If non-string keys are not allowed in __dict__, then the AddOns library
| should be changed to add another dict to the object of interest to track
| these AddOn instances.

There are three possibilities with respect to __dict__ and non-string keys.
1. All implementations must reject such.
2. Each implementation decides for itself.
3. All implementations must allow such.

Current, CPython does not reject, eliminating 1).  Since, as I understand 
it, at least 1 other implementation does reject, 3) is also eliminated 
until Guido decrees otherwise and such implementation(s) change.  This 
leaves 2) as the de facto situation, but this could be made clearer in the 
docs: "The result of trying to add non-string keys to any __dict__ 
attribute is implementation defined."  This means that portable Python code 
must act as if 1) were the case.

The Data Model chapter of the Reference Manual lists .__dict__ as a special 
attribute of callables, modules, classes, and instances.  It describes 
__dict__ as a "namespace dictionary" or "implementation of the namespace" 
thereof.  Since namespaces map names (or possibly non-name strings) to 
objects, this to me implies that an implementation is and should not be 
required to allow non-strings in __dict__.

The same chapter has more than one sentence saying something like "o.x is 
equivalent to o.__dict__['x']".  While one could read this as prohibiting 
o.__dict__[non_string], one could also read it as being silent, neither 
allowing nor prohibiting.

The builtin interface functions setattr, hasattr, getattr all require 
strings for accessing the underlying __dict__.  Since they could have been 
defined otherwise, to accept any object as an attribute 'name' (key), this 
again implies (to me) that __dict__s are only intended and only required to 
have string keys.  Hence, I was initially surprised that 1) above was not 
true.  So I might add something stronger to the docs, something like  "The 
special __dict__ attributes are only intended to hold string keys.  If an 
implementation allows other keys, that is only an current accidental 
side-effect of considerations of parsimony and efficiency."

Contrariwise, if 3) were mandated, then I would think that the xxxattr 
functions should be changed.

Terry Jan Reedy




From fijall at gmail.com  Thu Jun 12 02:59:27 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 12 Jun 2008 02:59:27 +0200
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <g2pqrc$pel$1@ger.gmane.org>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
Message-ID: <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>

On Thu, Jun 12, 2008 at 2:32 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>
> "Scott Dial" <scott+python-dev at scottdial.com> wrote in message
> news:4850263A.3040805 at scottdial.com...
> || If non-string keys are not allowed in __dict__, then the AddOns library
> | should be changed to add another dict to the object of interest to track
> | these AddOn instances.
>
> There are three possibilities with respect to __dict__ and non-string keys.
> 1. All implementations must reject such.
> 2. Each implementation decides for itself.
> 3. All implementations must allow such.
>
> Current, CPython does not reject, eliminating 1).  Since, as I understand
> it, at least 1 other implementation does reject, 3) is also eliminated
> until Guido decrees otherwise and such implementation(s) change.  This
> leaves 2) as the de facto situation, but this could be made clearer in the
> docs: "The result of trying to add non-string keys to any __dict__
> attribute is implementation defined."  This means that portable Python code
> must act as if 1) were the case.
>
> The Data Model chapter of the Reference Manual lists .__dict__ as a special
> attribute of callables, modules, classes, and instances.  It describes
> __dict__ as a "namespace dictionary" or "implementation of the namespace"
> thereof.  Since namespaces map names (or possibly non-name strings) to
> objects, this to me implies that an implementation is and should not be
> required to allow non-strings in __dict__.
>
> The same chapter has more than one sentence saying something like "o.x is
> equivalent to o.__dict__['x']".  While one could read this as prohibiting
> o.__dict__[non_string], one could also read it as being silent, neither
> allowing nor prohibiting.
>
> The builtin interface functions setattr, hasattr, getattr all require
> strings for accessing the underlying __dict__.  Since they could have been
> defined otherwise, to accept any object as an attribute 'name' (key), this
> again implies (to me) that __dict__s are only intended and only required to
> have string keys.  Hence, I was initially surprised that 1) above was not
> true.  So I might add something stronger to the docs, something like  "The
> special __dict__ attributes are only intended to hold string keys.  If an
> implementation allows other keys, that is only an current accidental
> side-effect of considerations of parsimony and efficiency."
>
> Contrariwise, if 3) were mandated, then I would think that the xxxattr
> functions should be changed.
>
> Terry Jan Reedy
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
>

This is completely irrelevant. This post is not about assigning
non-string stuff to __dict__ of class which works completely fine.
It's about abusing locals, which are not even given that they'll
modify this dict.

From tjreedy at udel.edu  Thu Jun 12 04:01:50 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 11 Jun 2008 22:01:50 -0400
Subject: [Python-Dev] bug or a feature?
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com><484F2C1B.2090404@scottdial.com><693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com><20080611161455.CBD703A405F@sparrow.telecommunity.com><4850263A.3040805@scottdial.com>
	<g2pqrc$pel$1@ger.gmane.org>
	<693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>
Message-ID: <g2q02d$4p5$1@ger.gmane.org>


"Maciej Fijalkowski" <fijall at gmail.com> wrote in message 
news:693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5 at mail.gmail.com...
| On Thu, Jun 12, 2008 at 2:32 AM, Terry Reedy <tjreedy at udel.edu> wrote:
| >
| > "Scott Dial" <scott+python-dev at scottdial.com> wrote in message
| > news:4850263A.3040805 at scottdial.com...
| > || If non-string keys are not allowed in __dict__, then the AddOns 
library
| > | should be changed to add another dict to the object of interest to 
track
| > | these AddOn instances.
| >
| > There are three possibilities with respect to __dict__ and non-string 
keys.
| > 1. All implementations must reject such.
| > 2. Each implementation decides for itself.
| > 3. All implementations must allow such.
| >
| > Current, CPython does not reject, eliminating 1).  Since, as I 
understand
| > it, at least 1 other implementation does reject, 3) is also eliminated
| > until Guido decrees otherwise and such implementation(s) change.  This
| > leaves 2) as the de facto situation, but this could be made clearer in 
the
| > docs: "The result of trying to add non-string keys to any __dict__
| > attribute is implementation defined."  This means that portable Python 
code
| > must act as if 1) were the case.
| >
| > The Data Model chapter of the Reference Manual lists .__dict__ as a 
special
| > attribute of callables, modules, classes, and instances.  It describes
| > __dict__ as a "namespace dictionary" or "implementation of the 
namespace"
| > thereof.  Since namespaces map names (or possibly non-name strings) to
| > objects, this to me implies that an implementation is and should not be
| > required to allow non-strings in __dict__.
| >
| > The same chapter has more than one sentence saying something like "o.x 
is
| > equivalent to o.__dict__['x']".  While one could read this as 
prohibiting
| > o.__dict__[non_string], one could also read it as being silent, neither
| > allowing nor prohibiting.
| >
| > The builtin interface functions setattr, hasattr, getattr all require
| > strings for accessing the underlying __dict__.  Since they could have 
been
| > defined otherwise, to accept any object as an attribute 'name' (key), 
this
| > again implies (to me) that __dict__s are only intended and only 
required to
| > have string keys.  Hence, I was initially surprised that 1) above was 
not
| > true.  So I might add something stronger to the docs, something like 
"The
| > special __dict__ attributes are only intended to hold string keys.  If 
an
| > implementation allows other keys, that is only an current accidental
| > side-effect of considerations of parsimony and efficiency."
| >
| > Contrariwise, if 3) were mandated, then I would think that the xxxattr
| > functions should be changed.

| This is completely irrelevant. This post is not about assigning
| non-string stuff to __dict__ of class which works completely fine.

My apologies for clipping too much of Scott's post.  Here is the rest that 
came before what I quoted, which makes clear, from first sentence to last 
line, that *he* *is* talking about assigning non-string stuff to __dict__ 
of a class.

"The AddOns library uses class objects as keys in
the __dict__, but that doesn't says anything about the usage of
locals(). At no point in the AddOns library is locals() abused like
this, so even if one asserts that assignment to the dict returned by
locals() is a bug, the underlying behavior of interest is whether
__dict__ is allowed to have non-string keys.

 >>> from peak.util.addons import AddOn
 >>> class C: pass
 >>> class A(AddOn): pass
 >>> spam = C()
 >>> print spam.__dict__
{}
 >>> A(spam)
 >>> print spam.__dict__
{<class 'A'>: <A object at ...>}
"

Whether non-strings keys in o.__dict__ 'works fine' depends on one's 
definition of 'works fine'.  In any case, as of 3.0a5, I thinks the docs 
could better clarify the status of *this* 'feature'.  Whatever 
pronouncement Guido has made has not, that I could find, made it in.

| It's about abusing locals, which are not even given that they'll
| modify this dict.

I thought it settled that that is a bad thing to do.  Here the doc is 
pretty clear.

tjr





From guido at python.org  Thu Jun 12 04:16:36 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Jun 2008 19:16:36 -0700
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <45AC8DA6-353F-41F5-997C-15E419D1D1F8@python.org>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
	<acd65fa20806111354h3788345xcea09b5d30cc8b0e@mail.gmail.com>
	<45AC8DA6-353F-41F5-997C-15E419D1D1F8@python.org>
Message-ID: <ca471dc20806111916g7ffdc072i55ae6d10af9d1f0c@mail.gmail.com>

On Wed, Jun 11, 2008 at 2:42 PM, Barry Warsaw <barry at python.org> wrote:
> On Jun 11, 2008, at 4:54 PM, Alexandre Vassalotti wrote:
>> I have two release blockers pending review:
>>
>>  http://bugs.python.org/issue2918
>>  http://bugs.python.org/issue2917
>>
>> I believe both patches are ready to be committed to the py3k branch.
>> However, I would certainly like that someone would review the patches
>> (or at least test them).
>
> In IRC, Benjamin agreed to review these and commit them if they look okay.
>  I'm still 3-4 hours away from making the betas, so there's time to do this
> and still see how the buildbots react.

That's great, and I see these were committed and the bugs closed. Next
time, can we also have some kind of OK from the reviewer (Benjamin, I
presume) in the issue tracker? IRC does not leave a usable trail of
decisions.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jun 12 04:18:53 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Jun 2008 19:18:53 -0700
Subject: [Python-Dev] [Python-3000] Potential beta showstopper (was Re:
	Have been sick, am behind on mail,
	let me know if there's anything urgent for me)
In-Reply-To: <1afaf6160806111518t3655df6bl80a6b4dc589d08c3@mail.gmail.com>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
	<0BFACB06-37F1-447A-8828-2B41DB7C98D1@python.org>
	<1afaf6160806111518t3655df6bl80a6b4dc589d08c3@mail.gmail.com>
Message-ID: <ca471dc20806111918j5bf58f9vbcc9d8d01c2d1bad@mail.gmail.com>

On Wed, Jun 11, 2008 at 3:18 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Wed, Jun 11, 2008 at 5:13 PM, Barry Warsaw <barry at python.org> wrote:
>> Okay, we have a potential showstopper for the betas today.  All the 3.0
>> buildbots are bloody red:
>>
>> http://www.python.org/dev/buildbot/stable/
>>
>> Running the tests locally on OS X 10.5 and Ubuntu 8.04 confirm that the
>> tests hang after test_xmlrpc.  The tests are uninterruptible.
>>
>> I hereby give permission to any dev with commit privileges to checkin a fix,
>> or back out the offending revision.  I will be off-line for about 3 more
>> hours and if this can't be fixed by then, we will have to postpone the
>> releases.
>
> Already done.

Done what? Fixed, or backed out? Any more details? Old farts who
aren't on IRC want to know. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Thu Jun 12 04:21:27 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jun 2008 22:21:27 -0400
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <ca471dc20806111916g7ffdc072i55ae6d10af9d1f0c@mail.gmail.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
	<acd65fa20806111354h3788345xcea09b5d30cc8b0e@mail.gmail.com>
	<45AC8DA6-353F-41F5-997C-15E419D1D1F8@python.org>
	<ca471dc20806111916g7ffdc072i55ae6d10af9d1f0c@mail.gmail.com>
Message-ID: <FDBE3994-7DD2-4CA6-869B-61DBDE346F82@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 11, 2008, at 10:16 PM, Guido van Rossum wrote:

> On Wed, Jun 11, 2008 at 2:42 PM, Barry Warsaw <barry at python.org>  
> wrote:
>> On Jun 11, 2008, at 4:54 PM, Alexandre Vassalotti wrote:
>>> I have two release blockers pending review:
>>>
>>> http://bugs.python.org/issue2918
>>> http://bugs.python.org/issue2917
>>>
>>> I believe both patches are ready to be committed to the py3k branch.
>>> However, I would certainly like that someone would review the  
>>> patches
>>> (or at least test them).
>>
>> In IRC, Benjamin agreed to review these and commit them if they  
>> look okay.
>> I'm still 3-4 hours away from making the betas, so there's time to  
>> do this
>> and still see how the buildbots react.
>
> That's great, and I see these were committed and the bugs closed. Next
> time, can we also have some kind of OK from the reviewer (Benjamin, I
> presume) in the issue tracker? IRC does not leave a usable trail of
> decisions.

Good point.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFCIJ3EjvBPtnXfVAQIvXAP/fB6WAD4lS3XEKyEt/FwEoBfgileV4/bj
4km/LPOEtjN4jNks8dSoL+JHsym/76r0uylx3s3jH1sLCKmM7i9tD/SNo/Cim6r9
5fgDmreZkDU+zyyvhdiuxUl+jxmroDPd3R6vPptVbwly4SHsftceZ6jToJUvcNPd
p0nDoyCI8VU=
=xktw
-----END PGP SIGNATURE-----

From musiccomposition at gmail.com  Thu Jun 12 04:32:46 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 11 Jun 2008 21:32:46 -0500
Subject: [Python-Dev] [Python-3000] Potential beta showstopper (was Re:
	Have been sick, am behind on mail,
	let me know if there's anything urgent for me)
In-Reply-To: <ca471dc20806111918j5bf58f9vbcc9d8d01c2d1bad@mail.gmail.com>
References: <ca471dc20806101707v5017b81fx2232353d85e39753@mail.gmail.com>
	<0BFACB06-37F1-447A-8828-2B41DB7C98D1@python.org>
	<1afaf6160806111518t3655df6bl80a6b4dc589d08c3@mail.gmail.com>
	<ca471dc20806111918j5bf58f9vbcc9d8d01c2d1bad@mail.gmail.com>
Message-ID: <1afaf6160806111932v2371bdbbhca3e25ccea7c5620@mail.gmail.com>

On Wed, Jun 11, 2008 at 9:18 PM, Guido van Rossum <guido at python.org> wrote:
> On Wed, Jun 11, 2008 at 3:18 PM, Benjamin Peterson
>>
>> Already done.
>
> Done what? Fixed, or backed out? Any more details? Old farts who
> aren't on IRC want to know. :-)

That would be fixed.

>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From scott+python-dev at scottdial.com  Thu Jun 12 04:44:17 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 11 Jun 2008 22:44:17 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>	<484F2C1B.2090404@scottdial.com>	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>	<20080611161455.CBD703A405F@sparrow.telecommunity.com>	<4850263A.3040805@scottdial.com>
	<g2pqrc$pel$1@ger.gmane.org>
	<693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>
Message-ID: <48508D81.2050304@scottdial.com>

Maciej Fijalkowski wrote:
> On Thu, Jun 12, 2008 at 2:32 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> "Scott Dial" <scott+python-dev at scottdial.com> wrote in message
>> news:4850263A.3040805 at scottdial.com...
>> || If non-string keys are not allowed in __dict__, then the AddOns library
>> | should be changed to add another dict to the object of interest to track
>> | these AddOn instances.
>>
>> There are three possibilities with respect to __dict__ and non-string keys.
>> 1. All implementations must reject such.
>> 2. Each implementation decides for itself.
>> 3. All implementations must allow such.
>>
>> Current, CPython does not reject, eliminating 1).  Since, as I understand
>> it, at least 1 other implementation does reject, 3) is also eliminated
>> until Guido decrees otherwise and such implementation(s) change.  This
>> leaves 2) as the de facto situation, but this could be made clearer in the
>> docs: "The result of trying to add non-string keys to any __dict__
>> attribute is implementation defined."  This means that portable Python code
>> must act as if 1) were the case.
> 
> This is completely irrelevant. This post is not about assigning
> non-string stuff to __dict__ of class which works completely fine.
> It's about abusing locals, which are not even given that they'll
> modify this dict.

Not withstanding Terry's respond, this is not as off-topic as you make
it out to be. The test case you cited is in fact test this exact
'feature'. And as Terry expounded on it, it was unclear to me whether
that was even of allowed. The only reason the test used locals() was
because it was the only way to insert a non-string key into the class
namespace.

>>> class A:
...     locals()[42] = 98
>>> A.__dict__
{'__module__': '__main__', 43: 1, '__doc__': None}

locals() has to be used because __dict__ is unavailable at definition.

>>> class A:
...     __dict__[42] = 98
NameError: name '__dict__' is not defined

So, while one can write:

>>> class A: pass
>>> a.__dict__[42] = 98

But that's not quite the same. Nevertheless, it was still unclear
whether there was any pronouncements on non-string keys. Sorry for
wasting your time.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From barry at python.org  Thu Jun 12 07:00:19 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Jun 2008 01:00:19 -0400
Subject: [Python-Dev] No betas today
Message-ID: <3F811617-7E55-4290-BB1E-14229E563C5F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Much thanks to Benjamin Peterson and Alexandre Vassalotti for  
providing much help today trying to get the betas out.  Unfortunately,  
it's not gonna happen today.  There are two issues blocking the release.

http://bugs.python.org/issue3088
http://bugs.python.org/issue3089

The main problem (issue 3089) is that none of the Python 3.0 buildbots  
are green.  Issue 3088 may only happen for me, but  
test_multiprocessing simply hangs for me on OS X 10.5.3.

I'm going to bed now.  As soon as we can turn the bots green we can  
try again.  This weekend might be a good time for me.

Please help out by resolving these two issues.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFCtZHEjvBPtnXfVAQLrHQQAoq1RCNeawpJHaefuvmZiS4VZzzvtV5Bm
HFW6FWyO86NNEcCK6Delthf+H1mgjCaoymHojxBJhWm7UJK0WjMz+0RpF8zcLo0R
i8I8SiFLy6kRB5UxlZHS/VXDi8QS92SogRvlnWLBwXNNK6wlgeqK8C0zt1ilL0q6
Fqk/AsGO/fA=
=HlEA
-----END PGP SIGNATURE-----

From thehazard at gmail.com  Wed Jun 11 22:25:21 2008
From: thehazard at gmail.com (Daniel Bonekeeper)
Date: Wed, 11 Jun 2008 17:25:21 -0300
Subject: [Python-Dev] OFF-TOPIC-BUT-IMPORTANT: End of free internet model by
	2012 ?
Message-ID: <e1e1d5f40806111325t3e6de92v9678c01b80726744@mail.gmail.com>

Hi everybody on the list !

Sorry about the off-topic message, but giving the nature of the
message and the nature of the list (after all, everybody here uses
Internet or develops for web in some way), I decided to post this here
and see what you guys think about it.

http://ipower.ning.com/netneutrality

To quote the article (so you know if it interests you or not):

"Bell Canada and TELUS (formerly owned by Verizon) employees
officially confirm that by 2012 ISP's all over the globe will reduce
Internet access to a TV-like subscription model, only offering access
to a small standard amount of commercial sites and require extra fees
for every other site you visit. These 'other' sites would then lose
all their exposure and eventually shut down, resulting in what could
be seen as the end of the Internet."

It is important to note that, although the article mentions Bell
Canada, it is not that hard to imagine that this will eventually be
expanded to all big ISPs.

Thanks and again, I apologize for this off-topic message !

Daniel

--
What this world needs is a good five-dollar plasma weapon.

From ncoghlan at gmail.com  Thu Jun 12 11:48:52 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Jun 2008 19:48:52 +1000
Subject: [Python-Dev] update_wrapper should preserve staticmethod
	behavior
In-Reply-To: <DF383E5E-53EA-4FB5-A539-6E67ECBC4FF4@socialserve.com>
References: <DF383E5E-53EA-4FB5-A539-6E67ECBC4FF4@socialserve.com>
Message-ID: <4850F104.60402@gmail.com>

Calvin Spealman wrote:
> I'd like to make a claim about the following example, that 
> update_wrapper should be improved to preserve the behavior of known, 
> built-in decorators. In this case, I'm talking about staticmethod. The 
> order I list here feels natural, but it obviously doesn't work. The only 
> reason it doesn't seems to be that it is trying to decorate the 
> descriptor, not the function itself. This is expected, but it could 
> certainly be smart enough to detect a descriptor and attempt to get the 
> actual callable underneath, could it not? It would not work for all 
> cases, of course.
> 
>  >>> def d(f):
> ...     def nf(*a, **kw):
> ...             print "decorated function called"
> ...             return f(*a, **kwargs)
> ...     functools.update_wrapper(nf, f)
> ...     return nf
> ...
>  >>> class A(object):
> ...     @d
> ...     @staticmethod
> ...     def a(self):
> ...             print "a"
> ...
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 3, in A
>   File "<stdin>", line 5, in d
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/functools.py", 
> line 33, in update_wrapper
>     setattr(wrapper, attr, getattr(wrapped, attr))
> AttributeError: 'staticmethod' object has no attribute '__module__'

d expects a function object, this code gives it a nonfunction object - 
that's a bug in the function definition. Decorators differ in whether or 
not they are stackable with other function decorators (i.e. they return 
a function object, or another callable with a compatible API), or 
whether they can only be used as terminal decorators (i.e. they return 
something that doesn't look like a function).

classmethod and staticmethod fit in the latter category, and I don't see 
any reason to change them.

Consider what happens in your example without the update_wrapper line:

 >>> def d(f):
...     def new(*args, **kwds):
...         print "Calling decorated function"
...         return f(*args, **kwds)
...     return new
...
 >>> class A(object):
...     @d
...     @staticmethod
...     def a(*args, **kwds):
...         print "Method called"
...         print "Args:", args
...         print "Keywords:", kwds
...
 >>> A().a()
Calling decorated function
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "<stdin>", line 4, in new
TypeError: 'staticmethod' object is not callable

If anything, the AttributeError from update wrapper is a net win since 
the buggy code breaks at class definition time rather than when you try 
to call the broken method.

Cheers,
Nick.

P.S. Checking for __get__ won't help with detecting non-function 
descriptors anyway - function objects themselves define that method in 
order to provide Python's normal instance method binding behaviour.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Thu Jun 12 11:59:04 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Jun 2008 19:59:04 +1000
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <1afaf6160806111121p37fd0071ne17cef0013a3412@mail.gmail.com>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<1afaf6160806111121p37fd0071ne17cef0013a3412@mail.gmail.com>
Message-ID: <4850F368.60106@gmail.com>

Benjamin Peterson wrote:
> On Wed, Jun 11, 2008 at 1:03 PM, Raymond Hettinger <python at rcn.com> wrote:
>> "Function names should be lowercase, with words separated by underscores as
>> necessary to improve readability." -- PEP 8
>>
>> If I'm reading this correctly, then underscores are not required
>>  everywhere.  Can some of these be shortened?
>>
>>   function:: active_count()
>>   method:: Thread.get_name()
>>   method:: Thread.is_alive()
>>   method:: Thread.is_daemon()
>>   method:: Thread.set_daemon(daemonic)
>>
>> In some cases, the mental pronounciation changes and affects my perception
>> of meaning.  For example, Thread.setName or Thread.setname both feel like a
>> setter to me, but Thread.set_name causes a mental pause and a momentary
>> double-take (is it the name of a set?).
> 
> Actually, in this case, I think the Pythonic thing to do would be to
> use properties.

Thus exposing those details as thread.name, thread.alive, 
thread.daemonic (with the first two being read-only and the last 
read/write)?

+1 for that approach on the threading.Thread and multiprocessing.Process 
classes.

Not sure what to do about the active thread count function - shortening 
it as Raymond suggests would probably be fine.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From pje at telecommunity.com  Thu Jun 12 12:01:18 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 12 Jun 2008 06:01:18 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <g2pqrc$pel$1@ger.gmane.org>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
Message-ID: <20080612100048.7EEEA3A405F@sparrow.telecommunity.com>

At 08:32 PM 6/11/2008 -0400, Terry Reedy wrote:
>The Data Model chapter of the Reference Manual lists .__dict__ as a special
>attribute of callables, modules, classes, and instances.  It describes
>__dict__ as a "namespace dictionary" or "implementation of the namespace"
>thereof.  Since namespaces map names (or possibly non-name strings) to
>objects, this to me implies that an implementation is and should not be
>required to allow non-strings in __dict__.
>
>The same chapter has more than one sentence saying something like "o.x is
>equivalent to o.__dict__['x']".  While one could read this as prohibiting
>o.__dict__[non_string], one could also read it as being silent, neither
>allowing nor prohibiting.

As it happens, most objects' __dict__ slots are settable by default, 
and *require* that you set it to a dict or subclass thereof.  This 
seems (to me) to imply that a standard dictionary (i.e. one 
supporting keys of any type) is required.  (In the sense that a dict 
subclass which rejects non-strings would be violating the Liskov principle.)


From ncoghlan at gmail.com  Thu Jun 12 12:02:00 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Jun 2008 20:02:00 +1000
Subject: [Python-Dev] Assignment to None
In-Reply-To: <d2155e360806111427v234f1ad5yd9870ace7fc7b9b9@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<484CBD70.3000808@v.loewis.de>	<d2155e360806111426u16747c12r27b4239b5dfa481b@mail.gmail.com>
	<d2155e360806111427v234f1ad5yd9870ace7fc7b9b9@mail.gmail.com>
Message-ID: <4850F418.2040301@gmail.com>

Curt Hagenlocher wrote:
> If I recall correctly, Jython handles this by appending a trailing
> underscore to the imported name and there's no reason why we couldn't
> do something similar.

It also has the virtue of being the common convention for attribute 
names that shadow keywords even in CPython (e.g. unittest.TestCase.assert_).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From pje at telecommunity.com  Thu Jun 12 12:06:32 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 12 Jun 2008 06:06:32 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.co
 m>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
	<693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>
Message-ID: <20080612100603.7A8203A40B0@sparrow.telecommunity.com>

At 02:59 AM 6/12/2008 +0200, Maciej Fijalkowski wrote:
>It's about abusing locals, which are not even given that they'll
>modify this dict.

Note that class bodies are a special case: as of PEP 3115, it's 
possible for a class body's locals to be a non-dictionary object, so 
it makes no sense to make a class body's locals() or f_locals return 
some *other* object.

Meanwhile, as a practicality-beats-purity matter, you're going to run 
into compatibility problems with a fair number of libraries in the 
field (including Zope and Twisted, and anything using them) if you 
*don't* support locals() modification in class bodies.  (Those other 
libraries don't use non-string keys like AddOns does, but they *do* 
depend on modifying a class body's frame locals.)


From ncoghlan at gmail.com  Thu Jun 12 12:35:40 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Jun 2008 20:35:40 +1000
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
	<ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>
Message-ID: <4850FBFC.9070501@gmail.com>

Guido van Rossum wrote:
> On Wed, Jun 11, 2008 at 4:35 AM, Barry Warsaw <barry at python.org> wrote:
>> So I had planned to do a bunch of work last night looking at the release
>> blocker issues, but nature intervened.  A bunch of severe thunderstorms
>> knock out my 'net access until this morning.
>>
>> I'll try to find some time during the day to look at the RB issues.
>>  Hopefully we can get Guido to look at them too and Pronounce on some of
>> them.  Guido please start with:
>>
>> http://bugs.python.org/issue643841
> 
> I've added a comment. Let me know if anything I said is unclear.

The bugtracker seems to be offline atm - I'll reply there once I can get 
to it again (as well as switching this issue back to being a 
documentation one).

I don't think we're going to see a major clamor for a value-based 
delegation mixin in the standard library until people using classic 
classes for value-based delegation start making serious attempts to port 
to Py3k (where that approach is no longer available). At the moment, 
such classes only need to care about the methods they want to fiddle 
with, leaving everything else to __getattr__ based delegation.

I've pushed as hard as I'm personally willing to for this without 
convincing anyone else that it's worth doing, so I'll start working on a 
documentation patch for the language reference instead which explicitly 
splits the special methods into the following categories:

1. Method lookup MUST bypass __getattribute__, shadowing the attribute 
in the instance dictionary MUST NOT have any ill effects. (All tp_* 
C-level slots and slots looked up via _PyType_Lookup will fit into this 
category)

2. Method lookup MAY bypass __getattribute__, shadowing the attribute in 
the instance dictionary MAY have ill effects. (slots such as __enter__ 
and __exit__ that are looked up via normal attribute lookup in CPython 
will fit into this category)

3. Technically a subcategory of group 1, these are special methods which 
can affect the interpreter's behaviour by their mere definition on a 
type. (The __get__, __set__ and __delete__ descriptor protocol methods 
fall into this category)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From antonxx at gmx.de  Thu Jun 12 11:48:40 2008
From: antonxx at gmx.de (anton)
Date: Thu, 12 Jun 2008 09:48:40 +0000 (UTC)
Subject: [Python-Dev] bugs.python.org down
Message-ID: <loom.20080612T093819-24@post.gmane.org>

Hi,

could someone look for this ... or I am the only one which noticed that.

ping works but http://bugs.python.org giges after a timeout:


Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /.

Reason: Error reading from remote server

Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 mod_ssl/2.2.3
OpenSSL/0.9.8c Server at bugs.python.org Port 80



From jnoller at gmail.com  Thu Jun 12 13:08:52 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 12 Jun 2008 07:08:52 -0400
Subject: [Python-Dev] [Python-3000] No betas today
In-Reply-To: <3F811617-7E55-4290-BB1E-14229E563C5F@python.org>
References: <3F811617-7E55-4290-BB1E-14229E563C5F@python.org>
Message-ID: <5F4CD1A3-867F-4E37-863A-C2B9324F02DD@gmail.com>

That's odd - I've been doing all of the mp work on osx 10.5.3 - I'll  
run the units in a loop today to see if I can catch it.
-Jesse

On Jun 12, 2008, at 1:00 AM, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Much thanks to Benjamin Peterson and Alexandre Vassalotti for  
> providing much help today trying to get the betas out.   
> Unfortunately, it's not gonna happen today.  There are two issues  
> blocking the release.
>
> http://bugs.python.org/issue3088
> http://bugs.python.org/issue3089
>
> The main problem (issue 3089) is that none of the Python 3.0  
> buildbots are green.  Issue 3088 may only happen for me, but  
> test_multiprocessing simply hangs for me on OS X 10.5.3.
>
> I'm going to bed now.  As soon as we can turn the bots green we can  
> try again.  This weekend might be a good time for me.
>
> Please help out by resolving these two issues.
>
> - -Barry
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
>
> iQCVAwUBSFCtZHEjvBPtnXfVAQLrHQQAoq1RCNeawpJHaefuvmZiS4VZzzvtV5Bm
> HFW6FWyO86NNEcCK6Delthf+H1mgjCaoymHojxBJhWm7UJK0WjMz+0RpF8zcLo0R
> i8I8SiFLy6kRB5UxlZHS/VXDi8QS92SogRvlnWLBwXNNK6wlgeqK8C0zt1ilL0q6
> Fqk/AsGO/fA=
> =HlEA
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/jnoller%40gmail.com

From hodgestar+pythondev at gmail.com  Thu Jun 12 13:31:43 2008
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 12 Jun 2008 13:31:43 +0200
Subject: [Python-Dev] bugs.python.org down
In-Reply-To: <loom.20080612T093819-24@post.gmane.org>
References: <loom.20080612T093819-24@post.gmane.org>
Message-ID: <fb73205e0806120431w26745c10qa78005b09a2fc1f1@mail.gmail.com>

On Thu, Jun 12, 2008 at 11:48 AM, anton <antonxx at gmx.de> wrote:
> ping works but http://bugs.python.org giges after a timeout:

Seeing the same thing here, so it's not just you.

Schiavo
Simon

From barry at python.org  Thu Jun 12 13:33:13 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Jun 2008 07:33:13 -0400
Subject: [Python-Dev] bugs.python.org down
In-Reply-To: <loom.20080612T093819-24@post.gmane.org>
References: <loom.20080612T093819-24@post.gmane.org>
Message-ID: <E1923715-8CB5-4543-BFEC-99D2124A7A60@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 12, 2008, at 5:48 AM, anton wrote:

> could someone look for this ... or I am the only one which noticed  
> that.
>
> ping works but http://bugs.python.org giges after a timeout:
>
>
> Proxy Error
>
> The proxy server received an invalid response from an upstream server.
> The proxy server could not handle the request GET /.
>
> Reason: Error reading from remote server
>
> Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 mod_ssl/2.2.3
> OpenSSL/0.9.8c Server at bugs.python.org Port 80

I can't even ssh into the machine this morning, so I've forwarded this  
off to the python.org sysadmins.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFEJenEjvBPtnXfVAQL1DwP/cpAACpa7KTL8BZcuQGcqgmbez1PmtXBG
jl85PaULlm6LAvTjtZMy/n2ribZ7TTGLRAHkEoGiQyN03xHFn+w8v6XpXwmvRMsw
19oLBxc3zOptNSfQwSUwdiSHwAdTPW9XjoR0xf7TwOQJZQ5PFtCPniQT2YxVnj4X
ZD/VmrIasUo=
=Jr3d
-----END PGP SIGNATURE-----

From cfbolz at gmx.de  Thu Jun 12 13:34:44 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 12 Jun 2008 13:34:44 +0200
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <20080612100048.7EEEA3A405F@sparrow.telecommunity.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>	<484F2C1B.2090404@scottdial.com>	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>	<20080611161455.CBD703A405F@sparrow.telecommunity.com>	<4850263A.3040805@scottdial.com>
	<g2pqrc$pel$1@ger.gmane.org>
	<20080612100048.7EEEA3A405F@sparrow.telecommunity.com>
Message-ID: <485109D4.2070407@gmx.de>

Phillip J. Eby wrote:
> At 08:32 PM 6/11/2008 -0400, Terry Reedy wrote:
>> The Data Model chapter of the Reference Manual lists .__dict__ as a
>> special
>> attribute of callables, modules, classes, and instances.  It describes
>> __dict__ as a "namespace dictionary" or "implementation of the namespace"
>> thereof.  Since namespaces map names (or possibly non-name strings) to
>> objects, this to me implies that an implementation is and should not be
>> required to allow non-strings in __dict__.
>>
>> The same chapter has more than one sentence saying something like "o.x is
>> equivalent to o.__dict__['x']".  While one could read this as prohibiting
>> o.__dict__[non_string], one could also read it as being silent, neither
>> allowing nor prohibiting.
> 
> As it happens, most objects' __dict__ slots are settable by default, and
> *require* that you set it to a dict or subclass thereof.

This is wrong for types:

$ python
Python 2.5.2 (r252:60911, Apr 21 2008, 11:12:42)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class A(object): pass
...
>>> A.__dict__ = {}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: attribute '__dict__' of 'type' objects is not writable

In fact it seems to me that there is no way to set non-string keys into
a type dict after class creation: Cls.__dict__ supports no setitem,
setattr checks for a string argument.

I think there are good arguments for not allowing strings keys in type
dicts, or at least leaving it up to the implementation. Using non-string
keys in type dicts is relatively awkward and allowing them makes many
interesting optimizations (like method caches) a _lot_ harder to get right.

Cheers,

Carl Friedrich Bolz


From barry at python.org  Thu Jun 12 13:35:15 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Jun 2008 07:35:15 -0400
Subject: [Python-Dev] [Python-3000] No betas today
In-Reply-To: <5F4CD1A3-867F-4E37-863A-C2B9324F02DD@gmail.com>
References: <3F811617-7E55-4290-BB1E-14229E563C5F@python.org>
	<5F4CD1A3-867F-4E37-863A-C2B9324F02DD@gmail.com>
Message-ID: <217F21B5-0747-46A4-8B44-11FA81D83F68@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 12, 2008, at 7:08 AM, Jesse Noller wrote:

> That's odd - I've been doing all of the mp work on osx 10.5.3 - I'll  
> run the units in a loop today to see if I can catch it.
> -Jesse

Thanks.  It's definitely reproducible on both my Leopard boxes.  I  
haven't tried it on my Tiger box.  The buildbots seem unhappy too.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFEJ9HEjvBPtnXfVAQJlNwQAsG2M/hNORYSuCsmKHU+nvbX3cASiFlhj
IIhNfdrtkRbjXBaDgxrr/Ps1+Q3I56lpSg5SDEWxXVufNdBdHXGAKKxnrYN22a1x
Nwuu+DXOGYmzBcQc2BS5L9/31yeDmSFBMP257BBstCZww97pt8VRLKpUL2Wzfzny
MZb6Km/NXwQ=
=Ab3l
-----END PGP SIGNATURE-----

From cfbolz at gmx.de  Thu Jun 12 13:34:44 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 12 Jun 2008 13:34:44 +0200
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <20080612100048.7EEEA3A405F@sparrow.telecommunity.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>	<484F2C1B.2090404@scottdial.com>	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>	<20080611161455.CBD703A405F@sparrow.telecommunity.com>	<4850263A.3040805@scottdial.com>
	<g2pqrc$pel$1@ger.gmane.org>
	<20080612100048.7EEEA3A405F@sparrow.telecommunity.com>
Message-ID: <485109D4.2070407@gmx.de>

Phillip J. Eby wrote:
> At 08:32 PM 6/11/2008 -0400, Terry Reedy wrote:
>> The Data Model chapter of the Reference Manual lists .__dict__ as a
>> special
>> attribute of callables, modules, classes, and instances.  It describes
>> __dict__ as a "namespace dictionary" or "implementation of the namespace"
>> thereof.  Since namespaces map names (or possibly non-name strings) to
>> objects, this to me implies that an implementation is and should not be
>> required to allow non-strings in __dict__.
>>
>> The same chapter has more than one sentence saying something like "o.x is
>> equivalent to o.__dict__['x']".  While one could read this as prohibiting
>> o.__dict__[non_string], one could also read it as being silent, neither
>> allowing nor prohibiting.
> 
> As it happens, most objects' __dict__ slots are settable by default, and
> *require* that you set it to a dict or subclass thereof.

This is wrong for types:

$ python
Python 2.5.2 (r252:60911, Apr 21 2008, 11:12:42)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class A(object): pass
...
>>> A.__dict__ = {}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: attribute '__dict__' of 'type' objects is not writable

In fact it seems to me that there is no way to set non-string keys into
a type dict after class creation: Cls.__dict__ supports no setitem,
setattr checks for a string argument.

I think there are good arguments for not allowing strings keys in type
dicts, or at least leaving it up to the implementation. Using non-string
keys in type dicts is relatively awkward and allowing them makes many
interesting optimizations (like method caches) a _lot_ harder to get right.

Cheers,

Carl Friedrich Bolz

From barry at python.org  Thu Jun 12 13:49:42 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Jun 2008 07:49:42 -0400
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
Message-ID: <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 11, 2008, at 2:03 PM, Raymond Hettinger wrote:

> "Function names should be lowercase, with words separated by  
> underscores as necessary to improve readability." -- PEP 8
>
> If I'm reading this correctly, then underscores are not required   
> everywhere.  Can some of these be shortened?
>
>   function:: active_count()
>   method:: Thread.get_name()
>   method:: Thread.is_alive()
>   method:: Thread.is_daemon()
>   method:: Thread.set_daemon(daemonic)

+1 on opting for properties in the specific cases here where it makes  
sense.

> In some cases, the mental pronounciation changes and affects my  
> perception of meaning.  For example, Thread.setName or  
> Thread.setname both feel like a setter to me, but Thread.set_name  
> causes a mental pause and a momentary double-take (is it the name of  
> a set?).
>
> A few months ago, I think there was a PEP 8 discussion rejecting  
> suggestions to make underscores required everywhere (leading to  
> getattr-->get_attr, iteritems-->iter_items,  staticmethod- 
> >static_method, setdefault->set_default, popitem->pop_item,  
> splitlines->split_lines etc.)
>
> Perhaps underscores should only be used when the contracted form  
> lacks clarity.

There are some builtins that I wish had used underscores to make them  
easier to read.   isinstance() and issubclass() come to mind.

OTOH, getattr and iteritems are already contractions, so leaving them  
underscoreless doesn't bother me too much.  For the others you  
mention, well I'll invoke ZoP 12 and 13.

There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFENVnEjvBPtnXfVAQIVhAQAn6liMZxAFKPmfvHp7UQLBdCEiI0rkGjR
OCxaEQLxpeuHfDGMS47kVJhyAwo3skKzv9Yv1+Vuor7SaP9hAO3h4r3SO2wM9jDF
jwrx6ajm9wtsTzSrh9QkYzOyegeMAfj3p8gLZ+eHiRdFHXj++biunvnn4GPl7O5x
E84j8rNlWkU=
=LYDl
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Thu Jun 12 14:25:02 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 12 Jun 2008 08:25:02 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <485109D4.2070407@gmx.de>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
	<20080612100048.7EEEA3A405F@sparrow.telecommunity.com>
	<485109D4.2070407@gmx.de>
Message-ID: <20080612122436.4D18F3A405F@sparrow.telecommunity.com>

At 01:34 PM 6/12/2008 +0200, Carl Friedrich Bolz wrote:
>Phillip J. Eby wrote:
> > As it happens, most objects' __dict__ slots are settable by default, and
> > *require* that you set it to a dict or subclass thereof.
>
>This is wrong for types:

Which is why I said "most" - to exclude types, and objects that don't 
have a __dict__ slot to begin with.


>I think there are good arguments for not allowing strings keys in type
>dicts, or at least leaving it up to the implementation.

That may well be, but there is nothing in Python's spec that I'm 
aware of that *forbids* it.  For example the type() constructor doc 
doesn't say anything about using string-only keys in the class dictionary.


>Using non-string
>keys in type dicts is relatively awkward and allowing them makes many
>interesting optimizations (like method caches) a _lot_ harder to get right.

Really?  Why?  Having non-string dict keys is NOT the same thing as 
having non-string attribute names, so attribute name lookups should 
be unaffected.  (Attribute names *are* required to be strings, and -- 
as far as I know -- always have been.)


From facundobatista at gmail.com  Thu Jun 12 14:29:24 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 12 Jun 2008 09:29:24 -0300
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
Message-ID: <e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>

2008/6/12 Barry Warsaw <barry at python.org>:

>>  function:: active_count()
>>  method:: Thread.get_name()
>>  method:: Thread.is_alive()
>>  method:: Thread.is_daemon()
>>  method:: Thread.set_daemon(daemonic)
>
> +1 on opting for properties in the specific cases here where it makes sense.

I'm +1 too... but which is the normal procedure here?

Should it be...

  2.n : .is_alive()
  2.n+1 : .is_alive() (deprecated), .alive (recommended)
  2.n+2 : .alive

...?

Regards,

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From facundobatista at gmail.com  Thu Jun 12 14:34:49 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 12 Jun 2008 09:34:49 -0300
Subject: [Python-Dev] bugs.python.org down
In-Reply-To: <loom.20080612T093819-24@post.gmane.org>
References: <loom.20080612T093819-24@post.gmane.org>
Message-ID: <e04bdf310806120534g675cf70end0efcd40e3d981b1@mail.gmail.com>

2008/6/12 anton <antonxx at gmx.de>:

> ping works but http://bugs.python.org giges after a timeout:

I can enter into it.

Also note that my issues showing system [1] was updated 6.5 hours ago,
so it was up at that time (my system goes and get some info twice a
day, you have the "last updated" info down on that page).

Regards,

[1] http://www.taniquetil.com.ar/cgi-bin/pytickets.py

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From musiccomposition at gmail.com  Thu Jun 12 14:54:02 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 12 Jun 2008 07:54:02 -0500
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
Message-ID: <1afaf6160806120554p1e1d617fidcdecee16a0c63b7@mail.gmail.com>

On Thu, Jun 12, 2008 at 7:29 AM, Facundo Batista
<facundobatista at gmail.com> wrote:
>> +1 on opting for properties in the specific cases here where it makes sense.

If you can get Guido to agree to it, I'll implement it.

>
> I'm +1 too... but which is the normal procedure here?
>
> Should it be...
>
>  2.n : .is_alive()
>  2.n+1 : .is_alive() (deprecated), .alive (recommended)
>  2.n+2 : .alive

I think:
2.x isAlive() (Py3kWarning), .alive (recommended for compatibility with 3.x)
3.x .alive


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From cfbolz at gmx.de  Thu Jun 12 14:56:46 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 12 Jun 2008 14:56:46 +0200
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <20080612122436.4D18F3A405F@sparrow.telecommunity.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>	<484F2C1B.2090404@scottdial.com>	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>	<20080611161455.CBD703A405F@sparrow.telecommunity.com>	<4850263A.3040805@scottdial.com>
	<g2pqrc$pel$1@ger.gmane.org>	<20080612100048.7EEEA3A405F@sparrow.telecommunity.com>	<485109D4.2070407@gmx.de>
	<20080612122436.4D18F3A405F@sparrow.telecommunity.com>
Message-ID: <48511D0E.9050309@gmx.de>

Phillip J. Eby wrote:
> At 01:34 PM 6/12/2008 +0200, Carl Friedrich Bolz wrote:
>> Phillip J. Eby wrote:
>> > As it happens, most objects' __dict__ slots are settable by default,
>> and
>> > *require* that you set it to a dict or subclass thereof.
>>
>> This is wrong for types:
> 
> Which is why I said "most" - to exclude types, and objects that don't
> have a __dict__ slot to begin with.

Sorry, I thought we were mostly discussing type dictionaries at the moment.

>> I think there are good arguments for not allowing strings keys in type
>> dicts, or at least leaving it up to the implementation.
> 
> That may well be, but there is nothing in Python's spec that I'm aware
> of that *forbids* it.  For example the type() constructor doc doesn't
> say anything about using string-only keys in the class dictionary.

Fine, but as far as I can see there is also nothing that *mandates* it.

>> Using non-string
>> keys in type dicts is relatively awkward and allowing them makes many
>> interesting optimizations (like method caches) a _lot_ harder to get
>> right.
> 
> Really?  Why?  Having non-string dict keys is NOT the same thing as
> having non-string attribute names, so attribute name lookups should be
> unaffected.  (Attribute names *are* required to be strings, and -- as
> far as I know -- always have been.)

Of course attribute name lookups are affected, because you can have a
non-string key that has a __hash__ and __eq__ method to make it look
sufficiently like a string to the dict. Then the attribute lookup needs
to take these into account. This makes a method cache implementation
much more complex, because suddenly arbitrary user code can run during
the lookup and your classes (and thus your method caches) can change
under your feed during the lookup. In this situation getting the corner
cases right is very hard.

Cheers,

Carl Friedrich


From barry at python.org  Thu Jun 12 15:04:14 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Jun 2008 09:04:14 -0400
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
Message-ID: <367889A5-6BD0-4106-948D-B5B181D4466D@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 12, 2008, at 8:29 AM, Facundo Batista wrote:

> 2008/6/12 Barry Warsaw <barry at python.org>:
>
>>> function:: active_count()
>>> method:: Thread.get_name()
>>> method:: Thread.is_alive()
>>> method:: Thread.is_daemon()
>>> method:: Thread.set_daemon(daemonic)
>>
>> +1 on opting for properties in the specific cases here where it  
>> makes sense.
>
> I'm +1 too... but which is the normal procedure here?
>
> Should it be...
>
>  2.n : .is_alive()
>  2.n+1 : .is_alive() (deprecated), .alive (recommended)
>  2.n+2 : .alive

Personally, I'd go with a property .is_alive

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFEeznEjvBPtnXfVAQKEeQP8CdF+N1wX4Qwxn7iglGYq0peZEYagn4JG
dOkP+TOkAzSciNfIotPaHJTGbyXsFtxLe3Rpq8r56/sPhHKS66+vCMojLBK64Iue
7/PDhZ300KRpPtbJOOA4OmqI2rz1fO+vflavICZlx7oIwC25L7dQSteu/NBJYGJN
QX/Z8WutBng=
=1B3v
-----END PGP SIGNATURE-----

From cfbolz at gmx.de  Thu Jun 12 14:56:46 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 12 Jun 2008 14:56:46 +0200
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <20080612122436.4D18F3A405F@sparrow.telecommunity.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>	<484F2C1B.2090404@scottdial.com>	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>	<20080611161455.CBD703A405F@sparrow.telecommunity.com>	<4850263A.3040805@scottdial.com>
	<g2pqrc$pel$1@ger.gmane.org>	<20080612100048.7EEEA3A405F@sparrow.telecommunity.com>	<485109D4.2070407@gmx.de>
	<20080612122436.4D18F3A405F@sparrow.telecommunity.com>
Message-ID: <48511D0E.9050309@gmx.de>

Phillip J. Eby wrote:
> At 01:34 PM 6/12/2008 +0200, Carl Friedrich Bolz wrote:
>> Phillip J. Eby wrote:
>> > As it happens, most objects' __dict__ slots are settable by default,
>> and
>> > *require* that you set it to a dict or subclass thereof.
>>
>> This is wrong for types:
> 
> Which is why I said "most" - to exclude types, and objects that don't
> have a __dict__ slot to begin with.

Sorry, I thought we were mostly discussing type dictionaries at the moment.

>> I think there are good arguments for not allowing strings keys in type
>> dicts, or at least leaving it up to the implementation.
> 
> That may well be, but there is nothing in Python's spec that I'm aware
> of that *forbids* it.  For example the type() constructor doc doesn't
> say anything about using string-only keys in the class dictionary.

Fine, but as far as I can see there is also nothing that *mandates* it.

>> Using non-string
>> keys in type dicts is relatively awkward and allowing them makes many
>> interesting optimizations (like method caches) a _lot_ harder to get
>> right.
> 
> Really?  Why?  Having non-string dict keys is NOT the same thing as
> having non-string attribute names, so attribute name lookups should be
> unaffected.  (Attribute names *are* required to be strings, and -- as
> far as I know -- always have been.)

Of course attribute name lookups are affected, because you can have a
non-string key that has a __hash__ and __eq__ method to make it look
sufficiently like a string to the dict. Then the attribute lookup needs
to take these into account. This makes a method cache implementation
much more complex, because suddenly arbitrary user code can run during
the lookup and your classes (and thus your method caches) can change
under your feed during the lookup. In this situation getting the corner
cases right is very hard.

Cheers,

Carl Friedrich

From metawilm at gmail.com  Thu Jun 12 15:47:59 2008
From: metawilm at gmail.com (Willem Broekema)
Date: Thu, 12 Jun 2008 15:47:59 +0200
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <48511D0E.9050309@gmx.de>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
	<20080612100048.7EEEA3A405F@sparrow.telecommunity.com>
	<485109D4.2070407@gmx.de>
	<20080612122436.4D18F3A405F@sparrow.telecommunity.com>
	<48511D0E.9050309@gmx.de>
Message-ID: <f6bc9b490806120647v2f7917f3r2ef9ef0be9e34557@mail.gmail.com>

On Thu, Jun 12, 2008 at 2:56 PM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
> Of course attribute name lookups are affected, because you can have a
> non-string key that has a __hash__ and __eq__ method to make it look
> sufficiently like a string to the dict. Then the attribute lookup needs
> to take these into account. This makes a method cache implementation
> much more complex, because suddenly arbitrary user code can run during
> the lookup and your classes (and thus your method caches) can change
> under your feed during the lookup. In this situation getting the corner
> cases right is very hard.

I fully agree with this assessment: the theoretical possibilities make
attribute handling very hairy.
Note that besides non-strings, also user-defined subclasses of string
can have their own hash/eq logic.

If namespace keys and attributes could be locked down to only be
regular strings, not of another type and not a string subclass
instance, then implementing attributes becomes a lot less hairy, and
optimizations much more straightforward.

- Willem (CLPython)

From guido at python.org  Thu Jun 12 15:55:07 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2008 06:55:07 -0700
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <4850FBFC.9070501@gmail.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
	<ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>
	<4850FBFC.9070501@gmail.com>
Message-ID: <ca471dc20806120655s60e378bj8850b37c676b313b@mail.gmail.com>

[Barry]
>>> http://bugs.python.org/issue643841

[Guido]
>> I've added a comment. Let me know if anything I said is unclear.

On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> The bugtracker seems to be offline atm - I'll reply there once I can get to
> it again (as well as switching this issue back to being a documentation
> one).
>
> I don't think we're going to see a major clamor for a value-based delegation
> mixin in the standard library until people using classic classes for
> value-based delegation start making serious attempts to port to Py3k (where
> that approach is no longer available). At the moment, such classes only need
> to care about the methods they want to fiddle with, leaving everything else
> to __getattr__ based delegation.

Whether they'll care about this issue of course depends on whether
they need overloaded operators and other special delegations to be
delegated transparently. We'll have to see how important this is.
New-style classes have been around and recommended for a long time --
why haven't people pushed for a proxy class before?

> I've pushed as hard as I'm personally willing to for this without convincing
> anyone else that it's worth doing,

What does *it* refer to? Changing the behavior, or adding a proxy
class to the stdlib? I'm -1000 on the former, but only -0 on the
latter -- as I wrote in the tracker, I just don't want to see an
unproven proxy class (and I don't like the module name).

> so I'll start working on a documentation
> patch for the language reference instead which explicitly splits the special
> methods into the following categories:

Thanks for doing this, it is needed regardless!

> 1. Method lookup MUST bypass __getattribute__, shadowing the attribute in
> the instance dictionary MUST NOT have any ill effects. (All tp_* C-level
> slots and slots looked up via _PyType_Lookup will fit into this category)

Watch out: I think the term "method lookup" may be confusing here.
Certainly when the user writes x.__foo__(), the instance dict *is*
consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where
the instance dict is bypassed.

> 2. Method lookup MAY bypass __getattribute__, shadowing the attribute in the
> instance dictionary MAY have ill effects. (slots such as __enter__ and
> __exit__ that are looked up via normal attribute lookup in CPython will fit
> into this category)

Why even have a  MAY category? Are you expecting these to become tp_
slots at some point?

> 3. Technically a subcategory of group 1, these are special methods which can
> affect the interpreter's behaviour by their mere definition on a type. (The
> __get__, __set__ and __delete__ descriptor protocol methods fall into this
> category)

I don't follow why this is relevant. This is a different, AFAIK
orthogonal issue, used in many places: *if* an object used in a
certain context has a specific attribute, *then* that attribute is
used, *otherwise* a default action is taken. Applies to __repr__ just
as much. These belong in category 1 if and only if the lookup bypasses
the instance dict.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jun 12 16:17:42 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2008 07:17:42 -0700
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
	<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
Message-ID: <ca471dc20806120717v1916a5f6of1a01b64bb848a98@mail.gmail.com>

If you want to have a meaningful discussion about this, the addition
of the multiprocessing package and the recent threading.py API changes
must be rolled back, so we can design a proper API without the beta 1
pressure. Some observations:

- If it's isAlive() in Java style, it should be is_alive in Python style.

- I like *methods* (and functions) named isfoo or is_foo, but I like
the corresponding properties to be called just foo.

- I think properties instead of simple setters and getters is great
(as long as the setters implied in the property enforce the
conditions).

- I hate typing underscores, since they require using the shift key
*and* moving my hands away from the home position on the keybord.

--Guido

On Thu, Jun 12, 2008 at 6:04 AM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Jun 12, 2008, at 8:29 AM, Facundo Batista wrote:
>
>> 2008/6/12 Barry Warsaw <barry at python.org>:
>>
>>>> function:: active_count()
>>>> method:: Thread.get_name()
>>>> method:: Thread.is_alive()
>>>> method:: Thread.is_daemon()
>>>> method:: Thread.set_daemon(daemonic)
>>>
>>> +1 on opting for properties in the specific cases here where it makes
>>> sense.
>>
>> I'm +1 too... but which is the normal procedure here?
>>
>> Should it be...
>>
>>  2.n : .is_alive()
>>  2.n+1 : .is_alive() (deprecated), .alive (recommended)
>>  2.n+2 : .alive
>
> Personally, I'd go with a property .is_alive
>
> - -Barry
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
>
> iQCVAwUBSFEeznEjvBPtnXfVAQKEeQP8CdF+N1wX4Qwxn7iglGYq0peZEYagn4JG
> dOkP+TOkAzSciNfIotPaHJTGbyXsFtxLe3Rpq8r56/sPhHKS66+vCMojLBK64Iue
> 7/PDhZ300KRpPtbJOOA4OmqI2rz1fO+vflavICZlx7oIwC25L7dQSteu/NBJYGJN
> QX/Z8WutBng=
> =1B3v
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fuzzyman at voidspace.org.uk  Thu Jun 12 16:43:02 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Thu, 12 Jun 2008 14:43:02 -0000
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <ca471dc20806120655s60e378bj8850b37c676b313b@mail.gmail.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>	<4850FBFC.9070501@gmail.com>
	<ca471dc20806120655s60e378bj8850b37c676b313b@mail.gmail.com>
Message-ID: <48EE15E7.7020701@voidspace.org.uk>

Guido van Rossum wrote:
> [Barry]
>   
>>>> http://bugs.python.org/issue643841
>>>>         
>
> [Guido]
>   
>>> I've added a comment. Let me know if anything I said is unclear.
>>>       
>
> On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>   
>> The bugtracker seems to be offline atm - I'll reply there once I can get to
>> it again (as well as switching this issue back to being a documentation
>> one).
>>
>> I don't think we're going to see a major clamor for a value-based delegation
>> mixin in the standard library until people using classic classes for
>> value-based delegation start making serious attempts to port to Py3k (where
>> that approach is no longer available). At the moment, such classes only need
>> to care about the methods they want to fiddle with, leaving everything else
>> to __getattr__ based delegation.
>>     
>
> Whether they'll care about this issue of course depends on whether
> they need overloaded operators and other special delegations to be
> delegated transparently. We'll have to see how important this is.
> New-style classes have been around and recommended for a long time --
> why haven't people pushed for a proxy class before?
>
>   

It's only in Python 3 that old style classes are going away fully, so up 
until now you could at least use a classic class to do the proxying.

I've written my own proxy classes before that look very similar to this, 
and there are other proxy classes around that do the same (I thought one 
was by Phillip J Eby but can't find a reference easily). The last one I 
wrote was to proxy CPython objects from IronPython via Python.NET...

I would prefer it if the proxy class wrapped the return values of 
inplace operations.

Michael Foord

>> I've pushed as hard as I'm personally willing to for this without convincing
>> anyone else that it's worth doing,
>>     
>
> What does *it* refer to? Changing the behavior, or adding a proxy
> class to the stdlib? I'm -1000 on the former, but only -0 on the
> latter -- as I wrote in the tracker, I just don't want to see an
> unproven proxy class (and I don't like the module name).
>
>   
>> so I'll start working on a documentation
>> patch for the language reference instead which explicitly splits the special
>> methods into the following categories:
>>     
>
> Thanks for doing this, it is needed regardless!
>
>   
>> 1. Method lookup MUST bypass __getattribute__, shadowing the attribute in
>> the instance dictionary MUST NOT have any ill effects. (All tp_* C-level
>> slots and slots looked up via _PyType_Lookup will fit into this category)
>>     
>
> Watch out: I think the term "method lookup" may be confusing here.
> Certainly when the user writes x.__foo__(), the instance dict *is*
> consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where
> the instance dict is bypassed.
>
>   
>> 2. Method lookup MAY bypass __getattribute__, shadowing the attribute in the
>> instance dictionary MAY have ill effects. (slots such as __enter__ and
>> __exit__ that are looked up via normal attribute lookup in CPython will fit
>> into this category)
>>     
>
> Why even have a  MAY category? Are you expecting these to become tp_
> slots at some point?
>
>   
>> 3. Technically a subcategory of group 1, these are special methods which can
>> affect the interpreter's behaviour by their mere definition on a type. (The
>> __get__, __set__ and __delete__ descriptor protocol methods fall into this
>> category)
>>     
>
> I don't follow why this is relevant. This is a different, AFAIK
> orthogonal issue, used in many places: *if* an object used in a
> certain context has a specific attribute, *then* that attribute is
> used, *otherwise* a default action is taken. Applies to __repr__ just
> as much. These belong in category 1 if and only if the lookup bypasses
> the instance dict.
>
>   


From ncoghlan at gmail.com  Thu Jun 12 16:50:36 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Jun 2008 00:50:36 +1000
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <ca471dc20806120655s60e378bj8850b37c676b313b@mail.gmail.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	
	<ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>	
	<4850FBFC.9070501@gmail.com>
	<ca471dc20806120655s60e378bj8850b37c676b313b@mail.gmail.com>
Message-ID: <485137BC.2070505@gmail.com>

Guido van Rossum wrote:
> [Barry]
>>>> http://bugs.python.org/issue643841
> 
> [Guido]
>>> I've added a comment. Let me know if anything I said is unclear.
> 
> On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> The bugtracker seems to be offline atm - I'll reply there once I can get to
>> it again (as well as switching this issue back to being a documentation
>> one).
>>
>> I don't think we're going to see a major clamor for a value-based delegation
>> mixin in the standard library until people using classic classes for
>> value-based delegation start making serious attempts to port to Py3k (where
>> that approach is no longer available). At the moment, such classes only need
>> to care about the methods they want to fiddle with, leaving everything else
>> to __getattr__ based delegation.
> 
> Whether they'll care about this issue of course depends on whether
> they need overloaded operators and other special delegations to be
> delegated transparently. We'll have to see how important this is.
> New-style classes have been around and recommended for a long time --
> why haven't people pushed for a proxy class before?

There was an easier way to do it in the form of classic classes - the 
2,x interpreter is riddled with special cases that ensure that 
__getattr__ is always consulted when looking for special methods on a 
classic class. The tracker issue regarding the fact that things aren't 
so simple with new style classes was actually raised way back in 2002 
when someone tried to port such a class to new-style and discovered that 
overriding __getattribute__ was no longer enough.

>> I've pushed as hard as I'm personally willing to for this without convincing
>> anyone else that it's worth doing,
> 
> What does *it* refer to? Changing the behavior, or adding a proxy
> class to the stdlib? I'm -1000 on the former, but only -0 on the
> latter -- as I wrote in the tracker, I just don't want to see an
> unproven proxy class (and I don't like the module name).

"It" referred to adding the proxy class - I'm personally ambivalent on 
adding it at this point, because the complexity of it reduces my 
confidence that I got it right, but it also makes it seem unfair to 
users of this feature of classic classes to take it away in 3.0 without 
giving them some kind of functional replacement.

As for as the module name goes, I don't particularly like it either - 
dropping something in the types module instead would be an alternative 
option.

>> so I'll start working on a documentation
>> patch for the language reference instead which explicitly splits the special
>> methods into the following categories:
> 
> Thanks for doing this, it is needed regardless!
> 
>> 1. Method lookup MUST bypass __getattribute__, shadowing the attribute in
>> the instance dictionary MUST NOT have any ill effects. (All tp_* C-level
>> slots and slots looked up via _PyType_Lookup will fit into this category)
> 
> Watch out: I think the term "method lookup" may be confusing here.
> Certainly when the user writes x.__foo__(), the instance dict *is*
> consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where
> the instance dict is bypassed.

Ah good point, I'll make sure to be careful with that.

>> 2. Method lookup MAY bypass __getattribute__, shadowing the attribute in the
>> instance dictionary MAY have ill effects. (slots such as __enter__ and
>> __exit__ that are looked up via normal attribute lookup in CPython will fit
>> into this category)
> 
> Why even have a  MAY category? Are you expecting these to become tp_
> slots at some point?

Either tp_* slots, or just having the invocation bypass the instance 
attributes and only look at the object's type.

I think it would actually be desirable for this category to be empty 
from a purity point of view (with all the special methods in category 
1), but given that CPython itself currently doesn't conform to such a 
language spec, this seems to be the next best option (i.e. allowing 
other implementations or later versions of CPython to put these special 
methods in category 1 along with the rest of the special methods)

>> 3. Technically a subcategory of group 1, these are special methods which can
>> affect the interpreter's behaviour by their mere definition on a type. (The
>> __get__, __set__ and __delete__ descriptor protocol methods fall into this
>> category)
> 
> I don't follow why this is relevant. This is a different, AFAIK
> orthogonal issue, used in many places: *if* an object used in a
> certain context has a specific attribute, *then* that attribute is
> used, *otherwise* a default action is taken. Applies to __repr__ just
> as much. These belong in category 1 if and only if the lookup bypasses
> the instance dict.

Actual hasattr() checks aren't a problem - those hit __getattribute__ 
and a delegating class can correctly check them against the target 
object. Methods like __str__ or __repr__ also aren't a major issue - 
those are easy to delegate in a way that reproduces the same behaviour 
as if the delegating class wasn't there (just reinvoke the appropriate 
builtin on your target object).

This category is specifically for method checks in category 1 which 
bypass __getattribute__ *and* have significant effects on the way an 
object gets handled that can't be readily dealt with by a value-based 
delegation class - the most significant methods I've actually found in 
that category so far are the descriptor protocol methods (that's why my 
ProxyMixin class skipped delegating them).

As long as the callable() builtin is around, __call__ actually lands in 
this category as well (since defining it will affect the answer returned 
by the callable() builtin). Being able to return different proxy classes 
with and without __callable__ defined is actually the reason 
weakref.proxy is a factory function rather than a type in its own right.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Thu Jun 12 16:54:08 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Jun 2008 00:54:08 +1000
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <48EE15E7.7020701@voidspace.org.uk>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>	<4850FBFC.9070501@gmail.com>
	<ca471dc20806120655s60e378bj8850b37c676b313b@mail.gmail.com>
	<48EE15E7.7020701@voidspace.org.uk>
Message-ID: <48513890.6080303@gmail.com>

Michael Foord wrote:
> Guido van Rossum wrote:
>> Whether they'll care about this issue of course depends on whether
>> they need overloaded operators and other special delegations to be
>> delegated transparently. We'll have to see how important this is.
>> New-style classes have been around and recommended for a long time --
>> why haven't people pushed for a proxy class before?
> 
> It's only in Python 3 that old style classes are going away fully, so up 
> until now you could at least use a classic class to do the proxying.
> 
> I've written my own proxy classes before that look very similar to this, 
> and there are other proxy classes around that do the same (I thought one 
> was by Phillip J Eby but can't find a reference easily). The last one I 
> wrote was to proxy CPython objects from IronPython via Python.NET...
> 
> I would prefer it if the proxy class wrapped the return values of 
> inplace operations.

Yeah, the latest version on the issue tracker does that, and allows 
subclasses to define a return_inplace() method to alter the behaviour 
(e.g. a weakref.proxy equivalent wouldn't want to wrap the return values 
so that it can ensure there is always at least one strong reference to 
the result of the operation).

Since you can also replace the .target attribute with a property to 
affect how the target object is stored and accessed, it's a reasonably 
flexible approach.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From walter at livinglogic.de  Thu Jun 12 16:59:54 2008
From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Thu, 12 Jun 2008 16:59:54 +0200
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <485003F8.40803@egenix.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<484FE1E9.2080904@egenix.com>	<484FEC2C.6070409@livinglogic.de>
	<485003F8.40803@egenix.com>
Message-ID: <485139EA.1000502@livinglogic.de>

M.-A. Lemburg wrote:
> On 2008-06-11 17:15, Walter D?rwald wrote:
>> M.-A. Lemburg wrote:
>>> On 2008-06-11 13:35, Barry Warsaw wrote:
>>>> So I had planned to do a bunch of work last night looking at the 
>>>> release blocker issues, but nature intervened.  A bunch of severe 
>>>> thunderstorms knock out my 'net access until this morning.
>>>>
>>>> I'll try to find some time during the day to look at the RB issues.  
>>>> Hopefully we can get Guido to look at them too and Pronounce on some 
>>>> of them.  Guido please start with:
>>>>
>>>> http://bugs.python.org/issue643841
>>>>
>>>> My plan is to begin building the betas tonight, at around 9 or 10pm 
>>>> EDT (0100 to 0200 UTC Thursday).  If a showstopper comes up before 
>>>> then, I'll email the list.  If you think we really aren't ready for 
>>>> beta, then I would still like to get a release out today.  In that 
>>>> case, we'll call it alpha and delay the betas.
>>>
>>> There are two things I'd like to get in to 3.0:
>>>
>>>  * .transform()/.untransform() methods (this is mostly done, just need
>>>    to add the methods to PyUnicode, PyBytes and PyByteArray)
>>
>> What would these methods do? Use the codec machinery without any type 
>> checks?
> 
> As discussed in another thread some weeks ago:
> 
> .transform() and .untransform() use the codecs to apply same-type
> conversions. They do apply type checks to make sure that the
> codec does indeed return the same type.
> 
> E.g. text.transform('xml-escape') or data.transform('base64').

So what would a base64 codec do with the errors argument?

>> I think for transformations we don't need the full codec machinery:
>  > ...
> 
> No need to invent another wheel :-) The codecs already exist for
> Py2.x and can be used by the .encode()/.decode() methods in Py2.x
> (where no type checks occur).

By using a new API we could get rid of old warts. For example: Why does 
the stateless encoder/decoder return how many input characters/bytes it 
has consumed? It must consume *all* bytes anyway!

> In Py3.x, .encode()/.decode() only allow conversions of the type
> unicode <-> bytes. .transform()/.untransform() add conversions
> of the type unicode <-> unicode or bytes <-> bytes.
> 
> All other conversions in Py3.x have to go through codecs.encode() and
> codecs.decode() which are the generic codec access functions from
> the codec registry.

Servus,
    Walter


From skip at pobox.com  Thu Jun 12 17:21:05 2008
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 12 Jun 2008 10:21:05 -0500
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
	<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
Message-ID: <18513.16097.394123.312491@montanaro-dyndns-org.local>


    >> Should it be...
    >> 
    >> 2.n : .is_alive()
    >> 2.n+1 : .is_alive() (deprecated), .alive (recommended)
    >> 2.n+2 : .alive

    Barry> Personally, I'd go with a property .is_alive

I'm not fond of using a property for this since it can lull you into the
false belief that what you are doing is less expensive than it really is
(attribute access vs method call).  I think this is a case where explicit is
better than implicit.

Skip

From arigo at tunes.org  Thu Jun 12 17:24:43 2008
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 12 Jun 2008 17:24:43 +0200
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <48508D81.2050304@scottdial.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
	<693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>
	<48508D81.2050304@scottdial.com>
Message-ID: <20080612152443.GA22637@code0.codespeak.net>

Hi,

On Wed, Jun 11, 2008 at 10:44:17PM -0400, Scott Dial wrote:
> The only reason the test used locals() was
> because it was the only way to insert a non-string key into the class
> namespace.

This discussion is mistakenly focused on locals().  There is a direct
way to have arbitrary keys in the dict of a type:

>>> MyClass = type('MyClass', (Base,), {42: 64})
>>> MyClass.__dict__[42]
64

There is, however, no way to modify or add non-string keys in the type
after its creation.  So the question is whether the type() constructor
is allowed to fail with a TypeError when the initial dict contains
non-string keys (this is PyPy's current behavior).


A bientot,

Armin.

From barry at python.org  Thu Jun 12 17:40:26 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Jun 2008 11:40:26 -0400
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <18513.16097.394123.312491@montanaro-dyndns-org.local>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
	<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
	<18513.16097.394123.312491@montanaro-dyndns-org.local>
Message-ID: <A0BCACBC-B884-4752-9E9A-C03500C05298@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 12, 2008, at 11:21 AM, skip at pobox.com wrote:

>
>>> Should it be...
>>>
>>> 2.n : .is_alive()
>>> 2.n+1 : .is_alive() (deprecated), .alive (recommended)
>>> 2.n+2 : .alive
>
>    Barry> Personally, I'd go with a property .is_alive
>
> I'm not fond of using a property for this since it can lull you into  
> the
> false belief that what you are doing is less expensive than it  
> really is
> (attribute access vs method call).  I think this is a case where  
> explicit is
> better than implicit.

I've heard this argument before, and it's one I'm not unsympathetic  
to.  Usually I've heard it in the context of remote network access,  
e.g. let's say .alive had to make a REST call over the net each time.   
Well, maybe there's caching involved which amortizes the costs, so the  
argument gets trickier.

Ideally, I would like for those considerations to not enter into the  
API design.  I'd rather keep it clean, with sufficient documentation  
to give hints about any additional costs involved.  Logically .alive  
is a property, so I'd like to write it that way.

Perhaps function annotations help out here.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFFDanEjvBPtnXfVAQJxbAQAq2BDNE7m+J+5Jy3E4XlHaw8JXsEaukEA
YvZ1M1R1mREGCstIf4tlScKko1eu9PfJIk7+kCmFYezighJ1tPZunyu5zLemlgQe
9rJ9keFIJBLtao8yv+FPkn56idixtkTGq+14Ef+EhjIhOnbonp+eDuhScbE2tdzM
5tuFO/hsrqw=
=66R7
-----END PGP SIGNATURE-----

From mal at egenix.com  Thu Jun 12 17:43:36 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 12 Jun 2008 17:43:36 +0200
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <485139EA.1000502@livinglogic.de>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<484FE1E9.2080904@egenix.com>	<484FEC2C.6070409@livinglogic.de>	<485003F8.40803@egenix.com>
	<485139EA.1000502@livinglogic.de>
Message-ID: <48514428.7040001@egenix.com>

On 2008-06-12 16:59, Walter D?rwald wrote:
> M.-A. Lemburg wrote:
>> .transform() and .untransform() use the codecs to apply same-type
>> conversions. They do apply type checks to make sure that the
>> codec does indeed return the same type.
>>
>> E.g. text.transform('xml-escape') or data.transform('base64').
> 
> So what would a base64 codec do with the errors argument?

It could use it to e.g. try to recover as much data as possible
from broken input data.

Currently (in Py2.x), it raises an exception if you pass in anything
but "strict".

>>> I think for transformations we don't need the full codec machinery:
>>  > ...
>>
>> No need to invent another wheel :-) The codecs already exist for
>> Py2.x and can be used by the .encode()/.decode() methods in Py2.x
>> (where no type checks occur).
> 
> By using a new API we could get rid of old warts. For example: Why does 
> the stateless encoder/decoder return how many input characters/bytes it 
> has consumed? It must consume *all* bytes anyway!

No, it doesn't and that's the point in having those return values :-)

Even though the encoder/decoders are stateless, that doesn't mean
they have to consume all input data. The caller is responsible to
make sure that all input data was in fact consumed.

You could for example have a decoder that stops decoding after
having seen a block end indicator, e.g. a base64 line end or
XML closing element.

Just because all codecs that ship with Python always try to decode
the complete input doesn't mean that the feature isn't being used.
The interface was designed to allow for the above situations.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 12 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            24 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From adlaiff6 at gmail.com  Thu Jun 12 19:13:45 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Thu, 12 Jun 2008 10:13:45 -0700
Subject: [Python-Dev] Assignment to None
In-Reply-To: <4850F418.2040301@gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<484CBD70.3000808@v.loewis.de>
	<d2155e360806111426u16747c12r27b4239b5dfa481b@mail.gmail.com>
	<d2155e360806111427v234f1ad5yd9870ace7fc7b9b9@mail.gmail.com>
	<4850F418.2040301@gmail.com>
Message-ID: <cc7430500806121013je4140ege3988ab48e6b41d9@mail.gmail.com>

I haven't been following this thread very closely, so I'm not sure
what the status is, but I'd just like to point out that yesterday I
used the fact that a[None] = b works, when I used the @memoize
decorator from the wiki.  This seems to provide an argument that, for
symmetry's sake, we might want to keep a.None = b as well.

--
Cheers,
Leif

From guido at python.org  Thu Jun 12 19:28:18 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2008 10:28:18 -0700
Subject: [Python-Dev] Assignment to None
In-Reply-To: <cc7430500806121013je4140ege3988ab48e6b41d9@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<484CBD70.3000808@v.loewis.de>
	<d2155e360806111426u16747c12r27b4239b5dfa481b@mail.gmail.com>
	<d2155e360806111427v234f1ad5yd9870ace7fc7b9b9@mail.gmail.com>
	<4850F418.2040301@gmail.com>
	<cc7430500806121013je4140ege3988ab48e6b41d9@mail.gmail.com>
Message-ID: <ca471dc20806121028m589b4f47q3a9b14adc924922f@mail.gmail.com>

On Thu, Jun 12, 2008 at 10:13 AM, Leif Walsh <adlaiff6 at gmail.com> wrote:
> I haven't been following this thread very closely, so I'm not sure
> what the status is, but I'd just like to point out that yesterday I
> used the fact that a[None] = b works, when I used the @memoize
> decorator from the wiki.  This seems to provide an argument that, for
> symmetry's sake, we might want to keep a.None = b as well.

That makes about as much sense as wanting to support a.42 = b since
a[42] = b works. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From adlaiff6 at gmail.com  Thu Jun 12 19:55:38 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Thu, 12 Jun 2008 10:55:38 -0700
Subject: [Python-Dev] Assignment to None
In-Reply-To: <ca471dc20806121028m589b4f47q3a9b14adc924922f@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<484CBD70.3000808@v.loewis.de>
	<d2155e360806111426u16747c12r27b4239b5dfa481b@mail.gmail.com>
	<d2155e360806111427v234f1ad5yd9870ace7fc7b9b9@mail.gmail.com>
	<4850F418.2040301@gmail.com>
	<cc7430500806121013je4140ege3988ab48e6b41d9@mail.gmail.com>
	<ca471dc20806121028m589b4f47q3a9b14adc924922f@mail.gmail.com>
Message-ID: <cc7430500806121055y10944373u2649501fc5c4908c@mail.gmail.com>

On Thu, Jun 12, 2008 at 10:28 AM, Guido van Rossum <guido at python.org> wrote:
> That makes about as much sense as wanting to support a.42 = b since
> a[42] = b works. :-)

Well don't I feel silly now.

-- 
Cheers,
Leif

From jimjjewett at gmail.com  Thu Jun 12 20:11:14 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 12 Jun 2008 14:11:14 -0400
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <4850FBFC.9070501@gmail.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>
	<ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>
	<4850FBFC.9070501@gmail.com>
Message-ID: <fb6fbf560806121111y6a9de736ra69b669211b55ff3@mail.gmail.com>

On 6/12/08, Nick Coghlan <ncoghlan at gmail.com> wrote:

> documentation patch for the language reference ...
> following categories:
...

>  2. Method lookup MAY bypass __getattribute__, shadowing the attribute in
> the instance dictionary MAY have ill effects. (slots such as __enter__ and
> __exit__ that are looked up via normal attribute lookup in CPython will fit
> into this category)

Should this category really be enumerated?

I thought that was the default meaning of __name__, so the real
clarification is:

(1)  Requiring that the specific names in category 1 MUST be treated this way.

(2)  Mentioning __*__ and listing any known exceptions.  (Can "next"
be treated this way despite the lack of __*__?  Is it forbidden to
treat __context__ this way?)

-jJ

From tjreedy at udel.edu  Thu Jun 12 21:38:09 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 12 Jun 2008 15:38:09 -0400
Subject: [Python-Dev] bug or a feature?
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com><484F2C1B.2090404@scottdial.com><693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com><20080611161455.CBD703A405F@sparrow.telecommunity.com><4850263A.3040805@scottdial.com>
	<g2pqrc$pel$1@ger.gmane.org>
	<20080612100048.7EEEA3A405F@sparrow.telecommunity.com>
Message-ID: <g2rtv0$58p$1@ger.gmane.org>


"Phillip J. Eby" <pje at telecommunity.com> wrote in message 
news:20080612100048.7EEEA3A405F at sparrow.telecommunity.com...
| At 08:32 PM 6/11/2008 -0400, Terry Reedy wrote:
| >The Data Model chapter of the Reference Manual lists .__dict__ as a 
special
| >attribute of callables, modules, classes, and instances.  It describes
| >__dict__ as a "namespace dictionary" or "implementation of the 
namespace"
| >thereof.  Since namespaces map names (or possibly non-name strings) to
| >objects, this to me implies that an implementation is and should not be
| >required to allow non-strings in __dict__.
| >
| >The same chapter has more than one sentence saying something like "o.x 
is
| >equivalent to o.__dict__['x']".  While one could read this as 
prohibiting
| >o.__dict__[non_string], one could also read it as being silent, neither
| >allowing nor prohibiting.
|
| As it happens, most objects' __dict__ slots are settable by default,
| and *require* that you set it to a dict or subclass thereof.  This
| seems (to me) to imply that a standard dictionary (i.e. one
| supporting keys of any type) is required.  (In the sense that a dict
| subclass which rejects non-strings would be violating the Liskov 
principle.)

Is this requirement a documented Python requirement (not that I found), an 
undocumented Python requirement, or a CPython-specific requirement (that 
other implementations may freely ignore)?

I don't have much opinion on what the rules for __dict__ attributes should 
be, just that whatever they are  should be documented.  This include 
implementation-defined aspects.  This will help both implementors and users 
who wish to write Python code rather that CPython code.

If there is a consensus and BDFL pronouncement, I will make or help review 
suggested doc changes.

tjr




From guido at python.org  Thu Jun 12 21:46:18 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2008 12:46:18 -0700
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <20080612152443.GA22637@code0.codespeak.net>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
	<693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>
	<48508D81.2050304@scottdial.com>
	<20080612152443.GA22637@code0.codespeak.net>
Message-ID: <ca471dc20806121246l68e5392bl693adb97f6bffe04@mail.gmail.com>

On Thu, Jun 12, 2008 at 8:24 AM, Armin Rigo <arigo at tunes.org> wrote:
> This discussion is mistakenly focused on locals().  There is a direct
> way to have arbitrary keys in the dict of a type:
>
>>>> MyClass = type('MyClass', (Base,), {42: 64})
>>>> MyClass.__dict__[42]
> 64
>
> There is, however, no way to modify or add non-string keys in the type
> after its creation.  So the question is whether the type() constructor
> is allowed to fail with a TypeError when the initial dict contains
> non-string keys (this is PyPy's current behavior).

The intention was for these dicts to be used as namespaces. I think of
it as follows:

(a) Using non-string keys is a no-no, but the implementation isn't
required to go out of its way to forbid it.

(b) Using non-empty string keys that aren't well-formed identifiers
should be allowed.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From metawilm at gmail.com  Thu Jun 12 22:01:28 2008
From: metawilm at gmail.com (Willem Broekema)
Date: Thu, 12 Jun 2008 22:01:28 +0200
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <ca471dc20806121246l68e5392bl693adb97f6bffe04@mail.gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
	<693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>
	<48508D81.2050304@scottdial.com>
	<20080612152443.GA22637@code0.codespeak.net>
	<ca471dc20806121246l68e5392bl693adb97f6bffe04@mail.gmail.com>
Message-ID: <f6bc9b490806121301q54d36cb6ref2223b475293f30@mail.gmail.com>

On Thu, Jun 12, 2008 at 9:46 PM, Guido van Rossum <guido at python.org> wrote:
> The intention was for these dicts to be used as namespaces. I think of
> it as follows:
>
> (a) Using non-string keys is a no-no, but the implementation isn't
> required to go out of its way to forbid it.

That will allow easier and more efficient implementation, good!

> (b) Using non-empty string keys that aren't well-formed identifiers
> should be allowed.

ok.

Is it allowed to "normalize" subclasses of strings to regular string,
e.g. after:

  class mystring(str): pass
  class C: pass

  x = C()
  setattr(x, mystring('foo'), 42)

is it allowed that the dict of x contains a regular string 'foo'
instead of the mystring instance?

- Willem

From pje at telecommunity.com  Thu Jun 12 23:42:50 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 12 Jun 2008 17:42:50 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <ca471dc20806121246l68e5392bl693adb97f6bffe04@mail.gmail.co
 m>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F2C1B.2090404@scottdial.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
	<693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>
	<48508D81.2050304@scottdial.com>
	<20080612152443.GA22637@code0.codespeak.net>
	<ca471dc20806121246l68e5392bl693adb97f6bffe04@mail.gmail.com>
Message-ID: <20080612214219.3C37A3A405F@sparrow.telecommunity.com>

At 12:46 PM 6/12/2008 -0700, Guido van Rossum wrote:
>The intention was for these dicts to be used as namespaces.

By "these" do you mean type object __dict__ attributes, or *all* 
__dict__ attributes?


From guido at python.org  Fri Jun 13 00:05:27 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2008 15:05:27 -0700
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <f6bc9b490806121301q54d36cb6ref2223b475293f30@mail.gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
	<693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>
	<48508D81.2050304@scottdial.com>
	<20080612152443.GA22637@code0.codespeak.net>
	<ca471dc20806121246l68e5392bl693adb97f6bffe04@mail.gmail.com>
	<f6bc9b490806121301q54d36cb6ref2223b475293f30@mail.gmail.com>
Message-ID: <ca471dc20806121505k148aeed6qbbda76c08ee2cad8@mail.gmail.com>

On Thu, Jun 12, 2008 at 1:01 PM, Willem Broekema <metawilm at gmail.com> wrote:
> On Thu, Jun 12, 2008 at 9:46 PM, Guido van Rossum <guido at python.org> wrote:
>> The intention was for these dicts to be used as namespaces. I think of
>> it as follows:
>>
>> (a) Using non-string keys is a no-no, but the implementation isn't
>> required to go out of its way to forbid it.
>
> That will allow easier and more efficient implementation, good!
>
>> (b) Using non-empty string keys that aren't well-formed identifiers
>> should be allowed.
>
> ok.
>
> Is it allowed to "normalize" subclasses of strings to regular string,
> e.g. after:
>
>  class mystring(str): pass
>  class C: pass
>
>  x = C()
>  setattr(x, mystring('foo'), 42)
>
> is it allowed that the dict of x contains a regular string 'foo'
> instead of the mystring instance?

I think yes, as this would allow for a more efficient implementation
of the custom dict class.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun 13 00:07:37 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2008 15:07:37 -0700
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <20080612214219.3C37A3A405F@sparrow.telecommunity.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com>
	<20080611161455.CBD703A405F@sparrow.telecommunity.com>
	<4850263A.3040805@scottdial.com> <g2pqrc$pel$1@ger.gmane.org>
	<693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com>
	<48508D81.2050304@scottdial.com>
	<20080612152443.GA22637@code0.codespeak.net>
	<ca471dc20806121246l68e5392bl693adb97f6bffe04@mail.gmail.com>
	<20080612214219.3C37A3A405F@sparrow.telecommunity.com>
Message-ID: <ca471dc20806121507v79bf3824l93a7e3822abfc136@mail.gmail.com>

On Thu, Jun 12, 2008 at 2:42 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 12:46 PM 6/12/2008 -0700, Guido van Rossum wrote:
>>
>> The intention was for these dicts to be used as namespaces.
>
> By "these" do you mean type object __dict__ attributes, or *all* __dict__
> attributes?

Hadn't thought of that one! Since __dict__ is widely advertised as
being, well, a dict, it may be harder to enforce this for __dict__
instances of other objects.  I'd like to tighten the rules here too,
but only if it actually helps Python implementations other than
CPython.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tlesher at gmail.com  Thu Jun 12 15:46:43 2008
From: tlesher at gmail.com (Tim Lesher)
Date: Thu, 12 Jun 2008 09:46:43 -0400
Subject: [Python-Dev] OFF-TOPIC-BUT-IMPORTANT: End of free internet
	model by 2012 ?
In-Reply-To: <e1e1d5f40806111325t3e6de92v9678c01b80726744@mail.gmail.com>
References: <e1e1d5f40806111325t3e6de92v9678c01b80726744@mail.gmail.com>
Message-ID: <9613db600806120646l21965124v5ae5577cb84f812@mail.gmail.com>

On Wed, Jun 11, 2008 at 4:25 PM, Daniel Bonekeeper <thehazard at gmail.com>
wrote:

> Hi everybody on the list !
>
> Sorry about the off-topic message, but giving the nature of the
> message and the nature of the list (after all, everybody here uses
> Internet or develops for web in some way), I decided to post this here
> and see what you guys think about it.
>

My gut feeling is that you are being taken by some fairly amateurish viral
marketing, and that you've played into it by uncritically regurgitating the
message onto these mailing lists.

Googling "Dylan Pattyn" yields only references to this very video and a site
for a performance art/activist group called IPower, which boasts such
"projects" as "an suicide-awareness campaign disguised as a personal suicide
blog", a self-described "online stunt" called "40,000 Blowjobs", and a
"wacky online reality series".

Really, do you honestly think that Time Magazine would use a
twenty-something freelancer, who has never published anything in any major
periodical, to break a news article that would at the least create a decade
or more of anti-trust lawsuits?

-- 
Tim Lesher <tlesher at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080612/06b283b2/attachment.htm>

From fwierzbicki at gmail.com  Fri Jun 13 05:06:23 2008
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Thu, 12 Jun 2008 23:06:23 -0400
Subject: [Python-Dev] Assignment to None
In-Reply-To: <d2155e360806111427v234f1ad5yd9870ace7fc7b9b9@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<484CBD70.3000808@v.loewis.de>
	<d2155e360806111426u16747c12r27b4239b5dfa481b@mail.gmail.com>
	<d2155e360806111427v234f1ad5yd9870ace7fc7b9b9@mail.gmail.com>
Message-ID: <4dab5f760806122006t5bfa9fd7h18dd6ef3f1300391@mail.gmail.com>

On Wed, Jun 11, 2008 at 5:27 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
> If I recall correctly, Jython handles this by appending a trailing
> underscore to the imported name and there's no reason why we couldn't
> do something similar.
In truth the current implementation of Jython allows keywords in many
strange places, I expect this was done to allow for method names that
are not keywords in Java so, for example, if there is a method called
"print" in a Java class that we want to call (quite common) then it
can be called.  As far as I know appended underscores don't enter into
it.  Some poking around reveals that the current Jython is probably
too lax here, even keywords that are common to both Python and Java
can be method names (like "if").  I plan to continue to allow Python
keywords that are not Java keywords to behave this way at least for
the 2.x series, though I don't think I'm going to go through the
effort of allowing keywords common to both Java and Python like "if"
(The 2.5 version of Jython will have a new parser so I do actually
need to make these explicit choices right now).  I think changing this
behavior would hurt backwards compatibility too much.  Maybe I'll
rethink it in our 3.0 timeframe.  If a convention like a trailing
underscore is adopted we might move to that in the move to 3.0.

-Frank

From curt at hagenlocher.org  Fri Jun 13 05:17:34 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Thu, 12 Jun 2008 20:17:34 -0700
Subject: [Python-Dev] Assignment to None
In-Reply-To: <4dab5f760806122006t5bfa9fd7h18dd6ef3f1300391@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<484CBD70.3000808@v.loewis.de>
	<d2155e360806111426u16747c12r27b4239b5dfa481b@mail.gmail.com>
	<d2155e360806111427v234f1ad5yd9870ace7fc7b9b9@mail.gmail.com>
	<4dab5f760806122006t5bfa9fd7h18dd6ef3f1300391@mail.gmail.com>
Message-ID: <d2155e360806122017v74015705v5375d16955a2999@mail.gmail.com>

On Thu, Jun 12, 2008 at 8:06 PM, Frank Wierzbicki <fwierzbicki at gmail.com> wrote:
>
> On Wed, Jun 11, 2008 at 5:27 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
> > If I recall correctly, Jython handles this by appending a trailing
> > underscore to the imported name and there's no reason why we couldn't
> > do something similar.
>
> In truth the current implementation of Jython allows keywords in many
> strange places, I expect this was done to allow for method names that
> are not keywords in Java so, for example, if there is a method called
> "print" in a Java class that we want to call (quite common) then it
> can be called.  As far as I know appended underscores don't enter into
> it.

After posting that message, I did what I should have done initially
which was to ask Jim Hugunin about it.  He said that Jython had gotten
Guido's blessing to parse keywords in a context-sensitive fashion --
so that "foo.{keyword}" might be considered legal under certain
circumstances.  I don't, alas, have any specific cites to that end,
but I suspect that we'll be following that precedent :).

--
Curt Hagenlocher
curt at hagenlocher.org

From fwierzbicki at gmail.com  Fri Jun 13 05:19:15 2008
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Thu, 12 Jun 2008 23:19:15 -0400
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <484FA003.5020703@gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F4E45.9060004@canterbury.ac.nz> <484FA003.5020703@gmail.com>
Message-ID: <4dab5f760806122019ob74feexbe60a4191630afd6@mail.gmail.com>

On Wed, Jun 11, 2008 at 5:50 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Greg Ewing wrote:
> Implementations are also permitted to restrict namespace dictionaries to
> only accept string keys (I believe Jython does this for performance reasons
> - CPython just optimised the hell out of normal dictionaries that happen to
> only contain string keys instead).
Jython's current version (2.2) and back did indeed restrict these
dictionaries to accept string keys only, but future versions will
allow non-string keys (the trunk version already does).  Many
frameworks use non-string keys in these dictionaries.

-Frank

From guido at python.org  Fri Jun 13 05:41:36 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2008 20:41:36 -0700
Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins
Message-ID: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>

My colleague and SVN developer Ben Sussman-Collins occasionally blogs
about the social side of (mostly open source) software development. He
just posted a new one that struck a chord:

http://blog.red-bean.com/sussman/?p=96

The story's main moral: submit your code for review early and often;
work in a branch if you need to, but don't hide your code from review
in a local repository until it's "perfect".

Let's all remember this and make sure not to drop "code bombs" on each
other. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun 13 05:42:58 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2008 20:42:58 -0700
Subject: [Python-Dev] bug or a feature?
In-Reply-To: <4dab5f760806122019ob74feexbe60a4191630afd6@mail.gmail.com>
References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com>
	<484F4E45.9060004@canterbury.ac.nz> <484FA003.5020703@gmail.com>
	<4dab5f760806122019ob74feexbe60a4191630afd6@mail.gmail.com>
Message-ID: <ca471dc20806122042k1d3a1e68q845277dc5531629d@mail.gmail.com>

On Thu, Jun 12, 2008 at 8:19 PM, Frank Wierzbicki <fwierzbicki at gmail.com> wrote:
> On Wed, Jun 11, 2008 at 5:50 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Greg Ewing wrote:
>> Implementations are also permitted to restrict namespace dictionaries to
>> only accept string keys (I believe Jython does this for performance reasons
>> - CPython just optimised the hell out of normal dictionaries that happen to
>> only contain string keys instead).
> Jython's current version (2.2) and back did indeed restrict these
> dictionaries to accept string keys only, but future versions will
> allow non-string keys (the trunk version already does).  Many
> frameworks use non-string keys in these dictionaries.

I think that's laudable from a compatibility POV, but at the same time
I think frameworks should rethink such usage and switch to strings
containing non-identifier characters, e.g. '$' or '.' or some such
convention.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nnorwitz at gmail.com  Fri Jun 13 05:49:11 2008
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 12 Jun 2008 20:49:11 -0700
Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins
In-Reply-To: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>
References: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>
Message-ID: <ee2a432c0806122049p71ce274ame2520d571b73798f@mail.gmail.com>

On Thu, Jun 12, 2008 at 8:41 PM, Guido van Rossum <guido at python.org> wrote:
> My colleague and SVN developer Ben Sussman-Collins occasionally blogs
> about the social side of (mostly open source) software development. He
> just posted a new one that struck a chord:
>
> http://blog.red-bean.com/sussman/?p=96
>
> The story's main moral: submit your code for review early and often;
> work in a branch if you need to, but don't hide your code from review
> in a local repository until it's "perfect".
>
> Let's all remember this and make sure not to drop "code bombs" on each
> other. :-)

Ben mentions this in the post, but it's a good reminder:  comments on
python-checkins are *not* personal.  The goal is to make the code
better and/or gain better understanding.  We all make mistakes, better
to correct them early before they become big problems..

n

From guido at python.org  Fri Jun 13 05:51:38 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2008 20:51:38 -0700
Subject: [Python-Dev] [Python-3000-checkins] r64217 - in
	python/branches/py3k/Lib: bsddb/test/test_associate.py
	bsddb/test/test_join.py bsddb/test/test_lock.py
	bsddb/test/test_thread.py idlelib/rpc.py idlelib/run.py
	socketserver.py test/test_threadedtempfile.py thread
Message-ID: <ca471dc20806122051r6f7b0087lb0f0d6606793430c@mail.gmail.com>

It's water under the bridge now, but IMO it was too rash to *remove*
the old threading API from Py3k, and doubly rash to do so one day
before the beta release. Running up to a release (whether alpha, beta
or final) we should practice extra restraint, not rush to get things
in right before the deadline. Let's all be more careful the rest of
this release cycle! (I think it wasn't just Benjamin who raced to get
things in...)

--Guido

On Thu, Jun 12, 2008 at 7:00 PM, benjamin.peterson
<python-3000-checkins at python.org> wrote:
> Author: benjamin.peterson
> Date: Fri Jun 13 04:00:47 2008
> New Revision: 64217
>
> Log:
> fix more threading API related bugs
>
> Modified:
>   python/branches/py3k/Lib/bsddb/test/test_associate.py
>   python/branches/py3k/Lib/bsddb/test/test_join.py
>   python/branches/py3k/Lib/bsddb/test/test_lock.py
>   python/branches/py3k/Lib/bsddb/test/test_thread.py
>   python/branches/py3k/Lib/idlelib/rpc.py
>   python/branches/py3k/Lib/idlelib/run.py
>   python/branches/py3k/Lib/socketserver.py
>   python/branches/py3k/Lib/test/test_threadedtempfile.py
>   python/branches/py3k/Lib/threading.py
>
> Modified: python/branches/py3k/Lib/bsddb/test/test_associate.py
> ==============================================================================
> --- python/branches/py3k/Lib/bsddb/test/test_associate.py       (original)
> +++ python/branches/py3k/Lib/bsddb/test/test_associate.py       Fri Jun 13 04:00:47 2008
> @@ -9,7 +9,7 @@
>  from pprint import pprint
>
>  try:
> -    from threading import Thread, currentThread
> +    from threading import Thread, current_thread
>     have_threads = 1
>  except ImportError:
>     have_threads = 0
>
[etc.]

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun 13 05:53:39 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Jun 2008 20:53:39 -0700
Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins
In-Reply-To: <ee2a432c0806122049p71ce274ame2520d571b73798f@mail.gmail.com>
References: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>
	<ee2a432c0806122049p71ce274ame2520d571b73798f@mail.gmail.com>
Message-ID: <ca471dc20806122053h1fca3028qaa603b466db15a9d@mail.gmail.com>

On Thu, Jun 12, 2008 at 8:49 PM, Neal Norwitz <nnorwitz at gmail.com> wrote:
> On Thu, Jun 12, 2008 at 8:41 PM, Guido van Rossum <guido at python.org> wrote:
>> My colleague and SVN developer Ben Sussman-Collins occasionally blogs
>> about the social side of (mostly open source) software development. He
>> just posted a new one that struck a chord:
>>
>> http://blog.red-bean.com/sussman/?p=96
>>
>> The story's main moral: submit your code for review early and often;
>> work in a branch if you need to, but don't hide your code from review
>> in a local repository until it's "perfect".
>>
>> Let's all remember this and make sure not to drop "code bombs" on each
>> other. :-)
>
> Ben mentions this in the post, but it's a good reminder:  comments on
> python-checkins are *not* personal.  The goal is to make the code
> better and/or gain better understanding.  We all make mistakes, better
> to correct them early before they become big problems..

And this reminder applies to reviewer *and* reviewees! (I know I've
made this mistake in both roles. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Fri Jun 13 10:54:12 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Jun 2008 18:54:12 +1000
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <18513.16097.394123.312491@montanaro-dyndns-org.local>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>	<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
	<18513.16097.394123.312491@montanaro-dyndns-org.local>
Message-ID: <485235B4.6030401@gmail.com>

skip at pobox.com wrote:
> I'm not fond of using a property for this since it can lull you into the
> false belief that what you are doing is less expensive than it really is
> (attribute access vs method call).  I think this is a case where explicit is
> better than implicit.

Have you looked at what the methods we're proposing to turn into 
properties are actually doing?:

     def getName(self):
         assert self.__initialized, "Thread.__init__() not called"
         return self.__name

     def setName(self, name):
         assert self.__initialized, "Thread.__init__() not called"
         self.__name = str(name)

     def getIdent(self):
         assert self.__initialized, "Thread.__init__() not called"
         return self.__ident

     def isAlive(self):
         assert self.__initialized, "Thread.__init__() not called"
         return self.__started.isSet() and not self.__stopped

     def isDaemon(self):
         assert self.__initialized, "Thread.__init__() not called"
         return self.__daemonic

     def setDaemon(self, daemonic):
         if not self.__initialized:
             raise RuntimeError("Thread.__init__() not called")
         if self.__started.isSet():
             raise RuntimeError("cannot set daemon status of active 
thread");
         self.__daemonic = daemonic

Checking whether or not an Event is set is hardly an expensive operation 
and well within the limits of what I think it is reasonable for a 
property access to do implicitly (e.g. it would also be reasonable for a 
thread safe object to acquire and release a synchronisation lock while 
retrieving or changing an attribute).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Fri Jun 13 10:56:01 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Jun 2008 18:56:01 +1000
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <g2s72u$2q5$1@ger.gmane.org>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>	<4850FBFC.9070501@gmail.com>
	<g2s72u$2q5$1@ger.gmane.org>
Message-ID: <48523621.6050604@gmail.com>

Georg Brandl wrote:
> Nick Coghlan schrieb:
> 
>> 2. Method lookup MAY bypass __getattribute__, shadowing the attribute 
>> in the instance dictionary MAY have ill effects. (slots such as 
>> __enter__ and __exit__ that are looked up via normal attribute lookup 
>> in CPython will fit into this category)
> 
> I would argue that the __enter__ and __exit__ behavior should be changed 
> too.
> The reason for the current behavior is this:
> 
>   2           0 LOAD_GLOBAL              0 (a)
>               3 DUP_TOP
>               4 LOAD_ATTR                1 (__exit__)
>               7 STORE_FAST               0 (_[1])
>              10 LOAD_ATTR                2 (__enter__)
>              13 CALL_FUNCTION            0
>              16 STORE_FAST               1 (_[2])
>              19 SETUP_FINALLY           18 (to 40)
> 
> IOW, when "with" is compiled, the attributes are retrieved using bytecode.
> It wouldn't be hard to have a WITH_SETUP opcode (along with the already
> existing WITH_CLEANUP) that would make the bytecode read like:
> 
>   2           0 LOAD_GLOBAL              0 (a)
>               3 WITH_SETUP               0 (_[1])
>               6 STORE_FAST               1 (_[2])
>               9 SETUP_FINALLY           18 (to 40)

No argument here - the PEP 343 implementation is the way it is mainly 
because it involved the least messing around with the 
performance-sensitive ceval loop.

That said, the with statement implementation is already a bit different 
in 2.6/3.0 (it moves things around on the stack so it can avoid the 
STORE_FAST/LOAD_FAST/DELETE_FAST operations):

def f():
   with a:
     pass

Python 2.5 disassembly:
  2           0 LOAD_GLOBAL              0 (a)
               3 DUP_TOP
               4 LOAD_ATTR                1 (__exit__)
               7 STORE_FAST               0 (_[1])
              10 LOAD_ATTR                2 (__enter__)
              13 CALL_FUNCTION            0
              16 POP_TOP
              17 SETUP_FINALLY            4 (to 24)
              20 POP_BLOCK
              21 LOAD_CONST               0 (None)
         >>   24 LOAD_FAST                0 (_[1])
              27 DELETE_FAST              0 (_[1])
              30 WITH_CLEANUP
              31 END_FINALLY

Python 2.6 disassembly:
   2           0 LOAD_GLOBAL              0 (a)
               3 DUP_TOP
               4 LOAD_ATTR                1 (__exit__)
               7 ROT_TWO
               8 LOAD_ATTR                2 (__enter__)
              11 CALL_FUNCTION            0
              14 POP_TOP
              15 SETUP_FINALLY            4 (to 22)
              18 POP_BLOCK
              19 LOAD_CONST               0 (None)
         >>   22 WITH_CLEANUP
              23 END_FINALLY

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Fri Jun 13 11:06:00 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Jun 2008 19:06:00 +1000
Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins
In-Reply-To: <ca471dc20806122053h1fca3028qaa603b466db15a9d@mail.gmail.com>
References: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>	<ee2a432c0806122049p71ce274ame2520d571b73798f@mail.gmail.com>
	<ca471dc20806122053h1fca3028qaa603b466db15a9d@mail.gmail.com>
Message-ID: <48523878.9030104@gmail.com>

Guido van Rossum wrote:
> On Thu, Jun 12, 2008 at 8:49 PM, Neal Norwitz <nnorwitz at gmail.com>
> wrote:
>> Ben mentions this in the post, but it's a good reminder:  comments
>> on python-checkins are *not* personal.  The goal is to make the
>> code better and/or gain better understanding.  We all make
>> mistakes, better to correct them early before they become big
>> problems..
> 
> And this reminder applies to reviewer *and* reviewees! (I know I've 
> made this mistake in both roles. :-)

I still love the last entry from Raymond's school of hard knocks [1]:

"""do everything right (formatting,
    procedure, profiling, testing, etc) and watch the Timbot
    come along five minutes later and improve your code
    making it faster, clearer, more conformant, more elegant,
    and also gel neatly with the vaguaries of memory allocation,
    cache performance, and compilers you've never heard of."""

Cheers,
Nick.

[1] http://mail.python.org/pipermail/python-dev/2002-September/028725.html


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From walter at livinglogic.de  Fri Jun 13 11:32:21 2008
From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Fri, 13 Jun 2008 11:32:21 +0200
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <48514428.7040001@egenix.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<484FE1E9.2080904@egenix.com>	<484FEC2C.6070409@livinglogic.de>	<485003F8.40803@egenix.com>
	<485139EA.1000502@livinglogic.de> <48514428.7040001@egenix.com>
Message-ID: <48523EA5.2070407@livinglogic.de>

M.-A. Lemburg wrote:
> On 2008-06-12 16:59, Walter D?rwald wrote:
>> M.-A. Lemburg wrote:
>>> .transform() and .untransform() use the codecs to apply same-type
>>> conversions. They do apply type checks to make sure that the
>>> codec does indeed return the same type.
>>>
>>> E.g. text.transform('xml-escape') or data.transform('base64').
>>
>> So what would a base64 codec do with the errors argument?
> 
> It could use it to e.g. try to recover as much data as possible
> from broken input data.
> 
> Currently (in Py2.x), it raises an exception if you pass in anything
> but "strict".
> 
>>>> I think for transformations we don't need the full codec machinery:
>>>  > ...
>>>
>>> No need to invent another wheel :-) The codecs already exist for
>>> Py2.x and can be used by the .encode()/.decode() methods in Py2.x
>>> (where no type checks occur).
>>
>> By using a new API we could get rid of old warts. For example: Why 
>> does the stateless encoder/decoder return how many input 
>> characters/bytes it has consumed? It must consume *all* bytes anyway!
> 
> No, it doesn't and that's the point in having those return values :-)
> 
> Even though the encoder/decoders are stateless, that doesn't mean
> they have to consume all input data. The caller is responsible to
> make sure that all input data was in fact consumed.
> 
> You could for example have a decoder that stops decoding after
> having seen a block end indicator, e.g. a base64 line end or
> XML closing element.

So how should the UTF-8 decoder know that it has to stop at a closing 
XML element?

> Just because all codecs that ship with Python always try to decode
> the complete input doesn't mean that the feature isn't being used.

I know of no other code that does. Do you have an example for this use.

> The interface was designed to allow for the above situations.

Then could we at least have a new codec method that does:

def statelesencode(self, input):
    (output, consumed) = self.encode(input)
    assert len(input) == consumed
    return output

Servus,
    Walter


From fuzzyman at voidspace.org.uk  Fri Jun 13 12:02:53 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 13 Jun 2008 11:02:53 +0100
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <ca471dc20806120655s60e378bj8850b37c676b313b@mail.gmail.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<ca471dc20806111002w300abfd6y47d6a41148bd56a@mail.gmail.com>	<4850FBFC.9070501@gmail.com>
	<ca471dc20806120655s60e378bj8850b37c676b313b@mail.gmail.com>
Message-ID: <485245CD.9000602@voidspace.org.uk>

Guido van Rossum wrote:
> [Barry]
>   
>>>> http://bugs.python.org/issue643841
>>>>         
>
> [Guido]
>   
>>> I've added a comment. Let me know if anything I said is unclear.
>>>       
>
> On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>   
>> The bugtracker seems to be offline atm - I'll reply there once I can get to
>> it again (as well as switching this issue back to being a documentation
>> one).
>>
>> I don't think we're going to see a major clamor for a value-based delegation
>> mixin in the standard library until people using classic classes for
>> value-based delegation start making serious attempts to port to Py3k (where
>> that approach is no longer available). At the moment, such classes only need
>> to care about the methods they want to fiddle with, leaving everything else
>> to __getattr__ based delegation.
>>     
>
> Whether they'll care about this issue of course depends on whether
> they need overloaded operators and other special delegations to be
> delegated transparently. We'll have to see how important this is.
> New-style classes have been around and recommended for a long time --
> why haven't people pushed for a proxy class before?
>
>   

It's only in Python 3 that old style classes are going away fully, so up
until now you could at least use a classic class to do the proxying.

I've written my own proxy classes before that look very similar to this,
and there are other proxy classes around that do the same (I thought one
was by Phillip J Eby but can't find a reference easily). The last one I
wrote was to proxy CPython objects from IronPython via Python.NET...

I would prefer it if the proxy class wrapped the return values of
inplace operations.

Michael Foord

>> I've pushed as hard as I'm personally willing to for this without convincing
>> anyone else that it's worth doing,
>>     
>
> What does *it* refer to? Changing the behavior, or adding a proxy
> class to the stdlib? I'm -1000 on the former, but only -0 on the
> latter -- as I wrote in the tracker, I just don't want to see an
> unproven proxy class (and I don't like the module name).
>
>   
>> so I'll start working on a documentation
>> patch for the language reference instead which explicitly splits the special
>> methods into the following categories:
>>     
>
> Thanks for doing this, it is needed regardless!
>
>   
>> 1. Method lookup MUST bypass __getattribute__, shadowing the attribute in
>> the instance dictionary MUST NOT have any ill effects. (All tp_* C-level
>> slots and slots looked up via _PyType_Lookup will fit into this category)
>>     
>
> Watch out: I think the term "method lookup" may be confusing here.
> Certainly when the user writes x.__foo__(), the instance dict *is*
> consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where
> the instance dict is bypassed.
>
>   
>> 2. Method lookup MAY bypass __getattribute__, shadowing the attribute in the
>> instance dictionary MAY have ill effects. (slots such as __enter__ and
>> __exit__ that are looked up via normal attribute lookup in CPython will fit
>> into this category)
>>     
>
> Why even have a  MAY category? Are you expecting these to become tp_
> slots at some point?
>
>   
>> 3. Technically a subcategory of group 1, these are special methods which can
>> affect the interpreter's behaviour by their mere definition on a type. (The
>> __get__, __set__ and __delete__ descriptor protocol methods fall into this
>> category)
>>     
>
> I don't follow why this is relevant. This is a different, AFAIK
> orthogonal issue, used in many places: *if* an object used in a
> certain context has a specific attribute, *then* that attribute is
> used, *otherwise* a default action is taken. Applies to __repr__ just
> as much. These belong in category 1 if and only if the lookup bypasses
> the instance dict.
>
>   



From mal at egenix.com  Fri Jun 13 12:14:49 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 13 Jun 2008 12:14:49 +0200
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <48523EA5.2070407@livinglogic.de>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<484FE1E9.2080904@egenix.com>	<484FEC2C.6070409@livinglogic.de>	<485003F8.40803@egenix.com>	<485139EA.1000502@livinglogic.de>
	<48514428.7040001@egenix.com> <48523EA5.2070407@livinglogic.de>
Message-ID: <48524899.2030609@egenix.com>

On 2008-06-13 11:32, Walter D?rwald wrote:
> M.-A. Lemburg wrote:
>> On 2008-06-12 16:59, Walter D?rwald wrote:
>>> M.-A. Lemburg wrote:
>>>> .transform() and .untransform() use the codecs to apply same-type
>>>> conversions. They do apply type checks to make sure that the
>>>> codec does indeed return the same type.
>>>>
>>>> E.g. text.transform('xml-escape') or data.transform('base64').
>>>
>>> So what would a base64 codec do with the errors argument?
>>
>> It could use it to e.g. try to recover as much data as possible
>> from broken input data.
>>
>> Currently (in Py2.x), it raises an exception if you pass in anything
>> but "strict".
>>
>>>>> I think for transformations we don't need the full codec machinery:
>>>>  > ...
>>>>
>>>> No need to invent another wheel :-) The codecs already exist for
>>>> Py2.x and can be used by the .encode()/.decode() methods in Py2.x
>>>> (where no type checks occur).
>>>
>>> By using a new API we could get rid of old warts. For example: Why 
>>> does the stateless encoder/decoder return how many input 
>>> characters/bytes it has consumed? It must consume *all* bytes anyway!
>>
>> No, it doesn't and that's the point in having those return values :-)
>>
>> Even though the encoder/decoders are stateless, that doesn't mean
>> they have to consume all input data. The caller is responsible to
>> make sure that all input data was in fact consumed.
>>
>> You could for example have a decoder that stops decoding after
>> having seen a block end indicator, e.g. a base64 line end or
>> XML closing element.
> 
> So how should the UTF-8 decoder know that it has to stop at a closing 
> XML element?

The UTF-8 decoder doesn't support this, but you could write a codec
that applies this kind of detection, e.g. to not try to decode
partial UTF-8 byte sequences at the end of input, which would then
result in error.

>> Just because all codecs that ship with Python always try to decode
>> the complete input doesn't mean that the feature isn't being used.
> 
> I know of no other code that does. Do you have an example for this use.

I already gave you a few examples.

>> The interface was designed to allow for the above situations.
> 
> Then could we at least have a new codec method that does:
> 
> def statelesencode(self, input):
>    (output, consumed) = self.encode(input)
>    assert len(input) == consumed
>    return output

You mean as method to the Codec class ?

Sure, we could do that, but please use a different name,
e.g. .encodeall() and .decodeall() - .encode() and .decode()
are already stateles (and so would the new methods be), so
"stateless" isn't all that meaningful in this context.

We could also add such a check to the PyCodec_Encode() and _Decode()
functions. They currently do not apply the above check.

In Python, those two functions are exposed as codecs.encode()
and codecs.decode().

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 13 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            23 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From steve at holdenweb.com  Fri Jun 13 12:53:14 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 13 Jun 2008 06:53:14 -0400
Subject: [Python-Dev] [Python-3000-checkins] r64217 - in
 python/branches/py3k/Lib: bsddb/test/test_associate.py
 bsddb/test/test_join.py bsddb/test/test_lock.py bsddb/test/test_thread.py
 idlelib/rpc.py idlelib/run.py socketserver.py test/test_threadedtempfile.py
 thread
In-Reply-To: <ca471dc20806122051r6f7b0087lb0f0d6606793430c@mail.gmail.com>
References: <ca471dc20806122051r6f7b0087lb0f0d6606793430c@mail.gmail.com>
Message-ID: <4852519A.6070506@holdenweb.com>

Guido van Rossum wrote:
> It's water under the bridge now, but IMO it was too rash to *remove*
> the old threading API from Py3k, and doubly rash to do so one day
> before the beta release. Running up to a release (whether alpha, beta
> or final) we should practice extra restraint, not rush to get things
> in right before the deadline. Let's all be more careful the rest of
> this release cycle! (I think it wasn't just Benjamin who raced to get
> things in...)

Well, it wouldn't be "adding a new feature" to reinstate the old API for 
beta two, would it, as long as we retain the new one too? It does seem 
that change was a little precipitate.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/



From steve at holdenweb.com  Fri Jun 13 12:53:14 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 13 Jun 2008 06:53:14 -0400
Subject: [Python-Dev] [Python-3000-checkins] r64217 - in
 python/branches/py3k/Lib: bsddb/test/test_associate.py
 bsddb/test/test_join.py bsddb/test/test_lock.py bsddb/test/test_thread.py
 idlelib/rpc.py idlelib/run.py socketserver.py test/test_threadedtempfile.py
 thread
In-Reply-To: <ca471dc20806122051r6f7b0087lb0f0d6606793430c@mail.gmail.com>
References: <ca471dc20806122051r6f7b0087lb0f0d6606793430c@mail.gmail.com>
Message-ID: <4852519A.6070506@holdenweb.com>

Guido van Rossum wrote:
> It's water under the bridge now, but IMO it was too rash to *remove*
> the old threading API from Py3k, and doubly rash to do so one day
> before the beta release. Running up to a release (whether alpha, beta
> or final) we should practice extra restraint, not rush to get things
> in right before the deadline. Let's all be more careful the rest of
> this release cycle! (I think it wasn't just Benjamin who raced to get
> things in...)

Well, it wouldn't be "adding a new feature" to reinstate the old API for 
beta two, would it, as long as we retain the new one too? It does seem 
that change was a little precipitate.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From ncoghlan at gmail.com  Fri Jun 13 15:07:47 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Jun 2008 23:07:47 +1000
Subject: [Python-Dev] [Python-3000] [Python-3000-checkins] r64217 - in
 python/branches/py3k/Lib: bsddb/test/test_associate.py
 bsddb/test/test_join.py bsddb/test/test_lock.py bsddb/test/test_thread.py
 idlelib/rpc.py idlelib/run.py socketserver.py test/test_threadedtempfile.py
 thread
In-Reply-To: <4852519A.6070506@holdenweb.com>
References: <ca471dc20806122051r6f7b0087lb0f0d6606793430c@mail.gmail.com>
	<4852519A.6070506@holdenweb.com>
Message-ID: <48527123.9010802@gmail.com>

Steve Holden wrote:
> Guido van Rossum wrote:
>> It's water under the bridge now, but IMO it was too rash to *remove*
>> the old threading API from Py3k, and doubly rash to do so one day
>> before the beta release. Running up to a release (whether alpha, beta
>> or final) we should practice extra restraint, not rush to get things
>> in right before the deadline. Let's all be more careful the rest of
>> this release cycle! (I think it wasn't just Benjamin who raced to get
>> things in...)
> 
> Well, it wouldn't be "adding a new feature" to reinstate the old API for 
> beta two, would it, as long as we retain the new one too? It does seem 
> that change was a little precipitate.

Although if we weren't actually planning on removing the old API in 3.0, 
I'm a little confused as to why we were adding Py3k warnings to it in 2.6...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From walter at livinglogic.de  Fri Jun 13 15:26:47 2008
From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Fri, 13 Jun 2008 15:26:47 +0200
Subject: [Python-Dev] [Python-3000] Betas today - I hope
In-Reply-To: <48524899.2030609@egenix.com>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<484FE1E9.2080904@egenix.com>	<484FEC2C.6070409@livinglogic.de>	<485003F8.40803@egenix.com>	<485139EA.1000502@livinglogic.de>
	<48514428.7040001@egenix.com> <48523EA5.2070407@livinglogic.de>
	<48524899.2030609@egenix.com>
Message-ID: <48527597.4090206@livinglogic.de>

M.-A. Lemburg wrote:
> On 2008-06-13 11:32, Walter D?rwald wrote:
>> M.-A. Lemburg wrote:
>>> On 2008-06-12 16:59, Walter D?rwald wrote:
>>>> M.-A. Lemburg wrote:
>>>>> .transform() and .untransform() use the codecs to apply same-type
>>>>> conversions. They do apply type checks to make sure that the
>>>>> codec does indeed return the same type.
>>>>>
>>>>> E.g. text.transform('xml-escape') or data.transform('base64').
>>>>
>>>> So what would a base64 codec do with the errors argument?
>>>
>>> It could use it to e.g. try to recover as much data as possible
>>> from broken input data.
>>>
>>> Currently (in Py2.x), it raises an exception if you pass in anything
>>> but "strict".
>>>
>>>>>> I think for transformations we don't need the full codec machinery:
>>>>>  > ...
>>>>>
>>>>> No need to invent another wheel :-) The codecs already exist for
>>>>> Py2.x and can be used by the .encode()/.decode() methods in Py2.x
>>>>> (where no type checks occur).
>>>>
>>>> By using a new API we could get rid of old warts. For example: Why 
>>>> does the stateless encoder/decoder return how many input 
>>>> characters/bytes it has consumed? It must consume *all* bytes anyway!
>>>
>>> No, it doesn't and that's the point in having those return values :-)
>>>
>>> Even though the encoder/decoders are stateless, that doesn't mean
>>> they have to consume all input data. The caller is responsible to
>>> make sure that all input data was in fact consumed.
>>>
>>> You could for example have a decoder that stops decoding after
>>> having seen a block end indicator, e.g. a base64 line end or
>>> XML closing element.
>>
>> So how should the UTF-8 decoder know that it has to stop at a closing 
>> XML element?
> 
> The UTF-8 decoder doesn't support this, but you could write a codec
> that applies this kind of detection, e.g. to not try to decode
> partial UTF-8 byte sequences at the end of input, which would then
> result in error.
> 
>>> Just because all codecs that ship with Python always try to decode
>>> the complete input doesn't mean that the feature isn't being used.
>>
>> I know of no other code that does. Do you have an example for this use.
> 
> I already gave you a few examples.

Maybe I was unclear, I meant real world examples, not hypothetical ones.

>>> The interface was designed to allow for the above situations.
>>
>> Then could we at least have a new codec method that does:
>>
>> def statelesencode(self, input):
>>    (output, consumed) = self.encode(input)
>>    assert len(input) == consumed
>>    return output
> 
> You mean as method to the Codec class ?

No, I meant as a method for the CodecInfo clas.

> Sure, we could do that, but please use a different name,
> e.g. .encodeall() and .decodeall() - .encode() and .decode()
> are already stateles (and so would the new methods be), so
> "stateless" isn't all that meaningful in this context.

I like the names encodeall/decodeall!

> We could also add such a check to the PyCodec_Encode() and _Decode()
> functions. They currently do not apply the above check.
> 
> In Python, those two functions are exposed as codecs.encode()
> and codecs.decode().

This change will probably have to wait for the 2.7 cycle.

Servus,
    Walter

From barry at python.org  Fri Jun 13 15:47:40 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 13 Jun 2008 09:47:40 -0400
Subject: [Python-Dev] First betas (perhaps) on June 18
Message-ID: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

It's unfortunate that I was not able to release the first betas on  
Wednesday.  The primary reason was that none of the 3.0 buildbots were  
green.

My proposal is this: I will spin another release this coming  
Wednesday, June 18.  If we can get both the 2.6 and 3.0 buildbots  
green by then, and close out all remaining release critical bugs, then  
Wednesday's release will be beta 1.  In that case, I will update PEP  
361 and push the entire schedule back by 2 weeks to account for our  
slippage.

If we can't get these releases stable by then, Wednesday's release  
will be alpha and I will make the July 2 release the first beta,  
updating and pushing back the PEP 361 schedule accordingly.

Thanks for all your help in getting 3.0 stable!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFJ6fXEjvBPtnXfVAQJekgP/XzGR0h6Kcgwe/eo/hlAP2RJiEBny7iIT
avSkq5sDD4YiUHeSSVy/8A5rahjR3Bsrzgh0MSePrf8coUHz3N+OCGsJK07BVJIT
9wan/OXTkpsrqa2xQgrokk0PqCE80S2Lp6+P5jMW06tujx3fFn+NJNDjaNUOKPUc
nlfjFpdwYxc=
=Nwmx
-----END PGP SIGNATURE-----

From facundobatista at gmail.com  Fri Jun 13 15:54:08 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 13 Jun 2008 10:54:08 -0300
Subject: [Python-Dev] [Python-3000] First betas (perhaps) on June 18
In-Reply-To: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org>
References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org>
Message-ID: <e04bdf310806130654o109c68e9l2e42e4fd899e1901@mail.gmail.com>

2008/6/13 Barry Warsaw <barry at python.org>:

> My proposal is this: I will spin another release this coming Wednesday, June
> 18.  If we can get both the 2.6 and 3.0 buildbots green by then, and close
> out all remaining release critical bugs, then Wednesday's release will be
> beta 1.  In that case, I will update PEP 361 and push the entire schedule
> back by 2 weeks to account for our slippage.

Next weekend, 21 and 22, it will be the Python Bug days.

Maybe it's worth to delay the betas one week to include this work?

Regards,

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From guido at python.org  Fri Jun 13 16:20:09 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 07:20:09 -0700
Subject: [Python-Dev] [Python-3000] First betas (perhaps) on June 18
In-Reply-To: <e04bdf310806130654o109c68e9l2e42e4fd899e1901@mail.gmail.com>
References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org>
	<e04bdf310806130654o109c68e9l2e42e4fd899e1901@mail.gmail.com>
Message-ID: <ca471dc20806130720p6f0ec3d0y9d0a5f4560347d55@mail.gmail.com>

On Fri, Jun 13, 2008 at 6:54 AM, Facundo Batista
<facundobatista at gmail.com> wrote:
> 2008/6/13 Barry Warsaw <barry at python.org>:
>
>> My proposal is this: I will spin another release this coming Wednesday, June
>> 18.  If we can get both the 2.6 and 3.0 buildbots green by then, and close
>> out all remaining release critical bugs, then Wednesday's release will be
>> beta 1.  In that case, I will update PEP 361 and push the entire schedule
>> back by 2 weeks to account for our slippage.
>
> Next weekend, 21 and 22, it will be the Python Bug days.
>
> Maybe it's worth to delay the betas one week to include this work?

You mean one more week? (Barry's already delaying them one week.)

I'm fine with whatever delay Barry approves, but I want EVERYONE to
stick to the purpose of the delay: get the buildbots green and close
out release critical bugs. This means no new last-minute features or
big refactorings (unless there's really no more conservative way to
fix a critical bug). It also means try to get at least one other
person to review every change, no matter how small (or big!) *before*
submitting. It also means run the tests before submitting.

On the specific purpose of delaying for the bug day, I'm hesitant --
bug days tend to cause a great deal of commits of code by relatively
new developers, which is counter to our purpose of stability. I would
almost say it's better for the bug day to fall right after the beta.
That also has the advantage that the bug day can focus on new bugs in
the beta, and squash them right away...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun 13 16:33:37 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 07:33:37 -0700
Subject: [Python-Dev] [Python-3000]  First betas (perhaps) on June 18
In-Reply-To: <g2u04a$v8$1@ger.gmane.org>
References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org>
	<e04bdf310806130654o109c68e9l2e42e4fd899e1901@mail.gmail.com>
	<ca471dc20806130720p6f0ec3d0y9d0a5f4560347d55@mail.gmail.com>
	<g2u04a$v8$1@ger.gmane.org>
Message-ID: <ca471dc20806130733g478181den1d0aa99f97b8b9ac@mail.gmail.com>

On Fri, Jun 13, 2008 at 7:27 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Guido van Rossum schrieb:
>>
>> On Fri, Jun 13, 2008 at 6:54 AM, Facundo Batista
>> <facundobatista at gmail.com> wrote:
>>>
>>> 2008/6/13 Barry Warsaw <barry at python.org>:
>>>
>>>> My proposal is this: I will spin another release this coming Wednesday,
>>>> June
>>>> 18.  If we can get both the 2.6 and 3.0 buildbots green by then, and
>>>> close
>>>> out all remaining release critical bugs, then Wednesday's release will
>>>> be
>>>> beta 1.  In that case, I will update PEP 361 and push the entire
>>>> schedule
>>>> back by 2 weeks to account for our slippage.
>>>
>>> Next weekend, 21 and 22, it will be the Python Bug days.
>>>
>>> Maybe it's worth to delay the betas one week to include this work?
>>
>> You mean one more week? (Barry's already delaying them one week.)
>>
>> I'm fine with whatever delay Barry approves, but I want EVERYONE to
>> stick to the purpose of the delay: get the buildbots green and close
>> out release critical bugs. This means no new last-minute features or
>> big refactorings (unless there's really no more conservative way to
>> fix a critical bug). It also means try to get at least one other
>> person to review every change, no matter how small (or big!) *before*
>> submitting. It also means run the tests before submitting.
>
> That is a new policy, isn't it?

Yes -- one that seems appropriate during a beta release period (i.e.
until 2.6 and 3.0 final are out).

>> On the specific purpose of delaying for the bug day, I'm hesitant --
>> bug days tend to cause a great deal of commits of code by relatively
>> new developers, which is counter to our purpose of stability. I would
>> almost say it's better for the bug day to fall right after the beta.
>> That also has the advantage that the bug day can focus on new bugs in
>> the beta, and squash them right away...
>
> +1.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From musiccomposition at gmail.com  Fri Jun 13 16:35:25 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 13 Jun 2008 09:35:25 -0500
Subject: [Python-Dev] [Python-3000]  First betas (perhaps) on June 18
In-Reply-To: <ca471dc20806130720p6f0ec3d0y9d0a5f4560347d55@mail.gmail.com>
References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org>
	<e04bdf310806130654o109c68e9l2e42e4fd899e1901@mail.gmail.com>
	<ca471dc20806130720p6f0ec3d0y9d0a5f4560347d55@mail.gmail.com>
Message-ID: <1afaf6160806130735v40252796laf478e9e99f45148@mail.gmail.com>

On Fri, Jun 13, 2008 at 9:20 AM, Guido van Rossum <guido at python.org> wrote:
>
> I'm fine with whatever delay Barry approves, but I want EVERYONE to
> stick to the purpose of the delay: get the buildbots green and close
> out release critical bugs. This means no new last-minute features or
> big refactorings (unless there's really no more conservative way to
> fix a critical bug). It also means try to get at least one other
> person to review every change, no matter how small (or big!) *before*
> submitting. It also means run the tests before submitting.

I will quite happily abide by this; it's already causing more than
enough stress over here.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From musiccomposition at gmail.com  Fri Jun 13 16:47:32 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 13 Jun 2008 09:47:32 -0500
Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins
In-Reply-To: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>
References: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>
Message-ID: <1afaf6160806130747k4835ab2cmb99ab3b87136142d@mail.gmail.com>

On Thu, Jun 12, 2008 at 10:41 PM, Guido van Rossum <guido at python.org> wrote:
>
> Let's all remember this and make sure not to drop "code bombs" on each
> other. :-)

Ok! I'd like everybody to know that I'm working on Python tests on a
Bazaar branch. [1] (That's only a bzr co --lightweight away.) Go ahead
and send me angry emails about how it's totally wrong, and your cat
could do better.

[1] http://code.python.org/python/users/benjamin.peterson/new_testing/main/



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From solipsis at pitrou.net  Fri Jun 13 17:54:31 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 13 Jun 2008 15:54:31 +0000 (UTC)
Subject: [Python-Dev] [OT] Interesting blog post by Ben Sussman-Collins
References: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>
	<1afaf6160806130747k4835ab2cmb99ab3b87136142d@mail.gmail.com>
Message-ID: <loom.20080613T155334-623@post.gmane.org>

Benjamin Peterson <musiccomposition <at> gmail.com> writes:

> 
> On Thu, Jun 12, 2008 at 10:41 PM, Guido van Rossum <guido <at> python.org> 
wrote:
> >
> > Let's all remember this and make sure not to drop "code bombs" on each
> > other. 
> 
> Ok! I'd like everybody to know that I'm working on Python tests on a
> Bazaar branch. [1] (That's only a bzr co --lightweight away.) Go ahead
> and send me angry emails about how it's totally wrong, and your cat
> could do better.

If I had a cat it would probably use Mercurial instead :-)




From status at bugs.python.org  Fri Jun 13 18:06:27 2008
From: status at bugs.python.org (Python tracker)
Date: Fri, 13 Jun 2008 18:06:27 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20080613160627.4766B7818E@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (06/06/08 - 06/13/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1899 open (+36) / 13028 closed (+15) / 14927 total (+51)

Open issues with patches:   574

Average duration of open issues: 713 days.
Median duration of open issues: 1464 days.

Open Issues Breakdown
   open  1873 (+35)
pending    26 ( +1)

Issues Created Or Reopened (54)
_______________________________

merge pickle and cPickle in 3.0                                  06/12/08
CLOSED http://bugs.python.org/issue2917    reopened benjamin.peterson         
       patch                                                                   

Implement PEP 371: multiprocessing module                        06/11/08
CLOSED http://bugs.python.org/issue3050    reopened benjamin.peterson         
       patch                                                                   

heapq change breaking compatibility                              06/09/08
       http://bugs.python.org/issue3051    reopened rhettinger                
       patch                                                                   

test_disutils fails                                              06/06/08
       http://bugs.python.org/issue3054    created  ronaldoussoren            
       patch                                                                   

test_list on 64-bit platforms                                    06/06/08
       http://bugs.python.org/issue3055    created  ronaldoussoren            
                                                                               

Simplify the Integral ABC                                        06/07/08
       http://bugs.python.org/issue3056    created  rhettinger                
       patch                                                                   

2.6 abc fixes                                                    06/07/08
CLOSED http://bugs.python.org/issue3057    created  georg.brandl              
       patch, 26backport                                                       

Let SimpleXMLRPCServer pass client_address to called functions.  06/07/08
       http://bugs.python.org/issue3058    created  cloverprince              
                                                                               

Removing .decode calls from Lib/calendar.py                      06/07/08
CLOSED http://bugs.python.org/issue3059    created  gpolo                     
       patch                                                                   

Warn about tuple parameters                                      06/08/08
CLOSED http://bugs.python.org/issue3060    created  benjamin.peterson         
       patch                                                                   

time.strftime() always decodes result with UTF-8                 06/08/08
       http://bugs.python.org/issue3061    created  georg.brandl              
                                                                               

Turtle speed() function has no effect under Mac OS X             06/08/08
       http://bugs.python.org/issue3062    created  delroth                   
                                                                               

memory leak in random number generation                          06/08/08
CLOSED http://bugs.python.org/issue3063    created  gtang                     
                                                                               

new turtle module for Python 3.0                                 06/08/08
CLOSED http://bugs.python.org/issue3064    created  gregorlingl               
       patch                                                                   

Fix pickling bug of collections.namedtuple                       06/08/08
CLOSED http://bugs.python.org/issue3065    created  alexandre.vassalotti      
       patch, patch                                                            

FD leak in urllib2                                               06/09/08
       http://bugs.python.org/issue3066    created  bohdan                    
                                                                               

setlocale Tracebacks on unicode locale strings                   06/09/08
       http://bugs.python.org/issue3067    created  vincent.chute             
                                                                               

IDLE - Add an extension configuration dialog                     06/09/08
       http://bugs.python.org/issue3068    created  taleinat                  
       patch                                                                   

Let set.union and set.intersection accept multiple arguments     06/09/08
       http://bugs.python.org/issue3069    created  arno                      
       patch                                                                   

Wrong size calculation in posix_execve                           06/09/08
       http://bugs.python.org/issue3070    created  Rhamphoryncus             
                                                                               

The ValueError raised by failing to unpack sequence should have  06/10/08
       http://bugs.python.org/issue3071    created  jml                       
                                                                               

Assignment to list of lists gives incorrect result               06/10/08
CLOSED http://bugs.python.org/issue3072    created  stesteve                  
                                                                               

Cookie.Morsel breaks in parsing cookie values with whitespace    06/10/08
       http://bugs.python.org/issue3073    created  rhymes                    
                                                                               

test_asyncore is failing                                         06/10/08
CLOSED http://bugs.python.org/issue3074    created  benjamin.peterson         
                                                                               

make minidom.toxml() encoding argument useful                    06/10/08
       http://bugs.python.org/issue3075    created  janssen                   
       easy                                                                    

test_xmlrpc_net fails                                            06/10/08
CLOSED http://bugs.python.org/issue3076    created  cartman                   
                                                                               

h2py char literal doesn work                                     06/11/08
       http://bugs.python.org/issue3077    created  grossetti                 
       patch                                                                   

tokenize.py improvements                                         06/11/08
       http://bugs.python.org/issue3078    created  Indy                      
       patch                                                                   

sys.exit() called from optparse - bad, bad, bad                  06/11/08
       http://bugs.python.org/issue3079    created  skip.montanaro            
       easy                                                                    

Full unicode import system                                       06/11/08
       http://bugs.python.org/issue3080    created  amaury.forgeotdarc        
                                                                               

Py_(X)SETREF macros                                              06/11/08
       http://bugs.python.org/issue3081    created  pitrou                    
       patch                                                                   

test_multiprocessing broken                                      06/11/08
CLOSED http://bugs.python.org/issue3082    created  gregory.p.smith           
                                                                               

Add alternate (#) formatting for bin, oct, hex output for str.fo 06/11/08
       http://bugs.python.org/issue3083    created  eric.smith                
                                                                               

sys.exit() called from optparse - bad, bad, bad                  06/11/08
CLOSED http://bugs.python.org/issue3084    created  skip.montanaro            
                                                                               

chapter 17.1.3.5 'Replacing os.popen*' in the Python library ref 06/11/08
       http://bugs.python.org/issue3085    created  afoo                      
                                                                               

sys.maxsize not available by using the latest Win32 build        06/12/08
CLOSED http://bugs.python.org/issue3086    created  giampaolo.rodola          
                                                                               

Clean up Demos and Tools                                         06/12/08
       http://bugs.python.org/issue3087    created  benjamin.peterson         
       easy                                                                    

test_multiprocessing hangs on OS X 10.5.3                        06/12/08
       http://bugs.python.org/issue3088    created  barry                     
                                                                               

All 3.0 buildbots are red                                        06/12/08
       http://bugs.python.org/issue3089    created  barry                     
                                                                               

ARCHFLAGS parsing/concatenation in unixccompiler.py breaks when  06/12/08
       http://bugs.python.org/issue3090    created  jnoller                   
       patch                                                                   

Can't build datetime or time on py3k (r64171)                    06/12/08
CLOSED http://bugs.python.org/issue3091    created  skip.montanaro            
                                                                               

Wrong unicode size detection in pybench                          06/12/08
       http://bugs.python.org/issue3092    created  pitrou                    
       patch                                                                   

Namespace polution from multiprocessing                          06/12/08
       http://bugs.python.org/issue3093    created  Rhamphoryncus             
                                                                               

By default, HTTPSConnection should send header "Host: somehost"  06/12/08
       http://bugs.python.org/issue3094    created  steven.wong               
       patch                                                                   

multiprocessing initializes flags dict unsafely                  06/12/08
CLOSED http://bugs.python.org/issue3095    created  Rhamphoryncus             
       patch                                                                   

sphinx: sort warnings by filename                                06/12/08
CLOSED http://bugs.python.org/issue3096    created  mgedmin                   
       patch                                                                   

sphinx: config option for exclude_dirnames                       06/12/08
       http://bugs.python.org/issue3097    created  mgedmin                   
                                                                               

sys.sizeof test fails with wide unicode                          06/12/08
       http://bugs.python.org/issue3098    created  benjamin.peterson         
                                                                               

On windows, "import nul" always succeed                          06/13/08
       http://bugs.python.org/issue3099    created  amaury.forgeotdarc        
                                                                               

weakref subclass segfault                                        06/13/08
       http://bugs.python.org/issue3100    created  Rhamphoryncus             
       patch                                                                   

global function _add_one_to_C                                    06/13/08
       http://bugs.python.org/issue3101    created  loewis                    
                                                                               

ctypes defines global symbols                                    06/13/08
       http://bugs.python.org/issue3102    created  loewis                    
                                                                               

sqlite defines a few global symbols.                             06/13/08
       http://bugs.python.org/issue3103    created  loewis                    
                                                                               

overzealous garbage collector (dict)                             06/13/08
       http://bugs.python.org/issue3104    created  GreaseMonkey              
                                                                               



Issues Now Closed (53)
______________________

Crash on Windows if Python runs from a directory with umlauts     229 days
       http://bugs.python.org/issue1342    amaury.forgeotdarc        
       patch                                                                   

Thread local storage and PyGILState_* mucked up by os.fork()      174 days
       http://bugs.python.org/issue1683    benjamin.peterson         
                                                                               

test_crasher in test_struct uses 8 GB memory on 64 bit systems    114 days
       http://bugs.python.org/issue2137    gregory.p.smith           
       patch, 64bit, easy                                                      

Add a factorial function                                           83 days
       http://bugs.python.org/issue2138    rhettinger                
                                                                               

Update PEP 3000 with new release schedule                          85 days
       http://bugs.python.org/issue2330    georg.brandl              
                                                                               

Py3K warn against assigning to True/False                          83 days
       http://bugs.python.org/issue2349    benjamin.peterson         
       patch, 26backport                                                       

Remove all IRIX dependant modules from aifc module                 85 days
       http://bugs.python.org/issue2419    georg.brandl              
                                                                               

No way to disable socket timeouts in httplib, etc.                 72 days
       http://bugs.python.org/issue2451    facundobatista            
       patch                                                                   

Remove unused constants from optimized code objects                78 days
       http://bugs.python.org/issue2493    rhettinger                
       patch                                                                   

Exception state lives too long in 3.0                              29 days
       http://bugs.python.org/issue2507    benjamin.peterson         
       patch                                                                   

itertools.permutations docstring is misleading                     69 days
       http://bugs.python.org/issue2536    georg.brandl              
                                                                               

Unpickling of range objects fail in Py3k                           63 days
       http://bugs.python.org/issue2582    alexandre.vassalotti      
       patch                                                                   

Descriptor instance attributes not interpreted consistently        62 days
       http://bugs.python.org/issue2605    gvanrossum                
                                                                               

Tile module: Add support for themed widgets                        58 days
       http://bugs.python.org/issue2618    benjamin.peterson         
                                                                               

repr() should not escape non-ASCII characters                      59 days
       http://bugs.python.org/issue2630    ishimoto                  
       patch                                                                   

__exit__ silences the active exception                             30 days
       http://bugs.python.org/issue2833    benjamin.peterson         
                                                                               

Remove cl usage from aifc                                          25 days
       http://bugs.python.org/issue2847    georg.brandl              
       patch                                                                   

Remove mimetools usage from the stdlib                             30 days
       http://bugs.python.org/issue2848    benjamin.peterson         
       patch                                                                   

Remove usage of rfc822 from the stdlib                             30 days
       http://bugs.python.org/issue2849    benjamin.peterson         
       patch                                                                   

rewrite test_struct as a unittest                                  24 days
       http://bugs.python.org/issue2911    benjamin.peterson         
       patch, easy                                                             

let platform.uname try harder                                      26 days
       http://bugs.python.org/issue2912    benjamin.peterson         
       patch, easy                                                             

merge pickle and cPickle in 3.0                                     1 days
       http://bugs.python.org/issue2917    alexandre.vassalotti      
       patch                                                                   

Merge StringIO/cStringIO in 3.0                                    23 days
       http://bugs.python.org/issue2918    alexandre.vassalotti      
       patch                                                                   

test_pydoc fails in trunk                                          15 days
       http://bugs.python.org/issue2976    benjamin.peterson         
                                                                               

PyNumberMethods has left-over fields in Py3                        10 days
       http://bugs.python.org/issue2997    benjamin.peterson         
       patch                                                                   

disutils fails with GNU ld (GNU Binutils) 2.18.50.20080523         12 days
       http://bugs.python.org/issue3013    georg.brandl              
       patch, easy                                                             

Lexical exception handlers                                         10 days
       http://bugs.python.org/issue3021    benjamin.peterson         
       patch                                                                   

tokenize module: normal lines, not "logical"                        6 days
       http://bugs.python.org/issue3028    noam                      
                                                                               

free list management - list, dict, set                              5 days
       http://bugs.python.org/issue3029    gregory.p.smith           
       patch, patch                                                            

Add PEP 8 compliant aliases to threading module                     6 days
       http://bugs.python.org/issue3042    benjamin.peterson         
       patch                                                                   

Some 3k sizeof tests fail                                           1 days
       http://bugs.python.org/issue3049    loewis                    
                                                                               

Implement PEP 371: multiprocessing module                           0 days
       http://bugs.python.org/issue3050    jnoller                   
       patch                                                                   

Mac Modules failing to compile                                      1 days
       http://bugs.python.org/issue3052    benjamin.peterson         
                                                                               

2.6 abc fixes                                                       0 days
       http://bugs.python.org/issue3057    georg.brandl              
       patch, 26backport                                                       

Removing .decode calls from Lib/calendar.py                         1 days
       http://bugs.python.org/issue3059    georg.brandl              
       patch                                                                   

Warn about tuple parameters                                         1 days
       http://bugs.python.org/issue3060    benjamin.peterson         
       patch                                                                   

memory leak in random number generation                             0 days
       http://bugs.python.org/issue3063    tim_one                   
                                                                               

new turtle module for Python 3.0                                    1 days
       http://bugs.python.org/issue3064    loewis                    
       patch                                                                   

Fix pickling bug of collections.namedtuple                          0 days
       http://bugs.python.org/issue3065    rhettinger                
       patch, patch                                                            

Assignment to list of lists gives incorrect result                  0 days
       http://bugs.python.org/issue3072    loewis                    
                                                                               

test_asyncore is failing                                            0 days
       http://bugs.python.org/issue3074    josiahcarlson             
                                                                               

test_xmlrpc_net fails                                               0 days
       http://bugs.python.org/issue3076    benjamin.peterson         
                                                                               

test_multiprocessing broken                                         0 days
       http://bugs.python.org/issue3082    gregory.p.smith           
                                                                               

sys.exit() called from optparse - bad, bad, bad                     0 days
       http://bugs.python.org/issue3084    skip.montanaro            
                                                                               

sys.maxsize not available by using the latest Win32 build           1 days
       http://bugs.python.org/issue3086    benjamin.peterson         
                                                                               

Can't build datetime or time on py3k (r64171)                       0 days
       http://bugs.python.org/issue3091    skip.montanaro            
                                                                               

multiprocessing initializes flags dict unsafely                     0 days
       http://bugs.python.org/issue3095    Rhamphoryncus             
       patch                                                                   

sphinx: sort warnings by filename                                   0 days
       http://bugs.python.org/issue3096    georg.brandl              
       patch                                                                   

SimpleHTTPServer reports wrong content-length for text files     1677 days
       http://bugs.python.org/issue839496  gagenellina               
       patch                                                                   

winerror module                                                   725 days
       http://bugs.python.org/issue1505257 giampaolo.rodola          
       patch                                                                   

__unicode__ breaks for exception class objects                    645 days
       http://bugs.python.org/issue1551432 davidfraser               
                                                                               

urlparse.urlunparse forms file urls incorrectly                   387 days
       http://bugs.python.org/issue1722348 georg.brandl              
                                                                               

"%d" format handling for long values                              353 days
       http://bugs.python.org/issue1742669 pixelbeat                 
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 24 Implement PEP 371: multiprocessing module                          0 days
closed  http://bugs.python.org/issue3050   

 17 let platform.uname try harder                                     26 days
closed  http://bugs.python.org/issue2912   

 17 Error when printing an exception containing a Unicode string      74 days
open    http://bugs.python.org/issue2517   

 14 test_multiprocessing hangs on OS X 10.5.3                          1 days
open    http://bugs.python.org/issue3088   

 14 heapq change breaking compatibility                                4 days
open    http://bugs.python.org/issue3051   

 12 Remove mimetools usage from the stdlib                            30 days
closed  http://bugs.python.org/issue2848   

 10 New class special method lookup change                          2027 days
open    http://bugs.python.org/issue643841 

 10 weakref subclass segfault                                          1 days
open    http://bugs.python.org/issue3100   

  9 memory leak in random number generation                            0 days
closed  http://bugs.python.org/issue3063   

  9 Lexical exception handlers                                        10 days
closed  http://bugs.python.org/issue3021   




From skip at pobox.com  Fri Jun 13 18:21:15 2008
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 13 Jun 2008 11:21:15 -0500
Subject: [Python-Dev] multiprocessing source not "Unix clean"
Message-ID: <18514.40571.3934.984310@montanaro-dyndns-org.local>

In trying to solve a build problem with the multiprocessing code on
Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all
appear to have Windows line endings.  Should those maybe be stripped to
conform to the other Python source?

FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does define
_SEM_VALUE_MAX in sys/params.h.

    .../Modules/_multiprocessing/multiprocessing.c: In function 'init_multiprocessing':
    .../Modules/_multiprocessing/multiprocessing.c:253: error: 'SEM_VALUE_MAX' undeclared (first use in this function)
    .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each undeclared identifier is reported only once
    .../Modules/_multiprocessing/multiprocessing.c:253: error: for each function it appears in.)

On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX.  I used
a little cpp action to define it:

    #ifndef SEM_VALUE_MAX
    #  ifdef _SEM_VALUE_MAX
    #    define SEM_VALUE_MAX _SEM_VALUE_MAX
    #  else
    #    define SEM_VALUE_MAX INT_MAX
    #  endif
    #endif

Skip

From skip at pobox.com  Fri Jun 13 18:29:07 2008
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 13 Jun 2008 11:29:07 -0500
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <485235B4.6030401@gmail.com>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
	<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
	<18513.16097.394123.312491@montanaro-dyndns-org.local>
	<485235B4.6030401@gmail.com>
Message-ID: <18514.41043.913466.680598@montanaro-dyndns-org.local>


    Nick> Have you looked at what the methods we're proposing to turn into
    Nick> properties are actually doing?:

    Nick>      def getName(self):
    Nick>          assert self.__initialized, "Thread.__init__() not called"
    Nick>          return self.__name
    ...
    Nick> Checking whether or not an Event is set is hardly an expensive
    Nick> operation...

Precisely my point.  The fact that the body of the method does almost
nothing means that the cost of method lookup and function call will likely
be much greater than the actual work being done.

Skip

From python at rcn.com  Fri Jun 13 18:40:16 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 13 Jun 2008 09:40:16 -0700
Subject: [Python-Dev] PEP 8 and optional underscores
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1><222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org><e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com><367889A5-6BD0-4106-948D-B5B181D4466D@python.org><18513.16097.394123.312491@montanaro-dyndns-org.local><485235B4.6030401@gmail.com>
	<18514.41043.913466.680598@montanaro-dyndns-org.local>
Message-ID: <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1>

>    Nick>      def getName(self):
>    Nick>          assert self.__initialized, "Thread.__init__() not called"
>    Nick>          return self.__name

Why is __name private to begin with?  Any reason for the getters and setters?  Why isn't this just an attribute?


Raymond

From musiccomposition at gmail.com  Fri Jun 13 18:42:12 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 13 Jun 2008 11:42:12 -0500
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
	<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
	<18513.16097.394123.312491@montanaro-dyndns-org.local>
	<485235B4.6030401@gmail.com>
	<18514.41043.913466.680598@montanaro-dyndns-org.local>
	<89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1>
Message-ID: <1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com>

On Fri, Jun 13, 2008 at 11:40 AM, Raymond Hettinger <python at rcn.com> wrote:
>>   Nick>      def getName(self):
>>   Nick>          assert self.__initialized, "Thread.__init__() not called"
>>   Nick>          return self.__name
>
> Why is __name private to begin with?  Any reason for the getters and
> setters?  Why isn't this just an attribute?

In 3.x, it's just an attribute.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From skip at pobox.com  Fri Jun 13 18:58:10 2008
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 13 Jun 2008 11:58:10 -0500
Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins
In-Reply-To: <1afaf6160806130747k4835ab2cmb99ab3b87136142d@mail.gmail.com>
References: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>
	<1afaf6160806130747k4835ab2cmb99ab3b87136142d@mail.gmail.com>
Message-ID: <18514.42786.53127.271780@montanaro-dyndns-org.local>


    >> Let's all remember this and make sure not to drop "code bombs" on
    >> each other. :-)

    Benjamin> Ok! I'd like everybody to know that I'm working on Python
    Benjamin> tests on a Bazaar branch. [1] (That's only a bzr co
    Benjamin> --lightweight away.) Go ahead and send me angry emails about
    Benjamin> how it's totally wrong, and your cat could do better.

No problem, as long as it doesn't take five months and touch every file in
the distribution. <wink>

I don't think Ben was suggesting there is never a good reason to work in
relative isolation.  I interpreted it as a rule of thumb for joint
projects.  In your case, while it can have a wide-ranging effect on the unit
test code it's likely that you will be able to manage syncing from
trunk/py3k just fine and you already have plenty of examples of how things
work now and how they should work when you're done.  That said, if you can
commit in chunks that would probably still be a win.

Where I work most of the projects are one-person, due largely to the
business we are in and thus how management wants things done.  I think that
contributes to a number of issues, the most problematic being that APIs are
developed largely in isolation, often with the needs of only one or two
applications in mind.  That creates problems for people who come along later
and need to use those APIs, only to discover that they are fundamentally
flawed in some way.

Skip

From guido at python.org  Fri Jun 13 19:12:01 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 10:12:01 -0700
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <A0BCACBC-B884-4752-9E9A-C03500C05298@python.org>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
	<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
	<18513.16097.394123.312491@montanaro-dyndns-org.local>
	<A0BCACBC-B884-4752-9E9A-C03500C05298@python.org>
Message-ID: <ca471dc20806131012h5be4d75cycc0fe346c3b1b11e@mail.gmail.com>

On Thu, Jun 12, 2008 at 8:40 AM, Barry Warsaw <barry at python.org> wrote:
> Ideally, I would like for those considerations [i.e. whether an access is expensive]
> to not enter into the API
> design.  I'd rather keep it clean, with sufficient documentation to give
> hints about any additional costs involved.  Logically .alive is a property,
> so I'd like to write it that way.

I beg to differ. Ideally, API design should suggest which operations
are efficient and which ones aren't, in order to (subconsciously) help
the developer develop a mental model about the performance
characteristics of an API.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun 13 19:14:14 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 10:14:14 -0700
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
	<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
	<18513.16097.394123.312491@montanaro-dyndns-org.local>
	<485235B4.6030401@gmail.com>
	<18514.41043.913466.680598@montanaro-dyndns-org.local>
	<89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1>
	<1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com>
Message-ID: <ca471dc20806131014g25fcad2du4401e787244601f8@mail.gmail.com>

On Fri, Jun 13, 2008 at 9:42 AM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Fri, Jun 13, 2008 at 11:40 AM, Raymond Hettinger <python at rcn.com> wrote:
>>>   Nick>      def getName(self):
>>>   Nick>          assert self.__initialized, "Thread.__init__() not called"
>>>   Nick>          return self.__name
>>
>> Why is __name private to begin with?  Any reason for the getters and
>> setters?  Why isn't this just an attribute?
>
> In 3.x, it's just an attribute.

Oh, is it? Where's the PEP with the API redesign? Did I miss some kind
of decision-making process, weighing compatibility concerns against
other issues?

Really, I think this all went way too fast, and should be rolled back
and discussed properly *in detail* first.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun 13 19:17:22 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 10:17:22 -0700
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <18514.40571.3934.984310@montanaro-dyndns-org.local>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
Message-ID: <ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>

On Fri, Jun 13, 2008 at 9:21 AM,  <skip at pobox.com> wrote:
> In trying to solve a build problem with the multiprocessing code on
> Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all
> appear to have Windows line endings.  Should those maybe be stripped to
> conform to the other Python source?

Ow. definitely.

> FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does define
> _SEM_VALUE_MAX in sys/params.h.
>
>    .../Modules/_multiprocessing/multiprocessing.c: In function 'init_multiprocessing':
>    .../Modules/_multiprocessing/multiprocessing.c:253: error: 'SEM_VALUE_MAX' undeclared (first use in this function)
>    .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each undeclared identifier is reported only once
>    .../Modules/_multiprocessing/multiprocessing.c:253: error: for each function it appears in.)
>
> On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX.  I used
> a little cpp action to define it:
>
>    #ifndef SEM_VALUE_MAX
>    #  ifdef _SEM_VALUE_MAX
>    #    define SEM_VALUE_MAX _SEM_VALUE_MAX
>    #  else
>    #    define SEM_VALUE_MAX INT_MAX
>    #  endif
>    #endif

Does this enable you to submit a patch?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From musiccomposition at gmail.com  Fri Jun 13 19:18:08 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 13 Jun 2008 12:18:08 -0500
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <ca471dc20806131014g25fcad2du4401e787244601f8@mail.gmail.com>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>
	<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>
	<18513.16097.394123.312491@montanaro-dyndns-org.local>
	<485235B4.6030401@gmail.com>
	<18514.41043.913466.680598@montanaro-dyndns-org.local>
	<89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1>
	<1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com>
	<ca471dc20806131014g25fcad2du4401e787244601f8@mail.gmail.com>
Message-ID: <1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com>

On Fri, Jun 13, 2008 at 12:14 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Jun 13, 2008 at 9:42 AM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> On Fri, Jun 13, 2008 at 11:40 AM, Raymond Hettinger <python at rcn.com> wrote:
>>>>   Nick>      def getName(self):
>>>>   Nick>          assert self.__initialized, "Thread.__init__() not called"
>>>>   Nick>          return self.__name
>>>
>>> Why is __name private to begin with?  Any reason for the getters and
>>> setters?  Why isn't this just an attribute?
>>
>> In 3.x, it's just an attribute.
>
> Oh, is it? Where's the PEP with the API redesign? Did I miss some kind
> of decision-making process, weighing compatibility concerns against
> other issues?

meaning that it only has one underscore. They methods still live.

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From walter at livinglogic.de  Fri Jun 13 19:20:03 2008
From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Fri, 13 Jun 2008 19:20:03 +0200
Subject: [Python-Dev] [Python-3000]   Betas today - I hope
In-Reply-To: <48527597.4090206@livinglogic.de>
References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org>	<484FE1E9.2080904@egenix.com>	<484FEC2C.6070409@livinglogic.de>	<485003F8.40803@egenix.com>	<485139EA.1000502@livinglogic.de>	<48514428.7040001@egenix.com>
	<48523EA5.2070407@livinglogic.de>	<48524899.2030609@egenix.com>
	<48527597.4090206@livinglogic.de>
Message-ID: <4852AC43.1090304@livinglogic.de>

Walter D?rwald wrote:

> [...] 
>> Sure, we could do that, but please use a different name,
>> e.g. .encodeall() and .decodeall() - .encode() and .decode()
>> are already stateles (and so would the new methods be), so
>> "stateless" isn't all that meaningful in this context.
> 
> I like the names encodeall/decodeall!
> 
>> We could also add such a check to the PyCodec_Encode() and _Decode()
>> functions. They currently do not apply the above check.
>>
>> In Python, those two functions are exposed as codecs.encode()
>> and codecs.decode().
> 
> This change will probably have to wait for the 2.7 cycle.

BTW, what I noticed is that the unicode-internal codec seems to be broken:

 >>> import codecs
 >>> codecs.getencoder("unicode-internal")(u"abc")
('a\x00b\x00c\x00', 6)

I would have expected it to return:

 >>> import codecs
 >>> codecs.getencoder("unicode-internal")(u"abc")
('a\x00b\x00c\x00', 3)

Servus,
    Walter


From guido at python.org  Fri Jun 13 19:25:41 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 10:25:41 -0700
Subject: [Python-Dev] [Python-3000] [Python-3000-checkins] r64217 - in
	python/branches/py3k/Lib: bsddb/test/test_associate.py
	bsddb/test/test_join.py bsddb/test/test_lock.py
	bsddb/test/test_thread.py idlelib/rpc.py idlelib/run.py
	socketserver.py test/test_threadedtemp
Message-ID: <ca471dc20806131025p53b39256ob3e181283f65bea3@mail.gmail.com>

On Fri, Jun 13, 2008 at 6:07 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Steve Holden wrote:
>>
>> Guido van Rossum wrote:
>>>
>>> It's water under the bridge now, but IMO it was too rash to *remove*
>>> the old threading API from Py3k, and doubly rash to do so one day
>>> before the beta release. Running up to a release (whether alpha, beta
>>> or final) we should practice extra restraint, not rush to get things
>>> in right before the deadline. Let's all be more careful the rest of
>>> this release cycle! (I think it wasn't just Benjamin who raced to get
>>> things in...)
>>
>> Well, it wouldn't be "adding a new feature" to reinstate the old API for
>> beta two, would it, as long as we retain the new one too? It does seem that
>> change was a little precipitate.
>
> Although if we weren't actually planning on removing the old API in 3.0, I'm
> a little confused as to why we were adding Py3k warnings to it in 2.6...

It all went way too fast for anyone to think about the decision, so I
suspect these two issues are directly related.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun 13 19:30:59 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 10:30:59 -0700
Subject: [Python-Dev] [Python-3000] [Python-3000-checkins] r64217 - in
	python/branches/py3k/Lib: bsddb/test/test_associate.py
	bsddb/test/test_join.py bsddb/test/test_lock.py
	bsddb/test/test_thread.py idlelib/rpc.py idlelib/run.py
	socketserver.py test/test_threadedtemp
Message-ID: <ca471dc20806131030w104111cbmd3c148f0ca73dcd0@mail.gmail.com>

On Thu, Jun 12, 2008 at 11:32 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Guido van Rossum schrieb:
>>
>> It's water under the bridge now, but IMO it was too rash to *remove*
>> the old threading API from Py3k, and doubly rash to do so one day
>> before the beta release. Running up to a release (whether alpha, beta
>> or final) we should practice extra restraint, not rush to get things
>> in right before the deadline. Let's all be more careful the rest of
>> this release cycle! (I think it wasn't just Benjamin who raced to get
>> things in...)
>
> Also, for any method or module renaming, there is no way around a full
> grep through the code base to really catch all uses of the old API.

Eh? Even a full grep won't reveal 3rd party uses of the old API. While
3.x is intentionally breaking some things, this is not (and was never
meant to be) as a blanket approval to break any API we feel like
breaking. Each specific breakage must have a good motivation, e.g. the
old API is confusing, or inefficient, or disallows certain uses --
non-PEP-8-conformance alone is not enough! In addition, each specific
breakage must be weighed against the cost of updating 3rd party code,
and whether a good (or at least fair) automatic conversion can be
added to 2to3.

> It may be tedious, especially with common names, but such bugs really
> must be avoided as they can easily be -- I still could find uses of
> old-style threading names, even in the stdlib. (Fixed in r64222.)

All the more reason not to rush into API renamings without due process
and lots of discussion. Let's err on the side of the status quo --
otherwise we'll never be ready for beta.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jnoller at gmail.com  Fri Jun 13 19:36:47 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 13 Jun 2008 13:36:47 -0400
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <18514.40571.3934.984310@montanaro-dyndns-org.local>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
Message-ID: <4222a8490806131036u708a5306lcd88bbd0e49e54aa@mail.gmail.com>

I don't know where the windows line endings came from honestly - I'm
on mac and linux boxes.

Yes, they should be stripped.

The solaris issue should be addressed in the setup.py section casing
the various sem implementations for the platforms

On Fri, Jun 13, 2008 at 12:21 PM,  <skip at pobox.com> wrote:
> In trying to solve a build problem with the multiprocessing code on
> Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all
> appear to have Windows line endings.  Should those maybe be stripped to
> conform to the other Python source?
>
> FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does define
> _SEM_VALUE_MAX in sys/params.h.
>
>    .../Modules/_multiprocessing/multiprocessing.c: In function 'init_multiprocessing':
>    .../Modules/_multiprocessing/multiprocessing.c:253: error: 'SEM_VALUE_MAX' undeclared (first use in this function)
>    .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each undeclared identifier is reported only once
>    .../Modules/_multiprocessing/multiprocessing.c:253: error: for each function it appears in.)
>
> On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX.  I used
> a little cpp action to define it:
>
>    #ifndef SEM_VALUE_MAX
>    #  ifdef _SEM_VALUE_MAX
>    #    define SEM_VALUE_MAX _SEM_VALUE_MAX
>    #  else
>    #    define SEM_VALUE_MAX INT_MAX
>    #  endif
>    #endif
>
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com
>

From musiccomposition at gmail.com  Fri Jun 13 21:32:41 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 13 Jun 2008 14:32:41 -0500
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
Message-ID: <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>

On Fri, Jun 13, 2008 at 12:17 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Jun 13, 2008 at 9:21 AM,  <skip at pobox.com> wrote:
>> In trying to solve a build problem with the multiprocessing code on
>> Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all
>> appear to have Windows line endings.  Should those maybe be stripped to
>> conform to the other Python source?
>
> Ow. definitely.

I've converted the line endings in the trunk and py3k.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From guido at python.org  Fri Jun 13 22:02:18 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 13:02:18 -0700
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>
Message-ID: <ca471dc20806131302q21d93a0bx4c7c5b9544d25997@mail.gmail.com>

On Fri, Jun 13, 2008 at 12:32 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Fri, Jun 13, 2008 at 12:17 PM, Guido van Rossum <guido at python.org> wrote:
>> On Fri, Jun 13, 2008 at 9:21 AM,  <skip at pobox.com> wrote:
>>> In trying to solve a build problem with the multiprocessing code on
>>> Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all
>>> appear to have Windows line endings.  Should those maybe be stripped to
>>> conform to the other Python source?
>>
>> Ow. definitely.
>
> I've converted the line endings in the trunk and py3k.

Thanks! Can we have a post-mortem of this? How did they get in there
in the first place without anybody noticing?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Fri Jun 13 22:06:52 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 13 Jun 2008 13:06:52 -0700
Subject: [Python-Dev] Advice on numbers.py implementation of binary mixins.
Message-ID: <1984993647E144708D965F1C7AED77C6@RaymondLaptop1>

PEP-3141 outlines an approach to writing binary operators to allow the
right operand to override the operation if the left operand inherits
the operation from the ABC.

Here is my first approximation at how to write them for the Integral mixins:

class Integral(Rational):

    def __and__(self, other):
        if isinstance(other, (type(self), int, long)):         # XXX
            return int(self) & int(other)
        return NotImplemented

    def __rand__(self, other):
        if isinstance(other, Integral):
            return int(other) & int(self)
        return NotImplemented

The question for the group is what to put on the XXX line.

1. Limit it to type(self).  This approach claims the least knowledge about other types and uses their __rand__ methods().

2. Use type(self), int, and long.  This approach says that we know that ints and longs can be reasonably converted to an int; 
however, it means that any int/long subtype cannot use __rand__ to override the mixin's __and__.

3. Emulate the PEP as closely as possible and accomodate subclassing without changing behaviors.  This approach is a bit complex:

        knowntypes = (set(type(self).__mro__)
                               & set(type(other.__mro__))
                               - set(type(Integral.__mro__))
                               | set([int, long]))
        if isinstance(other, tuple(knowntypes)):

I got this by analyzing the possible paths in an inheritance tree:

   class P(Integral): pass
   class Q(P): pass
   class R(Q): pass
   r = R(3)
   class S(Q): def __rand__(s,o) ...
   s = S(6)
   class T(Integral): def __rand__(s,o) ...
   t = T(5)

With r&t, there is no common ancestor below Integral, so we want
Integral.__and__() to return NotImplemented and allow T.__rand__()
to do its thing.

With r&s, both r and s share Q as a common ancenstor.
By using the mixin and not overriding __and__(), Q specifies that __and__()
should mean int(r)&int(s) whenever isinstance(r,Q) and isinstance(s,Q).

Approaches 1 & 2 don't search the mro for shared ancestors below
the level of Integral.  So, when evaluating r&s, Integral.__and__()
only checks that r is not an instance of S, int or long, and it
erroneously returns NotImplemented , leaving s.__rand__() to take over.

What do you guys think?


Raymond 


From musiccomposition at gmail.com  Fri Jun 13 22:09:09 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 13 Jun 2008 15:09:09 -0500
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <ca471dc20806131302q21d93a0bx4c7c5b9544d25997@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>
	<ca471dc20806131302q21d93a0bx4c7c5b9544d25997@mail.gmail.com>
Message-ID: <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com>

On Fri, Jun 13, 2008 at 3:02 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Jun 13, 2008 at 12:32 PM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> I've converted the line endings in the trunk and py3k.
>
> Thanks! Can we have a post-mortem of this? How did they get in there
> in the first place without anybody noticing?

svn cat tells me that they were CRLF when I initially imported them. I
wonder if this has something to do with the patch utility I used to
apply Jesse's patch.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From guido at python.org  Fri Jun 13 22:21:48 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 13:21:48 -0700
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>
	<ca471dc20806131302q21d93a0bx4c7c5b9544d25997@mail.gmail.com>
	<1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com>
Message-ID: <ca471dc20806131321p38021757meb7acf7f2303b7d7@mail.gmail.com>

On Fri, Jun 13, 2008 at 1:09 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Fri, Jun 13, 2008 at 3:02 PM, Guido van Rossum <guido at python.org> wrote:
>> On Fri, Jun 13, 2008 at 12:32 PM, Benjamin Peterson
>> <musiccomposition at gmail.com> wrote:
>>> I've converted the line endings in the trunk and py3k.
>>
>> Thanks! Can we have a post-mortem of this? How did they get in there
>> in the first place without anybody noticing?
>
> svn cat tells me that they were CRLF when I initially imported them. I
> wonder if this has something to do with the patch utility I used to
> apply Jesse's patch.

I'm guessing Jesse uploaded the patch from a Windows box.

It would be good to have something in the toolchain to change this.
Perhaps a submit trigger (like the whitespace check) can look for
this?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From musiccomposition at gmail.com  Fri Jun 13 22:23:19 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 13 Jun 2008 15:23:19 -0500
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <ca471dc20806131321p38021757meb7acf7f2303b7d7@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>
	<ca471dc20806131302q21d93a0bx4c7c5b9544d25997@mail.gmail.com>
	<1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com>
	<ca471dc20806131321p38021757meb7acf7f2303b7d7@mail.gmail.com>
Message-ID: <1afaf6160806131323u12ee0567qd9b72d85b0fd5426@mail.gmail.com>

On Fri, Jun 13, 2008 at 3:21 PM, Guido van Rossum <guido at python.org> wrote:
>
> I'm guessing Jesse uploaded the patch from a Windows box.

Actually he just told me that he uploaded it from a Mac, so it's my fault. :)

>
> It would be good to have something in the toolchain to change this.
> Perhaps a submit trigger (like the whitespace check) can look for
> this?

I'll figure out how to do that if you give me shell access. ;)



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From guido at python.org  Fri Jun 13 22:24:38 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 13:24:38 -0700
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <1afaf6160806131323u12ee0567qd9b72d85b0fd5426@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>
	<ca471dc20806131302q21d93a0bx4c7c5b9544d25997@mail.gmail.com>
	<1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com>
	<ca471dc20806131321p38021757meb7acf7f2303b7d7@mail.gmail.com>
	<1afaf6160806131323u12ee0567qd9b72d85b0fd5426@mail.gmail.com>
Message-ID: <ca471dc20806131324i3055a471wadef0b3f2d8b0084@mail.gmail.com>

On Fri, Jun 13, 2008 at 1:23 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Fri, Jun 13, 2008 at 3:21 PM, Guido van Rossum <guido at python.org> wrote:
>>
>> I'm guessing Jesse uploaded the patch from a Windows box.
>
> Actually he just told me that he uploaded it from a Mac, so it's my fault. :)

You're on Windows?!

>> It would be good to have something in the toolchain to change this.
>> Perhaps a submit trigger (like the whitespace check) can look for
>> this?
>
> I'll figure out how to do that if you give me shell access. ;)

Talk to MvL.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From musiccomposition at gmail.com  Fri Jun 13 22:26:32 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 13 Jun 2008 15:26:32 -0500
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <ca471dc20806131324i3055a471wadef0b3f2d8b0084@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>
	<ca471dc20806131302q21d93a0bx4c7c5b9544d25997@mail.gmail.com>
	<1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com>
	<ca471dc20806131321p38021757meb7acf7f2303b7d7@mail.gmail.com>
	<1afaf6160806131323u12ee0567qd9b72d85b0fd5426@mail.gmail.com>
	<ca471dc20806131324i3055a471wadef0b3f2d8b0084@mail.gmail.com>
Message-ID: <1afaf6160806131326s50a5e365q96485d325dc57486@mail.gmail.com>

On Fri, Jun 13, 2008 at 3:24 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Jun 13, 2008 at 1:23 PM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>>
>> Actually he just told me that he uploaded it from a Mac, so it's my fault. :)
>
> You're on Windows?!

Me!? I cringe at the accusation. No, I just think I have emacs set up wrong.

>
>>> It would be good to have something in the toolchain to change this.
>>> Perhaps a submit trigger (like the whitespace check) can look for
>>> this?
>>
>> I'll figure out how to do that if you give me shell access. ;)
>
> Talk to MvL.

I will do.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From benji at benjiyork.com  Fri Jun 13 22:28:29 2008
From: benji at benjiyork.com (Benji York)
Date: Fri, 13 Jun 2008 16:28:29 -0400
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <ca471dc20806131321p38021757meb7acf7f2303b7d7@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>
	<ca471dc20806131302q21d93a0bx4c7c5b9544d25997@mail.gmail.com>
	<1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com>
	<ca471dc20806131321p38021757meb7acf7f2303b7d7@mail.gmail.com>
Message-ID: <e5fff6640806131328n3f4cc7br1fa114f5363d8c4f@mail.gmail.com>

On Fri, Jun 13, 2008 at 4:21 PM, Guido van Rossum <guido at python.org> wrote:
> On Fri, Jun 13, 2008 at 1:09 PM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> On Fri, Jun 13, 2008 at 3:02 PM, Guido van Rossum <guido at python.org> wrote:
>>> On Fri, Jun 13, 2008 at 12:32 PM, Benjamin Peterson
>>> <musiccomposition at gmail.com> wrote:
>>>> I've converted the line endings in the trunk and py3k.
>>>
>>> Thanks! Can we have a post-mortem of this? How did they get in there
>>> in the first place without anybody noticing?
>>
>> svn cat tells me that they were CRLF when I initially imported them. I
>> wonder if this has something to do with the patch utility I used to
>> apply Jesse's patch.
>
> I'm guessing Jesse uploaded the patch from a Windows box.
>
> It would be good to have something in the toolchain to change this.

Subversion can be configured to normalize line endings, either manually
through properties, or automatically via ~/.subversion/config:

[miscellany]
enable-auto-props = yes

[auto-props]
*.c = svn:eol-style=native
*.h = svn:eol-style=native

etc.
-- 
Benji York

From guido at python.org  Fri Jun 13 22:28:34 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 13 Jun 2008 13:28:34 -0700
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <1afaf6160806131326s50a5e365q96485d325dc57486@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>
	<ca471dc20806131302q21d93a0bx4c7c5b9544d25997@mail.gmail.com>
	<1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com>
	<ca471dc20806131321p38021757meb7acf7f2303b7d7@mail.gmail.com>
	<1afaf6160806131323u12ee0567qd9b72d85b0fd5426@mail.gmail.com>
	<ca471dc20806131324i3055a471wadef0b3f2d8b0084@mail.gmail.com>
	<1afaf6160806131326s50a5e365q96485d325dc57486@mail.gmail.com>
Message-ID: <ca471dc20806131328s2437400fp69f920e49bb68de8@mail.gmail.com>

On Fri, Jun 13, 2008 at 1:26 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Fri, Jun 13, 2008 at 3:24 PM, Guido van Rossum <guido at python.org> wrote:
>> On Fri, Jun 13, 2008 at 1:23 PM, Benjamin Peterson
>> <musiccomposition at gmail.com> wrote:
>>>
>>> Actually he just told me that he uploaded it from a Mac, so it's my fault. :)
>>
>> You're on Windows?!
>
> Me!? I cringe at the accusation. No, I just think I have emacs set up wrong.

Ow. Are you sure you haven't made this mistake in other situations?
Please do try to debug your setup and avoid repeats!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From benji at benjiyork.com  Fri Jun 13 22:31:52 2008
From: benji at benjiyork.com (Benji York)
Date: Fri, 13 Jun 2008 16:31:52 -0400
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <e5fff6640806131328n3f4cc7br1fa114f5363d8c4f@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com>
	<ca471dc20806131302q21d93a0bx4c7c5b9544d25997@mail.gmail.com>
	<1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com>
	<ca471dc20806131321p38021757meb7acf7f2303b7d7@mail.gmail.com>
	<e5fff6640806131328n3f4cc7br1fa114f5363d8c4f@mail.gmail.com>
Message-ID: <e5fff6640806131331p1997de4aib9c399ce24aebf96@mail.gmail.com>

On Fri, Jun 13, 2008 at 4:28 PM, Benji York <benji at benjiyork.com> wrote:
> Subversion can be configured to normalize line endings, either manually
> through properties, or automatically via ~/.subversion/config:

After sending this I though "surely this is in the developer docs", and
indeed it is:

http://www.python.org/dev/faq/#what-configuration-settings-should-i-use
-- 
Benji York

From barry at python.org  Sat Jun 14 00:22:42 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 13 Jun 2008 18:22:42 -0400
Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins
In-Reply-To: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>
References: <ca471dc20806122041v46f4d8f7w10022c50037012dc@mail.gmail.com>
Message-ID: <0CC2A0C7-BA24-4C1E-B85E-BCA0B771B00B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 12, 2008, at 11:41 PM, Guido van Rossum wrote:

> My colleague and SVN developer Ben Sussman-Collins occasionally blogs
> about the social side of (mostly open source) software development. He
> just posted a new one that struck a chord:
>
> http://blog.red-bean.com/sussman/?p=96
>
> The story's main moral: submit your code for review early and often;
> work in a branch if you need to, but don't hide your code from review
> in a local repository until it's "perfect".
>
> Let's all remember this and make sure not to drop "code bombs" on each
> other. :-)

Very interesting article.  I'm short on time and don't want to rant  
(though I likely will ;), but I whole-heartedly agree with the moral  
of the story!

I disagree with some of the details though.  I actually think that a  
dvcs is /better/ suited to transparency, when used right, and when  
coupled with a public open code hosting facility.  Sure, a lot depends  
on social engineering, and I agree with Ben that the tools make a  
difference, I just think that a good dvcs solves more problems than it  
creates.

Also, there are a few things we do at my job that I think contribute  
significantly and positively to our productivity, quality and sense of  
shared community code.  Briefly:

* pre-implementation "calls" - you do not start hacking code until  
you've discussed your design or approach with at least one other  
person, either over the phone or on irc (preferably the former).  Yes,  
there are exceptions but they are discouraged.  This means that when  
you actually sit in front of your editor, you have a much better idea  
of what you are trying to accomplish.

* small branches - we have a strict limit on diffs no greater than 800  
lines.  Yes we have exceptions, but they are rare and pre-arranged.   
Having such a strict limit really forces you to be disciplined,  
organized and very effectively diffuses code bombs.

* everyone can see (lots of) everyone else's code - this is great  
because everyone needs some advice or guidance along the way.  If you  
get stuck, you can push a branch and I can pull it and look at it, run  
it, test it, even modify it and push my own branch for you to see.   
This is /much/ more effective than trading patches, and I don't see  
how this could even work without a dvcs.

* nothing lands without being reviewed - this is a hard and fast rule,  
no exceptions.  Someone else has to review your code, and most  
developers are also reviewers (we have a mentoring program to train  
new reviewers).  You get over the fear pretty quickly, and learn /a  
lot/ both by reviewing and getting reviewed.  Coding standards emerge,  
best practices are established, and overall team productivity goes way  
up.  Small branches are critical to this process, as is our goal of  
reviewing every branch within 24 hours of its submission.

* nothing lands without passing all tests - speaking from experience,  
this is the one thing I wish Python would adopt!  This means the trunk  
is /always/ releasable and stable.  The trade-off is that it can take  
quite a while for your branch to land once it's been approved, since  
this process is serialized and is dependent on full test suite  
execution time.  Python's challenge here is that what passes on one  
platform does not necessarily pass on another.  Still, if this week is  
any indication, passing on /any/ platform would be nice. ;)

I'm not saying Python can or should adopt these guidelines.  An open  
source volunteer project is different than a corporate environment,  
even if the latter is very open-source-y.  But it is worthwhile to  
continually evaluate and improve the process because over time, you  
definitely improve efficiency in ways that are happily adopted by the  
majority of the community.

- -Barry

P.S. I can't leave this without one little plug.  Some folks really  
like the model that a non-dvcs imposes on development, others thrive  
on the freedom a dvcs gives you.  Bazaar is flexible enough to support  
both models, even at the same time.  It's not either-or.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFLzMnEjvBPtnXfVAQLm3QQAptABUXBoWeshMJAvHno1IDMZavL9D2BG
q9d3wz8O5u2pbvuZyh44w4fhm7w7fscGvmPygifNbjPTVMeUYQUkunuEfWEIzf6B
f6hm1KQm5gi9B4eqSUh3ItIAjGAnkVnCJ8VHeRH/Hff9FZhHqPhtP26LBrecMoho
q0g80DrALB8=
=J936
-----END PGP SIGNATURE-----

From fuzzyman at voidspace.org.uk  Sat Jun 14 00:42:52 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 13 Jun 2008 23:42:52 +0100
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement?
Message-ID: <4852F7EC.6040403@voidspace.org.uk>

The Python FAQ has the following entry:

4.26   Why doesn't Python have a "with" statement like some other languages?

From:

http://www.python.org/doc/faq/general/

Even if the content is still relevant (I know nothing of the use of 
'with' outside Python), the entry probably needs at least clarifying now 
that Python does have a 'with' statement.

Michael Foord

-- 
http://www.ironpythoninaction.com/
http://www.theotherdelia.co.uk/
http://www.voidspace.org.uk/
http://www.ironpython.info/
http://www.resolverhacks.net/


From exarkun at divmod.com  Sat Jun 14 01:22:07 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 13 Jun 2008 19:22:07 -0400
Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins
In-Reply-To: <0CC2A0C7-BA24-4C1E-B85E-BCA0B771B00B@python.org>
Message-ID: <20080613232207.4714.1814579539.divmod.quotient.8878@ohm>

On Fri, 13 Jun 2008 18:22:42 -0400, Barry Warsaw <barry at python.org> wrote:
> [snip]
>
>* small branches - we have a strict limit on diffs no greater than 800 
>lines.  Yes we have exceptions, but they are rare and pre-arranged.   Having 
>such a strict limit really forces you to be disciplined,  organized and very 
>effectively diffuses code bombs.
>
>* everyone can see (lots of) everyone else's code - this is great  because 
>everyone needs some advice or guidance along the way.  If you  get stuck, 
>you can push a branch and I can pull it and look at it, run  it, test it, 
>even modify it and push my own branch for you to see.   This is /much/ more 
>effective than trading patches, and I don't see  how this could even work 
>without a dvcs.
>
>* nothing lands without being reviewed - this is a hard and fast rule,  no 
>exceptions.  Someone else has to review your code, and most  developers are 
>also reviewers (we have a mentoring program to train  new reviewers).  You 
>get over the fear pretty quickly, and learn /a  lot/ both by reviewing and 
>getting reviewed.  Coding standards emerge,  best practices are established, 
>and overall team productivity goes way  up.  Small branches are critical to 
>this process, as is our goal of  reviewing every branch within 24 hours of 
>its submission.
>
>* nothing lands without passing all tests - speaking from experience,  this 
>is the one thing I wish Python would adopt!  This means the trunk  is 
>/always/ releasable and stable.  The trade-off is that it can take  quite a 
>while for your branch to land once it's been approved, since  this process 
>is serialized and is dependent on full test suite  execution time.  Python's 
>challenge here is that what passes on one  platform does not necessarily 
>pass on another.  Still, if this week is  any indication, passing on /any/ 
>platform would be nice. ;)
>
>I'm not saying Python can or should adopt these guidelines.  An open  source 
>volunteer project is different than a corporate environment,  even if the 
>latter is very open-source-y.  But it is worthwhile to  continually evaluate 
>and improve the process because over time, you  definitely improve 
>efficiency in ways that are happily adopted by the  majority of the 
>community.

A big +1 on all these points.  I can also add that Twisted is developed
following many of these rules so it *can* work for open source volunteer
projects, if the developers want it to.

Jean-Paul

From ncoghlan at gmail.com  Sat Jun 14 03:23:19 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 14 Jun 2008 11:23:19 +1000
Subject: [Python-Dev] Advice on numbers.py implementation of binary
	mixins.
In-Reply-To: <1984993647E144708D965F1C7AED77C6@RaymondLaptop1>
References: <1984993647E144708D965F1C7AED77C6@RaymondLaptop1>
Message-ID: <48531D87.8020004@gmail.com>

Raymond Hettinger wrote:
> The question for the group is what to put on the XXX line.
> 
> 1. Limit it to type(self).  This approach claims the least knowledge 
> about other types and uses their __rand__ methods().
> 
> 2. Use type(self), int, and long.  This approach says that we know that 
> ints and longs can be reasonably converted to an int; however, it means 
> that any int/long subtype cannot use __rand__ to override the mixin's 
> __and__.
> 
> 3. Emulate the PEP as closely as possible and accomodate subclassing 
> without changing behaviors.  This approach is a bit complex:
> 
>        knowntypes = (set(type(self).__mro__)
>                               & set(type(other.__mro__))
>                               - set(type(Integral.__mro__))
>                               | set([int, long]))
>        if isinstance(other, tuple(knowntypes)):

4. As 2, but skip int/long subclasses:

   if type(other) in (int, long) or isinstance(other, type(self)):

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From skip at pobox.com  Sat Jun 14 06:31:34 2008
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 13 Jun 2008 23:31:34 -0500
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
Message-ID: <18515.18854.711188.716549@montanaro-dyndns-org.local>

>>>>> "Guido" == Guido van Rossum <guido at python.org> writes:

    Guido> On Fri, Jun 13, 2008 at 9:21 AM,  <skip at pobox.com> wrote:
    >> In trying to solve a build problem with the multiprocessing code on
    >> Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all
    >> appear to have Windows line endings.  Should those maybe be stripped to
    >> conform to the other Python source?

    Guido> Ow. definitely.

Yes.  If I have some time Saturday and nobody beats me to it I'll fix this.

    >> FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does
    >> define _SEM_VALUE_MAX in sys/params.h.
    >> 
    >> .../Modules/_multiprocessing/multiprocessing.c: In function 'init_multiprocessing':
    >> .../Modules/_multiprocessing/multiprocessing.c:253: error: 'SEM_VALUE_MAX' undeclared (first use in this function)
    >> .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each undeclared identifier is reported only once
    >> .../Modules/_multiprocessing/multiprocessing.c:253: error: for each function it appears in.)
    >> 
    >> On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX.  I used
    >> a little cpp action to define it:
    >> 
    >> #ifndef SEM_VALUE_MAX
    >> #  ifdef _SEM_VALUE_MAX
    >> #    define SEM_VALUE_MAX _SEM_VALUE_MAX
    >> #  else
    >> #    define SEM_VALUE_MAX INT_MAX
    >> #  endif
    >> #endif

    Guido> Does this enable you to submit a patch?

Sure, I can submit a patch, though while that got me going I have no real
idea if that is the correct way to worm around the problem.  I sort of think
this is something which should be tested for in configure.  The code above
was just a guess.

Skip

From cesare at pronto.it  Sat Jun 14 08:19:30 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Sat, 14 Jun 2008 08:19:30 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <4852F7EC.6040403@voidspace.org.uk>
References: <4852F7EC.6040403@voidspace.org.uk>
Message-ID: <op.ucp6isfyhlrjc9@conan>

I agree: Python's with statement does have a completely different meaning of the same Pascal & Co 's statement.

However, I don't agree with the FAQ on this point. I think that a Pascal-like with statement can be achieved, even with a dynamic language such as Python, and in a simple way.

We know that name resolution in Python uses the LEGB rule: first it starts looking at locals, then to enclosing functions, then to globals, and finally to the built-ins.
Assignament usually acts on locals (and with global statement, we can can even change globals).

A "with" statement (such as Pascal's one) can be implemented adding a new scope resolution on top of LEGB. So, taking the FAQ's example, and using a new keyword (because with is already assigned to a different kind of work):

def foo(a):
  on a:
     print x

the block defined by the "on" statement first must starts looking at the object's namespace. If no symbol was defined inside a, then it follows the traditional LEGB name resolution.

Assignament must work on the object's namespace, of course:

def foo(a):
  on a:
     x += 1
     print x
    will be equivalent to:

def foo(a):
  a.x += 1
  print a.x

A more complex example (taking "A Simple Hello World Program" in http://docs.python.org/lib/node688.html ) will show a major benefit on using this statement:

from Tkinter import *

class Application(Frame):
   def say_hi(self):
       print "hi there, everyone!"

   def createWidgets(self):
       on Button(self):
           text = "QUIT"
           fg = "red"
           command = self.quit

           pack({"side": "left"})

       on Button(self):
           text = "Hello",
           command = self.say_hi

           pack({"side": "left"})

   def __init__(self, master=None):
       Frame.__init__(self, master)
       self.pack()
       self.createWidgets()

root = Tk()
on Application(master=root):
   mainloop()
root.destroy()


Notice that adding a new step in scope resolution doesn't necessarily increase execution speed. Taking a look at the above example, accessing to the Button instance's attributes can be as fast as the original example, if not better.

Also, the new step in scope resolution will be needed only inside the block defined by the "on" statement. Outside the "on" statement, the usual LEGB method will be used.

Obviously "on" statements can be nested, adding each one a new step on scope resolution.

IMO such kind of statement can be very useful to make the source code more readable, especially working with GUIs.

If somebody knows SmallTalk, it looks similar to the "cascade messages" syntax:

(Window new)
   label: 'Hello';
   open

Cesare Di Mauro

In data 14 giugno 2008 alle ore 00:42:52, Michael Foord <fuzzyman at voidspace.org.uk> ha scritto:

> The Python FAQ has the following entry:
>
> 4.26   Why doesn't Python have a "with" statement like some other languages?
>
> From:
>
> http://www.python.org/doc/faq/general/
>
> Even if the content is still relevant (I know nothing of the use of
> 'with' outside Python), the entry probably needs at least clarifying now
> that Python does have a 'with' statement.
>
> Michael Foord
>



From martin at v.loewis.de  Sat Jun 14 08:55:58 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Jun 2008 08:55:58 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <op.ucp6isfyhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
Message-ID: <48536B7E.30103@v.loewis.de>

> the block defined by the "on" statement first must starts looking at
> the object's namespace. If no symbol was defined inside a, then it
> follows the traditional LEGB name resolution.
> 
> Assignament must work on the object's namespace, of course:

This probably belongs to python-ideas or some such, but I don't
think this approach can work. People will want to assign to local
variables in an "ob" block, and then be surprised that the
assignment actually modified their object:

def f(L):
    total = 0
    for h in L:
        on h:
            more code accessing h's attributes
            if x: # reads h.x
                total = total+1
    return total

People will be surprised that total is always 0 (and that
some objects have an attribute total with a value of 1).
Likewise

on x:
    for e in L:
        counts[e] += 1 # modifies x.counts

People will be surprised that x also grows an attribute
e, as the for loop involves an assignment, which you say
goes to the object's namespace, of course.

Regards,
Martin



From steve at pearwood.info  Sat Jun 14 10:16:17 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 14 Jun 2008 18:16:17 +1000
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <op.ucp6isfyhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
Message-ID: <200806141816.18526.steve@pearwood.info>

On Sat, 14 Jun 2008 04:19:30 pm Cesare Di Mauro wrote:

> A "with" statement (such as Pascal's one) can be implemented adding a
> new scope resolution on top of LEGB. So, taking the FAQ's example,
> and using a new keyword (because with is already assigned to a
> different kind of work):
>
> def foo(a):
>   on a:
>      print x
>
> the block defined by the "on" statement first must starts looking at
> the object's namespace. If no symbol was defined inside a, then it
> follows the traditional LEGB name resolution.

I am a great fan of Pascal-style with blocks, and I'd love Python to get 
something like them. But I have concluded that they simply aren't 
practical in Python. Your suggestion sounds reasonable at first glance, 
but it isn't really.

def foo(a):
    on a:
        x = round(x, n)
    return x

What does this function do? There's no way of telling, because it 
depends on what attributes a has. It might be equivalent to any of:

a.x = a.round(a.x, a.n)  # all attributes
a.x = round(a.x, a.n)  # built-in function round
a.x = round(x, a.n)  # global x and attribute n
etc.

and there's no way of predicting which it will be until you know what 
attributes a has. Even something like this can fail unexpectedly:

on a:
    import math
    y = math.sin(x)

if object a happens to have an attribute called 'math'.



> Assignament must work on the object's namespace, of course:

There's no "of course" about it. That rule implies that it is impossible 
to modify the local namespace inside an 'on block', because the 
object's namespace will always take priority:

def foo(a):
    y = 1
    on a:
        y = x + 1  # creates a new a.y attribute!
    return y  # always returns 1

Perhaps you want this behaviour, but I don't think it is either obvious 
or desirable.

Pascal doesn't have this problem because you always explicitly define 
your local variables.


-- 
Steven

From hodgestar+pythondev at gmail.com  Sat Jun 14 11:03:09 2008
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Sat, 14 Jun 2008 11:03:09 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <200806141816.18526.steve@pearwood.info>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<200806141816.18526.steve@pearwood.info>
Message-ID: <fb73205e0806140203n4ea56cf4gd5f24bbac724b247@mail.gmail.com>

On Sat, Jun 14, 2008 at 10:16 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> I am a great fan of Pascal-style with blocks, and I'd love Python to get
> something like them. But I have concluded that they simply aren't
> practical in Python. Your suggestion sounds reasonable at first glance,
> but it isn't really.
>
> def foo(a):
>    on a:
>        x = round(x, n)
>    return x
>
> What does this function do? There's no way of telling, because it
> depends on what attributes a has. It might be equivalent to any of:

After having read all the examples in the thread so far, let's never
add anything like this to Python ever. It breaks so many of the
"import this" guidelines it's scary. The biggest loss is readability.
Sure in a statically typed language it might be possible for the
compiler figure out which things are attributes and which are not, but
I have to be able to do the same. And lets not even think about what
happens when some adds a new attribute to a and code using "on a"
suddenly starts silently breaking.

Schiavo
Simon

From musiccomposition at gmail.com  Sat Jun 14 14:33:27 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sat, 14 Jun 2008 07:33:27 -0500
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <18515.18854.711188.716549@montanaro-dyndns-org.local>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<18515.18854.711188.716549@montanaro-dyndns-org.local>
Message-ID: <1afaf6160806140533w6efa9adarcb854ff3a3318556@mail.gmail.com>

On Fri, Jun 13, 2008 at 11:31 PM,  <skip at pobox.com> wrote:
>>>>>> "Guido" == Guido van Rossum <guido at python.org> writes:
>
>    Guido> Ow. definitely.
>
> Yes.  If I have some time Saturday and nobody beats me to it I'll fix this.

Sorry, it seems I've beaten you. :)



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From pje at telecommunity.com  Sat Jun 14 16:43:46 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 14 Jun 2008 10:43:46 -0400
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <op.ucp6isfyhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk>
 <op.ucp6isfyhlrjc9@conan>
Message-ID: <20080614144312.EA7C63A405F@sparrow.telecommunity.com>

At 08:19 AM 6/14/2008 +0200, Cesare Di Mauro wrote:
>Assignament must work on the object's namespace, of course:
>
>def foo(a):
>   on a:
>      x += 1
>      print x
>     will be equivalent to:
>
>def foo(a):
>   a.x += 1
>   print a.x

Er, you need a syntactic disambiguation here to distinguish 
attributes from locals or globals:

def foo(a):
   on a:
      .x += 1
      print .x

Otherwise, this leads to all sorts of craziness.  You'd also have to 
restrict what could be referenced in a nested "on" block, in order to 
avoid further ambiguities.


From cesare at pronto.it  Sat Jun 14 21:31:23 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Sat, 14 Jun 2008 21:31:23 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <48536B7E.30103@v.loewis.de>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de>
Message-ID: <op.ucq66l2nhlrjc9@conan>

In data 14 giugno 2008 alle ore 08:55:58, Martin v. L?wis <martin at v.loewis.de> ha scritto:

> This probably belongs to python-ideas or some such, but I don't
> think this approach can work. People will want to assign to local
> variables in an "ob" block, and then be surprised that the
> assignment actually modified their object:
>
> def f(L):
>     total = 0
>     for h in L:
>         on h:
>             more code accessing h's attributes
>             if x: # reads h.x
>                 total = total+1
>     return total
>
> People will be surprised that total is always 0 (and that
> some objects have an attribute total with a value of 1).
> Likewise
>
> on x:
>     for e in L:
>         counts[e] += 1 # modifies x.counts
>
> People will be surprised that x also grows an attribute
> e, as the for loop involves an assignment, which you say
> goes to the object's namespace, of course.
>
> Regards,
> Martin

I think there can be two solutions to the local variable assignament.

The first is to declare them with an "local" instruction, similar to the global one. So we can have:

def f(L):
    total = 0
    for h in L:
        on h:
            local total
            more code accessing h's attributes
            if x: # reads h.x
                total = total+1
    return total

on x:
    local e
    for e in L:
        counts[e] += 1 # modifies x.counts


The second solution is to leave assignament to the local namespace, and let assignament to object's attributes happens.
So your examples will work without changes, and the object never "inherits" new attributes.

Also, taking the Tk example that I used, it can be changed in the following way:

       on Button(self) as b:
           b.text = "QUIT"
           b.fg = "red"
           b.command = self.quit
 
           pack({"side": "left"})
 
       on Button(self) as b:
           b.text = "Hello"
           b.command = self.say_hi
 
           pack({"side": "left"})

Using a syntax which reseambles the with one.

Cesare Di Mauro

From g.brandl at gmx.net  Sat Jun 14 21:33:40 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 14 Jun 2008 21:33:40 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <op.ucq66l2nhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk>
	<op.ucp6isfyhlrjc9@conan>	<48536B7E.30103@v.loewis.de>
	<op.ucq66l2nhlrjc9@conan>
Message-ID: <g316es$687$1@ger.gmane.org>

Cesare Di Mauro schrieb:

> Also, taking the Tk example that I used, it can be changed in the following way:
> 
>        on Button(self) as b:
>            b.text = "QUIT"
>            b.fg = "red"
>            b.command = self.quit
>  
>            pack({"side": "left"})
>  
>        on Button(self) as b:
>            b.text = "Hello"
>            b.command = self.say_hi
>  
>            pack({"side": "left"})
> 
> Using a syntax which reseambles the with one.

So what is the advantage to

b = Button(self)
b.text = "QUIT"
b.fg = "red"
b.command = self.quit

?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From jnoller at gmail.com  Sat Jun 14 21:38:53 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Sat, 14 Jun 2008 15:38:53 -0400
Subject: [Python-Dev] multiprocessing source not "Unix clean"
In-Reply-To: <18515.18854.711188.716549@montanaro-dyndns-org.local>
References: <18514.40571.3934.984310@montanaro-dyndns-org.local>
	<ca471dc20806131017h20fa4238o2b5fb6ae32f6c841@mail.gmail.com>
	<18515.18854.711188.716549@montanaro-dyndns-org.local>
Message-ID: <4222a8490806141238x31ee38abk17f20e0ecaa052b8@mail.gmail.com>

On Sat, Jun 14, 2008 at 12:31 AM,  <skip at pobox.com> wrote:
>>>>>> "Guido" == Guido van Rossum <guido at python.org> writes:
>
>    Guido> On Fri, Jun 13, 2008 at 9:21 AM,  <skip at pobox.com> wrote:
>    >> In trying to solve a build problem with the multiprocessing code on
>    >> Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all
>    >> appear to have Windows line endings.  Should those maybe be stripped to
>    >> conform to the other Python source?
>
>    Guido> Ow. definitely.
>
> Yes.  If I have some time Saturday and nobody beats me to it I'll fix this.
>
>    >> FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does
>    >> define _SEM_VALUE_MAX in sys/params.h.
>    >>
>    >> .../Modules/_multiprocessing/multiprocessing.c: In function 'init_multiprocessing':
>    >> .../Modules/_multiprocessing/multiprocessing.c:253: error: 'SEM_VALUE_MAX' undeclared (first use in this function)
>    >> .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each undeclared identifier is reported only once
>    >> .../Modules/_multiprocessing/multiprocessing.c:253: error: for each function it appears in.)
>    >>
>    >> On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX.  I used
>    >> a little cpp action to define it:
>    >>
>    >> #ifndef SEM_VALUE_MAX
>    >> #  ifdef _SEM_VALUE_MAX
>    >> #    define SEM_VALUE_MAX _SEM_VALUE_MAX
>    >> #  else
>    >> #    define SEM_VALUE_MAX INT_MAX
>    >> #  endif
>    >> #endif
>
>    Guido> Does this enable you to submit a patch?
>
> Sure, I can submit a patch, though while that got me going I have no real
> idea if that is the correct way to worm around the problem.  I sort of think
> this is something which should be tested for in configure.  The code above
> was just a guess.
>
> Skip

I filed http://bugs.python.org/issue3110 for the solaris issue. The
best way to fix this is to patch setup.py to in the multiprocessing
section to account for the solaris issue.

-jesse

From cesare at pronto.it  Sat Jun 14 21:53:14 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Sat, 14 Jun 2008 21:53:14 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <fb73205e0806140203n4ea56cf4gd5f24bbac724b247@mail.gmail.com>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<200806141816.18526.steve@pearwood.info>
	<fb73205e0806140203n4ea56cf4gd5f24bbac724b247@mail.gmail.com>
Message-ID: <op.ucq7601bhlrjc9@conan>

In data 14 giugno 2008 alle ore 11:03:09, Simon Cross <hodgestar+pythondev at gmail.com> ha scritto:

> After having read all the examples in the thread so far, let's never
> add anything like this to Python ever. It breaks so many of the
> "import this" guidelines it's scary. The biggest loss is readability.
> Sure in a statically typed language it might be possible for the
> compiler figure out which things are attributes and which are not, but
> I have to be able to do the same. And lets not even think about what
> happens when some adds a new attribute to a and code using "on a"
> suddenly starts silently breaking.
>
> Schiavo
> Simon

It was just a "rough" idea, and I have suggested two solutions to make it better.

Obviously there can be situations where the new instruction will be useful, and other where it will not.

In my experience I found so many times coding "patterns" where I work "inside" an object,
calling many methods and accessing its variabiles (primarily reading them),
and the new instruction which I proposed to have in Python can make this kind of code much easier
to write and managed (especially with GUIs, there are many common "patterns" that can benefit
from it).

Also, I think that the code can be more readable.
Just take a look at the example I reported: don't you find it easier to read?

Cesare Di Mauro

From cesare at pronto.it  Sat Jun 14 22:05:56 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Sat, 14 Jun 2008 22:05:56 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <g316es$687$1@ger.gmane.org>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org>
Message-ID: <op.ucq8r6s8hlrjc9@conan>

In data 14 giugno 2008 alle ore 21:33:40, Georg Brandl <g.brandl at gmx.net> ha scritto:

> So what is the advantage to
>
> b = Button(self)
> b.text = "QUIT"
> b.fg = "red"
> b.command = self.quit
>
> ?
>
> Georg

In this example there are many assignaments, so there aren't many advantages.

But this

    t = ScrolledText.ScrolledText(master, width=60, height=37)
    t.insert(Tkinter.END, self.log.getText())
    t.configure(state=Tkinter.DISABLED)
    t.see(Tkinter.END)
    t.pack(fill=Tkinter.BOTH)

can look like:

  on Tkinter:
    on ScrolledText.ScrolledText(master, width=60, height=37):
      insert(END, self.log.getText())
      configure(state=DISABLED)
      see(END)
      pack(fill=BOTH)

Cesare Di Mauro


From ggpolo at gmail.com  Sat Jun 14 22:09:28 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Sat, 14 Jun 2008 17:09:28 -0300
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <op.ucq8r6s8hlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org> <op.ucq8r6s8hlrjc9@conan>
Message-ID: <ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>

On Sat, Jun 14, 2008 at 5:05 PM, Cesare Di Mauro <cesare at pronto.it> wrote:
> In data 14 giugno 2008 alle ore 21:33:40, Georg Brandl <g.brandl at gmx.net> ha scritto:
>
>> So what is the advantage to
>>
>> b = Button(self)
>> b.text = "QUIT"
>> b.fg = "red"
>> b.command = self.quit
>>
>> ?
>>
>> Georg
>
> In this example there are many assignaments, so there aren't many advantages.
>
> But this
>
>    t = ScrolledText.ScrolledText(master, width=60, height=37)
>    t.insert(Tkinter.END, self.log.getText())
>    t.configure(state=Tkinter.DISABLED)
>    t.see(Tkinter.END)
>    t.pack(fill=Tkinter.BOTH)
>
> can look like:
>
>  on Tkinter:
>    on ScrolledText.ScrolledText(master, width=60, height=37):
>      insert(END, self.log.getText())
>      configure(state=DISABLED)
>      see(END)
>      pack(fill=BOTH)
>

Then you have to start guessing from where these names came from.

> Cesare Di Mauro
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>



-- 
-- Guilherme H. Polo Goncalves

From aleaxit at gmail.com  Sat Jun 14 22:09:25 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Sat, 14 Jun 2008 13:09:25 -0700
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <20080614144312.EA7C63A405F@sparrow.telecommunity.com>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<20080614144312.EA7C63A405F@sparrow.telecommunity.com>
Message-ID: <e8a0972d0806141309j38a788bfuc8e485df2861f694@mail.gmail.com>

Yep.  Javascript's totally botched and unusable 'with' statement is an
excellent warning of what horrors that kind of thing can wreck in a
dynamic language unless carefully designed.  I also agree that .foo is
a good disambiguation syntax -- unfortunately it doesn't disambiguate
among *nested* with-blocks; maybe one should use multiple dots for the
purpose, i.e.

  on a:
    on b:
      .foo = 23

would assign to b.foo, while

  on a:
    on b:
      ..foo = 23

would assign to a.foo ...?

Disambiguation might be required only for assignment, maybe -- with
all other accesses to identifiers without leading dots doing nested
lookup (so that e.g. "print foo" might emit b.foo if present, else
a.foo if present, else local foo if present, else global foo if
present).

I'm not sure I like this whole arrangement, but maybe overall I do,
perhaps +0 ...


Alex


On Sat, Jun 14, 2008 at 7:43 AM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 08:19 AM 6/14/2008 +0200, Cesare Di Mauro wrote:
>>
>> Assignament must work on the object's namespace, of course:
>>
>> def foo(a):
>>  on a:
>>     x += 1
>>     print x
>>    will be equivalent to:
>>
>> def foo(a):
>>  a.x += 1
>>  print a.x
>
> Er, you need a syntactic disambiguation here to distinguish attributes from
> locals or globals:
>
> def foo(a):
>  on a:
>     .x += 1
>     print .x
>
> Otherwise, this leads to all sorts of craziness.  You'd also have to
> restrict what could be referenced in a nested "on" block, in order to avoid
> further ambiguities.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
>

From martin at v.loewis.de  Sat Jun 14 22:09:56 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Jun 2008 22:09:56 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <op.ucq7601bhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk>
	<op.ucp6isfyhlrjc9@conan>	<200806141816.18526.steve@pearwood.info>	<fb73205e0806140203n4ea56cf4gd5f24bbac724b247@mail.gmail.com>
	<op.ucq7601bhlrjc9@conan>
Message-ID: <48542594.4000107@v.loewis.de>

> Just take a look at the example I reported: don't you find it easier to read?

No, not at all. I actually *like* having to specify the object on a
method call. I find C++/Java code very hard to read which invokes a
method without specifying this. infront of the call - in particular
when you are unfamiliar with the class hierarchy you are looking at.
(due to late binding, it is still difficult to find out what specific
method is being called, but atleast you can be certain it's a method,
and you might even have a clue what the dynamic type of the object
is).

So I think the FAQ should continue to claim that Python cannot grow
a Pascal-like with statement.

Regards,
Martin

From cesare at pronto.it  Sat Jun 14 22:19:49 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Sat, 14 Jun 2008 22:19:49 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org> <op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
Message-ID: <op.ucq9fbiuhlrjc9@conan>

In data 14 giugno 2008 alle ore 22:09:28, Guilherme Polo <ggpolo at gmail.com> ha scritto:

>>  on Tkinter:
>>    on ScrolledText.ScrolledText(master, width=60, height=37):
>>      insert(END, self.log.getText())
>>      configure(state=DISABLED)
>>      see(END)
>>      pack(fill=BOTH)
>>
>
> Then you have to start guessing from where these names came from.

The same happens with:

from Tkinter import *

which is a fair common instruction...

P.S. Object hierarchy can be a knightmare with or without the new instruction: it's a matter of programmer's knowledge base.

Cesare Di Mauro

From cesare at pronto.it  Sat Jun 14 22:20:44 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Sat, 14 Jun 2008 22:20:44 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <48542594.4000107@v.loewis.de>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<200806141816.18526.steve@pearwood.info>
	<fb73205e0806140203n4ea56cf4gd5f24bbac724b247@mail.gmail.com>
	<op.ucq7601bhlrjc9@conan> <48542594.4000107@v.loewis.de>
Message-ID: <op.ucq9gu0bhlrjc9@conan>

In data 14 giugno 2008 alle ore 22:09:56, Martin v. L?wis <martin at v.loewis.de> ha scritto:

> No, not at all. I actually *like* having to specify the object on a
> method call. I find C++/Java code very hard to read which invokes a
> method without specifying this. infront of the call - in particular
> when you are unfamiliar with the class hierarchy you are looking at.
> (due to late binding, it is still difficult to find out what specific
> method is being called, but atleast you can be certain it's a method,
> and you might even have a clue what the dynamic type of the object
> is).
>
> So I think the FAQ should continue to claim that Python cannot grow
> a Pascal-like with statement.
>
> Regards,
> Martin

OK. It was just an idea...

Thanks

Cesare Di Mauro

From hodgestar at gmail.com  Sat Jun 14 22:22:22 2008
From: hodgestar at gmail.com (Simon Cross)
Date: Sat, 14 Jun 2008 22:22:22 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <op.ucq7601bhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<200806141816.18526.steve@pearwood.info>
	<fb73205e0806140203n4ea56cf4gd5f24bbac724b247@mail.gmail.com>
	<op.ucq7601bhlrjc9@conan>
Message-ID: <fb73205e0806141322p3f822802s8878ad5d946ec5e3@mail.gmail.com>

On Sat, Jun 14, 2008 at 9:53 PM, Cesare Di Mauro <cesare at pronto.it> wrote:
> Just take a look at the example I reported: don't you find it easier to read?

Sure, it's perhaps a bit easier on the eyes, but readability includes
understanding what's the code does.

Let's take an example:

>on Tkinter:
>   on ScrolledText.ScrolledText(master, width=60, height=37):
>     insert(END, self.log.getText())
>     configure(state=DISABLED)
>     see(END)
>     pack(fill=BOTH)

Is END an attribute of ScrolledText? Tkinter? Neither? Both? Who
knows. If we allow assignments it's even more painful. Throwing random
junk into the current scope isn't my idea of fun.

Schiavo
Simon

From hodgestar at gmail.com  Sat Jun 14 22:24:25 2008
From: hodgestar at gmail.com (Simon Cross)
Date: Sat, 14 Jun 2008 22:24:25 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <op.ucq9fbiuhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org> <op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan>
Message-ID: <fb73205e0806141324n7f49d87cxe4bd85b4afc17722@mail.gmail.com>

On Sat, Jun 14, 2008 at 10:19 PM, Cesare Di Mauro <cesare at pronto.it> wrote:
> from Tkinter import *

I discourage this too. :)

Schiavo
Simob

From barry at python.org  Sat Jun 14 23:18:34 2008
From: barry at python.org (Barry Warsaw)
Date: Sat, 14 Jun 2008 17:18:34 -0400
Subject: [Python-Dev] [Python-3000] First betas (perhaps) on June 18
In-Reply-To: <e04bdf310806130654o109c68e9l2e42e4fd899e1901@mail.gmail.com>
References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org>
	<e04bdf310806130654o109c68e9l2e42e4fd899e1901@mail.gmail.com>
Message-ID: <4F7E8331-D77E-4449-B8DD-305F0D3A50AB@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 13, 2008, at 9:54 AM, Facundo Batista wrote:

> 2008/6/13 Barry Warsaw <barry at python.org>:
>
>> My proposal is this: I will spin another release this coming  
>> Wednesday, June
>> 18.  If we can get both the 2.6 and 3.0 buildbots green by then,  
>> and close
>> out all remaining release critical bugs, then Wednesday's release  
>> will be
>> beta 1.  In that case, I will update PEP 361 and push the entire  
>> schedule
>> back by 2 weeks to account for our slippage.
>
> Next weekend, 21 and 22, it will be the Python Bug days.
>
> Maybe it's worth to delay the betas one week to include this work?

I think I'd rather stick to a one week delay, otherwise I think we're  
really talking about pushing the whole schedule back a month.  I think  
we should focus right now on getting the buildbots as green as  
possible (yay for some 3.0 greenness! thanks everyone!) and closing  
all the release critical bugs.  I think this will actually free us up  
a bit more on bug day to concentrate on fixing other issues.

Cheers,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFQ1qnEjvBPtnXfVAQI13wP/Xj+my8YUNHuLG8i7pggHNC1Vn7/j5K7H
4l5vT9EMgkT6OHXKvrVPxOJSbm7TkoHd4k3M524IYwnKigFAVH/e9WmgTChp4hPv
7UsF9MGKcSsgnjB2LpWvbV9A6lSRLsNLjuLqqJSQgS7TvrD+0Omd91RM9twmV9io
BndCSNAALyE=
=g3RH
-----END PGP SIGNATURE-----

From musiccomposition at gmail.com  Sat Jun 14 23:22:35 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sat, 14 Jun 2008 16:22:35 -0500
Subject: [Python-Dev] [Python-3000] First betas (perhaps) on June 18
In-Reply-To: <4F7E8331-D77E-4449-B8DD-305F0D3A50AB@python.org>
References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org>
	<e04bdf310806130654o109c68e9l2e42e4fd899e1901@mail.gmail.com>
	<4F7E8331-D77E-4449-B8DD-305F0D3A50AB@python.org>
Message-ID: <1afaf6160806141422y14bfd5dcn1bc900bf984cbdf2@mail.gmail.com>

On Sat, Jun 14, 2008 at 4:18 PM, Barry Warsaw <barry at python.org> wrote:
> On Jun 13, 2008, at 9:54 AM, Facundo Batista wrote:
>> Next weekend, 21 and 22, it will be the Python Bug days.
>>
>> Maybe it's worth to delay the betas one week to include this work?
>
> I think I'd rather stick to a one week delay, otherwise I think we're really
> talking about pushing the whole schedule back a month.  I think we should
> focus right now on getting the buildbots as green as possible (yay for some
> 3.0 greenness! thanks everyone!) and closing all the release critical bugs.
>  I think this will actually free us up a bit more on bug day to concentrate
> on fixing other issues.

I kinda hate to do this to you, but I had to file another release blocker: #3114


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From aleaxit at gmail.com  Sun Jun 15 01:12:47 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Sat, 14 Jun 2008 16:12:47 -0700
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <fb73205e0806141324n7f49d87cxe4bd85b4afc17722@mail.gmail.com>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org> <op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan>
	<fb73205e0806141324n7f49d87cxe4bd85b4afc17722@mail.gmail.com>
Message-ID: <e8a0972d0806141612v11ed81ddm169cb7ee362baa07@mail.gmail.com>

On Sat, Jun 14, 2008 at 1:24 PM, Simon Cross <hodgestar at gmail.com> wrote:
> On Sat, Jun 14, 2008 at 10:19 PM, Cesare Di Mauro <cesare at pronto.it> wrote:
>> from Tkinter import *
>
> I discourage this too. :)

And so do I, every single time I teach Python (which is pretty often,
even though many of those cases don't make it into YouTube videos).  I
don't need to rant against it within Google, because Google's Python
Style Guide already absolutely forbids it, fortunately;-).


Alex

From armin.ronacher at active-4.com  Sun Jun 15 01:39:11 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sat, 14 Jun 2008 23:39:11 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
Message-ID: <loom.20080614T231548-103@post.gmane.org>

Hi,

I noticed lately that quite a few projects are implementing their own
subclasses of `dict` that retain the order of the key/value pairs.
However half of the implementations I came across are not implementing
the whole dict interface which leads to weird bugs, also the performance
of a Python implementation is not that great.

To fight that problem I want to proposed a new class in "collections"
called odict which is a dict that keeps the items sorted, similar to
a PHP array.

The interface would be fully compatible with dict and implemented as
dict subclass.  Updates to existing keys does not change the order of
a key but new keys are inserted at the end.

Additionally it would support slicing where a list of key, value tuples
is returned and sort/reverse/index methods that work like their list
equivalents.  Index based lookup could work via odict.byindex().

An implementation of that exists as part of the ordereddict implementation
which however goes beyond that and is pretty much a fork of the python
dict[1].

Some reasons why ordered dicts are a useful feature:

  - in XML/HTML processing it's often desired to keep the attributes of
    an tag ordered during processing.  So that input ordering is the
    same as the output ordering.

  - Form data transmitted via HTTP is usually ordered by the position
    of the input/textarea/select field in the HTML document.  That
    information is currently lost in most Python web applications /
    frameworks.

  - Eaiser transition of code from Ruby/PHP which have sorted
    associative arrays / hashmaps.

  - Having an ordered dict in the standard library would allow other
    libraries support them.  For example a PHP serializer could return
    odicts rather then dicts which drops the ordering information.
    XML libraries such as etree could add support for it when creating
    elements or return attribute dicts.

Regards,
Armin


[1]: http://www.xs4all.nl/~anthon/Python/ordereddict/


From fuzzyman at voidspace.org.uk  Sun Jun 15 01:44:24 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Jun 2008 00:44:24 +0100
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080614T231548-103@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>
Message-ID: <485457D8.2090205@voidspace.org.uk>

Armin Ronacher wrote:
> Hi,
>
> I noticed lately that quite a few projects are implementing their own
> subclasses of `dict` that retain the order of the key/value pairs.
> However half of the implementations I came across are not implementing
> the whole dict interface which leads to weird bugs, also the performance
> of a Python implementation is not that great.
>
>   

I'm +1 - but this proposal has been made many times before and people 
always argue about what features are needed or desirable. :-(

Michael Foord

> To fight that problem I want to proposed a new class in "collections"
> called odict which is a dict that keeps the items sorted, similar to
> a PHP array.
>
> The interface would be fully compatible with dict and implemented as
> dict subclass.  Updates to existing keys does not change the order of
> a key but new keys are inserted at the end.
>
> Additionally it would support slicing where a list of key, value tuples
> is returned and sort/reverse/index methods that work like their list
> equivalents.  Index based lookup could work via odict.byindex().
>
> An implementation of that exists as part of the ordereddict implementation
> which however goes beyond that and is pretty much a fork of the python
> dict[1].
>
> Some reasons why ordered dicts are a useful feature:
>
>   - in XML/HTML processing it's often desired to keep the attributes of
>     an tag ordered during processing.  So that input ordering is the
>     same as the output ordering.
>
>   - Form data transmitted via HTTP is usually ordered by the position
>     of the input/textarea/select field in the HTML document.  That
>     information is currently lost in most Python web applications /
>     frameworks.
>
>   - Eaiser transition of code from Ruby/PHP which have sorted
>     associative arrays / hashmaps.
>
>   - Having an ordered dict in the standard library would allow other
>     libraries support them.  For example a PHP serializer could return
>     odicts rather then dicts which drops the ordering information.
>     XML libraries such as etree could add support for it when creating
>     elements or return attribute dicts.
>
> Regards,
> Armin
>
>
> [1]: http://www.xs4all.nl/~anthon/Python/ordereddict/
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.theotherdelia.co.uk/
http://www.voidspace.org.uk/
http://www.ironpython.info/
http://www.resolverhacks.net/


From greg.ewing at canterbury.ac.nz  Sun Jun 15 01:42:14 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 15 Jun 2008 11:42:14 +1200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <op.ucp6isfyhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
Message-ID: <48545756.9060901@canterbury.ac.nz>

Cesare Di Mauro wrote:

> However, I don't agree with the FAQ on this point. I think that a
 > Pascal-like with statement can be achieved, even with a dynamic
 > language such as Python, and in a simple way.

It's not so much a matter of whether it *can* be done, but
whether there's any substantial need for it. I don't believe
there is in Python, because you can always put a reference
to the target object into a succinctly-named local and access
it through that.

You can't always do that in Pascal, because it provides no
way of obtaining a pointer a record that wasn't heap-allocated.

-- 
Greg

From nd at perlig.de  Sun Jun 15 01:57:23 2008
From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=)
Date: Sun, 15 Jun 2008 01:57:23 +0200
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080614T231548-103@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>
Message-ID: <200806150157.23985@news.perlig.de>

* Armin Ronacher wrote:

> Some reasons why ordered dicts are a useful feature:
>
>   - in XML/HTML processing it's often desired to keep the attributes of
>     an tag ordered during processing.  So that input ordering is the
>     same as the output ordering.
>
>   - Form data transmitted via HTTP is usually ordered by the position
>     of the input/textarea/select field in the HTML document.  That
>     information is currently lost in most Python web applications /
>     frameworks.
>
>   - Eaiser transition of code from Ruby/PHP which have sorted
>     associative arrays / hashmaps.
>
>   - Having an ordered dict in the standard library would allow other
>     libraries support them.  For example a PHP serializer could return
>     odicts rather then dicts which drops the ordering information.
>     XML libraries such as etree could add support for it when creating
>     elements or return attribute dicts.

I find this collection of cases pretty weak as an argument for implementing 
that in the stdlib. A lot of special purpose types would fit into such 
reasoning, but do you want to have all of them maintained here?

nd
-- 
Da f?llt mir ein, wieso gibt es eigentlich in Unicode kein
"i" mit einem Herzchen als T?pfelchen? Das w?r sooo s??ss!

                                 -- Bj?rn H?hrmann in darw

From marek at xivilization.net  Sun Jun 15 01:53:39 2008
From: marek at xivilization.net (Marek Kubica)
Date: Sat, 14 Jun 2008 23:53:39 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org>
Message-ID: <loom.20080614T234840-435@post.gmane.org>

Hi,

Armin Ronacher <armin.ronacher <at> active-4.com> writes:

> To fight that problem I want to proposed a new class in "collections"
> called odict which is a dict that keeps the items sorted, similar to
> a PHP array.

I'm also +1 on this. I've used the implementation you mentioned in a compiler
project of mine and found it to be quite useful. It is hard for me to mention
any other uses but there definitely are many occasions where people need them
and either go for (key, value)-tuples or use some lib which does only provide
this single data type. I am pretty optimistic that the addition will find its
usecases, once it is in the stdlib.

regards,
Marek


From greg.ewing at canterbury.ac.nz  Sun Jun 15 02:24:43 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 15 Jun 2008 12:24:43 +1200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <op.ucq9fbiuhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org> <op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan>
Message-ID: <4854614B.9070903@canterbury.ac.nz>

Cesare Di Mauro wrote:

> The same happens with:
> 
> from Tkinter import *
> 
> which is a fair common instruction...

...and which should *not* be used in most cases, for
the same reason.

All those tutorials that start out with 'from something
import *' are doing a lot of harm to the impressionable
minds of new programmers, IMO.

-- 
Greg

From jimjjewett at gmail.com  Sun Jun 15 03:54:36 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sat, 14 Jun 2008 21:54:36 -0400
Subject: [Python-Dev] Advice on numbers.py implementation of binary mixins.
Message-ID: <fb6fbf560806141854h3dce3582ne842f7b61fefb543@mail.gmail.com>

Raymond Hettinger wrote:

> PEP-3141 outlines an approach to writing binary
> operators to allow the right operand to override
> the operation if the left operand inherits the
> operation from the ABC.

> Here is my first approximation at how to write
> them for the Integral mixins:

> class Integral(Rational):

>    def __and__(self, other):
>        if isinstance(other, (type(self), int, long)):  # XXX
>            return int(self) & int(other)

I think for this mixin, it doesn't matter whether other is an Integral
instance; it matter whether it is has a more specific solution.

So instead of checking whether isinstance, check whether its __rand__
method is Integral.__rand__.

I think you also may want to guard against incomplete right-hand
operations, by doing something like replacing the simple

>     return NotImplemented

with

    try:
        val = other.__rand__(self)
        if val is not NotImplemented:
            return val
    except (TypeError, AttributeError):
        pass
    # Use the generic fallback after all
    return int(self) & int(other)

From jimjjewett at gmail.com  Sun Jun 15 04:06:31 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sat, 14 Jun 2008 22:06:31 -0400
Subject: [Python-Dev] Proposal: add odict to collections
Message-ID: <fb6fbf560806141906o100b701et75227886c6ce2d2@mail.gmail.com>

The odict (as proposed here, ordered on time of key insertion) looks
like a close match to the dlict needed by some of the optimization
proposals.

http://python.org/dev/peps/pep-0267/

-jJ

From guido at python.org  Sun Jun 15 05:26:58 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 14 Jun 2008 20:26:58 -0700
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <200806150157.23985@news.perlig.de>
References: <loom.20080614T231548-103@post.gmane.org>
	<200806150157.23985@news.perlig.de>
Message-ID: <ca471dc20806142026i22064b45k7dc73c80e47045c1@mail.gmail.com>

On Sat, Jun 14, 2008 at 4:57 PM, Andr? Malo <nd at perlig.de> wrote:
> * Armin Ronacher wrote:
>
>> Some reasons why ordered dicts are a useful feature:
>>
>>   - in XML/HTML processing it's often desired to keep the attributes of
>>     an tag ordered during processing.  So that input ordering is the
>>     same as the output ordering.
>>
>>   - Form data transmitted via HTTP is usually ordered by the position
>>     of the input/textarea/select field in the HTML document.  That
>>     information is currently lost in most Python web applications /
>>     frameworks.
>>
>>   - Eaiser transition of code from Ruby/PHP which have sorted
>>     associative arrays / hashmaps.
>>
>>   - Having an ordered dict in the standard library would allow other
>>     libraries support them.  For example a PHP serializer could return
>>     odicts rather then dicts which drops the ordering information.
>>     XML libraries such as etree could add support for it when creating
>>     elements or return attribute dicts.
>
> I find this collection of cases pretty weak as an argument for implementing
> that in the stdlib. A lot of special purpose types would fit into such
> reasoning, but do you want to have all of them maintained here?

No, but an ordered dict happens to be a *very* common thing to need,
for a variety of reasons. So I'm +0.5 on adding this to the
collections module. However someone needs to contribute working code.
It would also be useful to verify that it actually fulfills the needs
of some actual use case. Perhaps looking at how Django uses its
version would be helpful.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From talin at acm.org  Sun Jun 15 05:42:24 2008
From: talin at acm.org (Talin)
Date: Sat, 14 Jun 2008 20:42:24 -0700
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <485457D8.2090205@voidspace.org.uk>
References: <loom.20080614T231548-103@post.gmane.org>
	<485457D8.2090205@voidspace.org.uk>
Message-ID: <48548FA0.5020009@acm.org>

Michael Foord wrote:
> Armin Ronacher wrote:
>> Hi,
>>
>> I noticed lately that quite a few projects are implementing their own
>> subclasses of `dict` that retain the order of the key/value pairs.
>> However half of the implementations I came across are not implementing
>> the whole dict interface which leads to weird bugs, also the performance
>> of a Python implementation is not that great.
>>
>>   
> 
> I'm +1 - but this proposal has been made many times before and people 
> always argue about what features are needed or desirable. :-(

There's been a lot of controversy/confusion about ordered dicts. One of 
the sources of confusion is that people mean different things when they 
use the term "ordered dict": In some cases, the term is used to mean a 
dictionary that remembers the order of insertions, and in other cases it 
is used to mean a sorted dict, i.e. an associative data structure in 
which the entries are kept sorted. (And I'm not sure that those are the 
only two possibilities.)

I would be more in favor of the idea if we could come up with a less 
ambiguous naming scheme.

-- Talin


From bjourne at gmail.com  Sun Jun 15 06:37:11 2008
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Sun, 15 Jun 2008 04:37:11 +0000
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080614T231548-103@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>
Message-ID: <740c3aec0806142137r79bb8a84k411155d75b2df592@mail.gmail.com>

On Sat, Jun 14, 2008 at 11:39 PM, Armin Ronacher
<armin.ronacher at active-4.com> wrote:
> Some reasons why ordered dicts are a useful feature:


And for metaclasses or for LRU caches or for removing duplicates in a
list while maintaining order. I think that the name ordereddict would
be more readable though, and match the existing defaultdict class in
the collections module.


-- 
mvh Bj?rn

From hasan.diwan at gmail.com  Sun Jun 15 06:38:01 2008
From: hasan.diwan at gmail.com (Hasan Diwan)
Date: Sat, 14 Jun 2008 21:38:01 -0700
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <48548FA0.5020009@acm.org>
References: <loom.20080614T231548-103@post.gmane.org>
	<485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org>
Message-ID: <2cda2fc90806142138x253894e5vc199609db4244829@mail.gmail.com>

2008/6/14 Talin <talin at acm.org>:
> There's been a lot of controversy/confusion about ordered dicts. One of the
> sources of confusion is that people mean different things when they use the
> term "ordered dict": In some cases, the term is used to mean a dictionary
> that remembers the order of insertions, and in other cases it is used to
> mean a sorted dict, i.e. an associative data structure in which the entries
> are kept sorted. (And I'm not sure that those are the only two
> possibilities.)

Have the comparison function passed in as a parameter then, if it's
None, then have it maintain the order of insertion? Something like:
def __init__(self, cmpfunc = None):
   self.dict = dict()

def __getattr__(self, key):
   try: return self.key

-- 
Cheers,
Hasan Diwan <hasan.diwan at gmail.com>

From martin at v.loewis.de  Sun Jun 15 06:55:22 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Jun 2008 06:55:22 +0200
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <2cda2fc90806142138x253894e5vc199609db4244829@mail.gmail.com>
References: <loom.20080614T231548-103@post.gmane.org>	<485457D8.2090205@voidspace.org.uk>
	<48548FA0.5020009@acm.org>
	<2cda2fc90806142138x253894e5vc199609db4244829@mail.gmail.com>
Message-ID: <4854A0BA.4070801@v.loewis.de>

> Have the comparison function passed in as a parameter then, if it's
> None, then have it maintain the order of insertion?

No. This would contribute to the confusion, not resolve it. If it's
called "ordered dictionary", it shouldn't do sorting at all. If it does
sorting, it should be called "sorted dictionary", and mandate a
comparison function (which might default to cmp).

Regards,
Martin

From ncoghlan at gmail.com  Sun Jun 15 07:14:41 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Jun 2008 15:14:41 +1000
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <op.ucp6isfyhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
Message-ID: <4854A541.5080101@gmail.com>

Cesare Di Mauro wrote:
> I agree: Python's with statement does have a completely different
> meaning of the same Pascal & Co 's statement.
> 
> However, I don't agree with the FAQ on this point. I think that a
> Pascal-like with statement can be achieved, even with a dynamic
> language such as Python, and in a simple way.

It isn't that a Pascal-with-style statement can't be achieved, it's that 
it is pointless syntactic sugar in Python. Just use o = 
long_complicated_object_name instead.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sun Jun 15 07:20:32 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Jun 2008 15:20:32 +1000
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <4854614B.9070903@canterbury.ac.nz>
References: <4852F7EC.6040403@voidspace.org.uk>
	<op.ucp6isfyhlrjc9@conan>	<48536B7E.30103@v.loewis.de>
	<op.ucq66l2nhlrjc9@conan>	<g316es$687$1@ger.gmane.org>
	<op.ucq8r6s8hlrjc9@conan>	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>	<op.ucq9fbiuhlrjc9@conan>
	<4854614B.9070903@canterbury.ac.nz>
Message-ID: <4854A6A0.3030302@gmail.com>

Greg Ewing wrote:
> Cesare Di Mauro wrote:
> 
>> The same happens with:
>>
>> from Tkinter import *
>>
>> which is a fair common instruction...
> 
> ...and which should *not* be used in most cases, for
> the same reason.
> 
> All those tutorials that start out with 'from something
> import *' are doing a lot of harm to the impressionable
> minds of new programmers, IMO.
> 
Yeah, the only remotely legitimate usage of it that I am aware of is for 
modules with a hybrid implementation where the public Python module does 
a "from _native_module import *" to get the rest of the implementation. 
And even that is somewhat arguable.

To go back to Cesare's most recent example:

     t = ScrolledText.ScrolledText(master, width=60, height=37)
     t.insert(Tkinter.END, self.log.getText())
     t.configure(state=Tkinter.DISABLED)
     t.see(Tkinter.END)
     t.pack(fill=Tkinter.BOTH)

can look like:

   tk = Tkinter:
   st = ScrolledText.ScrolledText(master, width=60, height=37):
   st.insert(tk.END, self.log.getText())
   st.configure(state=tk.DISABLED)
   st.see(tk.END)
   st.pack(fill=tk.BOTH)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sun Jun 15 07:25:31 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Jun 2008 15:25:31 +1000
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <48548FA0.5020009@acm.org>
References: <loom.20080614T231548-103@post.gmane.org>	<485457D8.2090205@voidspace.org.uk>
	<48548FA0.5020009@acm.org>
Message-ID: <4854A7CB.7050508@gmail.com>

Talin wrote:
> Michael Foord wrote:
>> Armin Ronacher wrote:
>>> Hi,
>>>
>>> I noticed lately that quite a few projects are implementing their own
>>> subclasses of `dict` that retain the order of the key/value pairs.
>>> However half of the implementations I came across are not implementing
>>> the whole dict interface which leads to weird bugs, also the performance
>>> of a Python implementation is not that great.
>>>
>>>   
>>
>> I'm +1 - but this proposal has been made many times before and people 
>> always argue about what features are needed or desirable. :-(
> 
> There's been a lot of controversy/confusion about ordered dicts. One of 
> the sources of confusion is that people mean different things when they 
> use the term "ordered dict": In some cases, the term is used to mean a 
> dictionary that remembers the order of insertions, and in other cases it 
> is used to mean a sorted dict, i.e. an associative data structure in 
> which the entries are kept sorted. (And I'm not sure that those are the 
> only two possibilities.)
> 
> I would be more in favor of the idea if we could come up with a less 
> ambiguous naming scheme.

I think Armin's proposal addresses this nicely by the analogy to lists: 
the ordered dict is in key insertion order by default, but you can 
invoke odict.sort() to sort it instead.

Given the synergy with the Py3k metaclass enhancements, I believe this 
would be a good thing to have.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sun Jun 15 07:33:49 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Jun 2008 15:33:49 +1000
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>	
	<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>	
	<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>	
	<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>	
	<18513.16097.394123.312491@montanaro-dyndns-org.local>	
	<485235B4.6030401@gmail.com>	
	<18514.41043.913466.680598@montanaro-dyndns-org.local>	
	<89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1>	
	<1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com>	
	<ca471dc20806131014g25fcad2du4401e787244601f8@mail.gmail.com>
	<1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com>
Message-ID: <4854A9BD.5040905@gmail.com>

Benjamin Peterson wrote:
> On Fri, Jun 13, 2008 at 12:14 PM, Guido van Rossum <guido at python.org> wrote:
>> On Fri, Jun 13, 2008 at 9:42 AM, Benjamin Peterson
>> <musiccomposition at gmail.com> wrote:
>>> On Fri, Jun 13, 2008 at 11:40 AM, Raymond Hettinger <python at rcn.com> wrote:
>>>>>   Nick>      def getName(self):
>>>>>   Nick>          assert self.__initialized, "Thread.__init__() not called"
>>>>>   Nick>          return self.__name
>>>> Why is __name private to begin with?  Any reason for the getters and
>>>> setters?  Why isn't this just an attribute?
>>> In 3.x, it's just an attribute.
>> Oh, is it? Where's the PEP with the API redesign? Did I miss some kind
>> of decision-making process, weighing compatibility concerns against
>> other issues?
> 
> meaning that it only has one underscore. They methods still live.

That's worse - it means it can now collide with _name attributes on 
subclasses.

This has become a lot more complicated than what I thought we were 
doing: adding PEP 8 compliant aliases for the existing methods without 
otherwise changing the threading implementation. As far as I can recall, 
that is all that was contained in the 2.x patch I reviewed.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From armin.ronacher at active-4.com  Sun Jun 15 07:34:45 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 15 Jun 2008 05:34:45 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org>
	<200806150157.23985@news.perlig.de>
	<ca471dc20806142026i22064b45k7dc73c80e47045c1@mail.gmail.com>
Message-ID: <loom.20080615T053145-449@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:

> 
> On Sat, Jun 14, 2008 at 4:57 PM, Andr? Malo <nd <at> perlig.de> wrote:
> > I find this collection of cases pretty weak as an argument for implementing
> > that in the stdlib. A lot of special purpose types would fit into such
> > reasoning, but do you want to have all of them maintained here?
> 
> No, but an ordered dict happens to be a *very* common thing to need,
> for a variety of reasons. So I'm +0.5 on adding this to the
> collections module. However someone needs to contribute working code.
> It would also be useful to verify that it actually fulfills the needs
> of some actual use case. Perhaps looking at how Django uses its
> version would be helpful.

I compared multiple ordered dicts now (including Babel, Django and the
C-implementation I mentioned earlier) and implemented a python version
of the ordered dict as reference implementation:

   http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py


Regards,
Armin




From armin.ronacher at active-4.com  Sun Jun 15 07:37:20 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 15 Jun 2008 05:37:20 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org>
	<740c3aec0806142137r79bb8a84k411155d75b2df592@mail.gmail.com>
Message-ID: <loom.20080615T053534-898@post.gmane.org>

BJ?rn Lindqvist <bjourne <at> gmail.com> writes:

> I think that the name ordereddict would be more readable though, and
> match the existing defaultdict class in the collections module.
While I agree that "ordered" makes more sense, it's pretty stupid to type
with two 'd's right after another.


Regards,
Armin


From python at rcn.com  Sun Jun 15 07:43:32 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 14 Jun 2008 22:43:32 -0700
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org><485457D8.2090205@voidspace.org.uk>
	<48548FA0.5020009@acm.org>
Message-ID: <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>

From: "Talin" <talin at acm.org>
> There's been a lot of controversy/confusion about ordered dicts.

I think that is why all earlier proposals all died.


> One of 
> the sources of confusion is that people mean different things when they 
> use the term "ordered dict": In some cases, the term is used to mean a 
> dictionary that remembers the order of insertions, and in other cases it 
> is used to mean a sorted dict, i.e. an associative data structure in 
> which the entries are kept sorted. (And I'm not sure that those are the 
> only two possibilities.)

For an insertion order dictionary, there was also a question about
how to handle duplicate keys.

Given odict([(k1,v1), (k2,v2), (k1,v3)]), what should the odict.items() return?

   [(k1,v3), (k2,v2)]
   [(k2,v2), (k1,v3)]

The first maintains the original sort order and is consistent with the usual
idea that d[k]=v simply replaces the value but does not alter the hash table.
It is a bit weird though because v3 appears earlier than v2 which was
inserted earlier.  It's especially weird for keys that are equal but not
identical: d[0]=v1  d[0.0]=v3.  Do you want 0.0 to remain associated
with v3 or should the items list contain a pair that was not in the original
insertion list, (0, v3)?

The second one is a bit weird because a key "loses its place" whenever
the value is updated.

IIRC, previous discussions showed an interest in odicts but that
there were a lot of questions of the specific semantics of the API.
This would no doubt be compounded by attempts to emulate
dict views. Regardless, there should probably be a PEP and
alternate pure python versions should be posted on ASPN so people
can try them out.
post


Raymond  


From armin.ronacher at active-4.com  Sun Jun 15 07:42:19 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 15 Jun 2008 05:42:19 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org>
	<485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org>
	<2cda2fc90806142138x253894e5vc199609db4244829@mail.gmail.com>
Message-ID: <loom.20080615T053846-392@post.gmane.org>

Hasan Diwan <hasan.diwan <at> gmail.com> writes:

> 
> 2008/6/14 Talin <talin <at> acm.org>:
> > There's been a lot of controversy/confusion about ordered dicts. One of the
> > sources of confusion is that people mean different things when they use the
> > term "ordered dict": In some cases, the term is used to mean a dictionary
> > that remembers the order of insertions, and in other cases it is used to
> > mean a sorted dict, i.e. an associative data structure in which the entries
> > are kept sorted. (And I'm not sure that those are the only two
> > possibilities.)
> 
> Have the comparison function passed in as a parameter then, if it's
> None, then have it maintain the order of insertion? Something like:
> def __init__(self, cmpfunc = None):
>    self.dict = dict()
> 
> 
I think that would be contraproductive and would make the constructor
incompatible with the normal dict constructor which accepts keyword
arguments too.  Also that dict is just in order, but not sorted by
something.  You could still do something like this:

    d = odict()
    d['Pinguin'] = 'telly'
    d['Parrot'] = 'cage'
    d['Mouse'] = 'Napoleon'
    d.sort(key=lambda x: x[1].lower())

That of course would not sort items inserted after the sort call.

Regards,
Armin


From wolever at cs.toronto.edu  Sun Jun 15 06:59:51 2008
From: wolever at cs.toronto.edu (David Wolever)
Date: Sun, 15 Jun 2008 01:59:51 -0300
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080614T231548-103@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>
Message-ID: <B27A70C2-E5DD-42B0-8D0D-F038C5A446E7@cs.toronto.edu>

On 14-Jun-08, at 8:39 PM, Armin Ronacher wrote:
> ...
> I noticed lately that quite a few projects are implementing their own
> subclasses of `dict` that retain the order of the key/value pairs.
> ...
I'm +1 on this one too, as there are at least a couple of times in  
recent memory when I would have found this useful.

And, as far as questions about the definition of an ordered  
dictionary, is there any good reason not to simply treat the dict as  
a list?  Something like (with the obvious bits left out):

class odict(dict):
   def __init__(self, *args):  self._order = []
   def __setitem__(self, key, val):  if key not in self:  
self._order.append(key)
   def __iter__(self):  return self._order
   def items(self):  return ([item, self[item] for item in self._order])
   def sort(self):  self._order.sort()
... and so on ...

That way all the order-related functions are well defined, so it  
would be hard claim that it doesn't do the "right thing" without  
claiming that lists don't do the "right thing".

From armin.ronacher at active-4.com  Sun Jun 15 08:07:47 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 15 Jun 2008 06:07:47 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org><485457D8.2090205@voidspace.org.uk>
	<48548FA0.5020009@acm.org>
	<758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
Message-ID: <loom.20080615T060333-203@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:

> For an insertion order dictionary, there was also a question about
> how to handle duplicate keys.
> 
> Given odict([(k1,v1), (k2,v2), (k1,v3)]), what should the odict.items() return?
> 
>    [(k1,v3), (k2,v2)]
>    [(k2,v2), (k1,v3)]
All the ordered dict implementations I saw behave like this:

    >>> odict([(1, 'foo'), (2, 'bar'), (1, 'baz')]).items()
    [(1, 'baz'), (2, 'bar')]

And that makes sense because it's consistent with the dict interface.
But I guess that is a pretty small issue and most of the time you are
not dealing with double keys when using an ordered dict.

> IIRC, previous discussions showed an interest in odicts but that
> there were a lot of questions of the specific semantics of the API.
No doubt there.  But 

> This would no doubt be compounded by attempts to emulate
> dict views. Regardless, there should probably be a PEP and
> alternate pure python versions should be posted on ASPN so people
> can try them out.
That's true, but by now there are countless of ordered dict implementations
with a mostly-compatible interface and applications and libraries are
using them already.

I have an example implementation here that incorporates the ideas
from ordereddict, Django's OrderedDict and Babel's odict:

    http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py


Regards,
Armin


From cesare at pronto.it  Sun Jun 15 08:16:38 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Sun, 15 Jun 2008 08:16:38 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <4854614B.9070903@canterbury.ac.nz>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org> <op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan> <4854614B.9070903@canterbury.ac.nz>
Message-ID: <op.ucr0100rhlrjc9@conan>

In data 15 giugno 2008 alle ore 02:24:43, Greg Ewing <greg.ewing at canterbury.ac.nz> ha scritto:

> ...and which should *not* be used in most cases, for
> the same reason.
>
> All those tutorials that start out with 'from something
> import *' are doing a lot of harm to the impressionable
> minds of new programmers, IMO.

OK, but nobody have questioned about removing 'from something import *' just to help noobs...
That's because the instruction *can* be useful in *some* (hopely limited, but existent) contexts.
It's a matter of programmer choises.

Anyway (and dropping my proposal), I think that the FAQ needs to be changed to advice that the
'with' keyword in Python makes a completely different kind of work.

Cesare

From cesare at pronto.it  Sun Jun 15 08:20:22 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Sun, 15 Jun 2008 08:20:22 +0200
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
References: <loom.20080614T231548-103@post.gmane.org>
	<485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org>
	<758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
Message-ID: <op.ucr078vrhlrjc9@conan>

In data 15 giugno 2008 alle ore 07:43:32, Raymond Hettinger <python at rcn.com> ha scritto:

> For an insertion order dictionary, there was also a question about
> how to handle duplicate keys.
>
> Given odict([(k1,v1), (k2,v2), (k1,v3)]), what should the odict.items() return?
>
>    [(k1,v3), (k2,v2)]
>    [(k2,v2), (k1,v3)]
>
> The first maintains the original sort order and is consistent with the usual
> idea that d[k]=v simply replaces the value but does not alter the hash table.
> It is a bit weird though because v3 appears earlier than v2 which was
> inserted earlier.  It's especially weird for keys that are equal but not
> identical: d[0]=v1  d[0.0]=v3.  Do you want 0.0 to remain associated
> with v3 or should the items list contain a pair that was not in the original
> insertion list, (0, v3)?
>
> The second one is a bit weird because a key "loses its place" whenever
> the value is updated.
>
> IIRC, previous discussions showed an interest in odicts but that
> there were a lot of questions of the specific semantics of the API.
> This would no doubt be compounded by attempts to emulate
> dict views. Regardless, there should probably be a PEP and
> alternate pure python versions should be posted on ASPN so people
> can try them out.
> post
>
>
> Raymond

The same problem happens with dictionary updates:

d = {}
d[k1] = v1
d[k2] = v2
d[k1] = v3

The last instruction just replaces the existing entry, so I'm +0 for the first result.

Cesare

From dbpokorny at gmail.com  Sun Jun 15 09:09:59 2008
From: dbpokorny at gmail.com (dbpokorny at gmail.com)
Date: Sun, 15 Jun 2008 00:09:59 -0700 (PDT)
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080614T231548-103@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>
Message-ID: <09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com>

On Jun 14, 4:39 pm, Armin Ronacher <armin.ronac... at active-4.com>
wrote:
>   - in XML/HTML processing it's often desired to keep the attributes of
>     an tag ordered during processing.  So that input ordering is the
>     same as the output ordering.

Munging XML is a niche.

>
>   - Form data transmitted via HTTP is usually ordered by the position
>     of the input/textarea/select field in the HTML document.  That
>     information is currently lost in most Python web applications /
>     frameworks.

If you don't like the fact that your web application framework loses
the order of its key:value pairs relative to the page, then you should
bring it up with the developers.

Ordered dicts, dicts that remember the chronological order of their
insertion, don't sound generally useful. In contrast, sorted dicts
sound appealing, but I can't think of any use cases where
sorted(mydict.keys()) fails to be effective; it seems to me the main
advantage of a sorted dict is that you don't have to remember to sort
the keys every time you want to use them.

> Additionally it would support slicing where a list of key, value tuples
> is returned and sort/reverse/index methods that work like their list
> equivalents.  Index based lookup could work via odict.byindex().

Slicing tuples, lists, and strings return objects of the same type, so
slicing a sorted dict should return a sorted dict.

A sorted dict is the same data structure as a Berkeley DB table. Why
not use the term 'table' instead of 'sorteddict'?

Unless I am missing something big, the implementation of new key
insert for sorteddict looks inefficient - on average you have to move
half of your elements over one space. A B tree or B+ tree (or other
tree) would be a more efficient algorithm.

-1 for ordered dict
+1 for sorted dict

David

From g.brandl at gmx.net  Sun Jun 15 09:11:43 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 15 Jun 2008 09:11:43 +0200
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <4854A9BD.5040905@gmail.com>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>		<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>		<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>		<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>		<18513.16097.394123.312491@montanaro-dyndns-org.local>		<485235B4.6030401@gmail.com>		<18514.41043.913466.680598@montanaro-dyndns-org.local>		<89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1>		<1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com>		<ca471dc20806131014g25fcad2du4401e787244601f8@mail.gmail.com>	<1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com>
	<4854A9BD.5040905@gmail.com>
Message-ID: <g32fbo$u0t$1@ger.gmane.org>

Nick Coghlan schrieb:
> Benjamin Peterson wrote:
>> On Fri, Jun 13, 2008 at 12:14 PM, Guido van Rossum <guido at python.org> wrote:
>>> On Fri, Jun 13, 2008 at 9:42 AM, Benjamin Peterson
>>> <musiccomposition at gmail.com> wrote:
>>>> On Fri, Jun 13, 2008 at 11:40 AM, Raymond Hettinger <python at rcn.com> wrote:
>>>>>>   Nick>      def getName(self):
>>>>>>   Nick>          assert self.__initialized, "Thread.__init__() not called"
>>>>>>   Nick>          return self.__name
>>>>> Why is __name private to begin with?  Any reason for the getters and
>>>>> setters?  Why isn't this just an attribute?
>>>> In 3.x, it's just an attribute.
>>> Oh, is it? Where's the PEP with the API redesign? Did I miss some kind
>>> of decision-making process, weighing compatibility concerns against
>>> other issues?
>> 
>> meaning that it only has one underscore. They methods still live.
> 
> That's worse - it means it can now collide with _name attributes on 
> subclasses.
> 
> This has become a lot more complicated than what I thought we were 
> doing: adding PEP 8 compliant aliases for the existing methods without 
> otherwise changing the threading implementation. As far as I can recall, 
> that is all that was contained in the 2.x patch I reviewed.

The move to _attributes instead of __attributes was done quite a while
earlier, in r57222, by Guido himself.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From python at rcn.com  Sun Jun 15 09:12:38 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 15 Jun 2008 00:12:38 -0700
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org><485457D8.2090205@voidspace.org.uk>
	<48548FA0.5020009@acm.org><758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
	<op.ucr078vrhlrjc9@conan>
Message-ID: <BD99F89BF3164159904E42E5D328DC2A@RaymondLaptop1>

From: "Cesare Di Mauro" <cesare at pronto.it>
> The same problem happens with dictionary updates:
>
> d = {}
> d[k1] = v1
> d[k2] = v2
> d[k1] = v3
>
> The last instruction just replaces the existing entry, so I'm +0 for the first result.

There's a difference.  With dicts, the third insertion *replaces* the value while leaving data structure unmolested.  Also, the key 
doesn't update (an equal but identical key doesn't get substituted).

With an odict that preserves insertion order, you're talking about *deleting* the old entry and *appending* the new one, complete 
with both the new key and new value.  If that is the desired behavior, then it becomes impossible to update the value of an odict 
entry while leaving its insertion order unchanged.  What a bummer.

An alternative behavior is to replace the value, leaving the key in its original position.  But then, you've messed-up the 
expectation that v2 occurs before v3 eventhough v3 was inserted later.  This is especially weird because you keep k1 which was 
inserted earlier, not k3 which is equivalent but not necessarily identical.

Neither behavior is de facto, TheRightWay(tm).  Each has its uses.  Each has its oddities.  Each will have its fans who find that 
particular way to be the best and most intuitive.

One other issue arises if you choose the approach where updating a value triggers re-ordering -- the the dict view concept no longer 
translates very well.  With regular dicts, you can update values while iterating.  Losing that would be a bummer too.

I don't favor one over the other.  Am just pointing-out that the proposal is a little more complex than simply wishing for an 
ordered verion of a dictionary and expecting that that wish is self-defining in a way the matches everyone's intuition, use cases, 
and expectations.


Raymond 


From martin at v.loewis.de  Sun Jun 15 09:41:32 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 15 Jun 2008 09:41:32 +0200
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080615T053145-449@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>	<200806150157.23985@news.perlig.de>	<ca471dc20806142026i22064b45k7dc73c80e47045c1@mail.gmail.com>
	<loom.20080615T053145-449@post.gmane.org>
Message-ID: <4854C7AC.4000508@v.loewis.de>

> I compared multiple ordered dicts now (including Babel, Django and the
> C-implementation I mentioned earlier) and implemented a python version
> of the ordered dict as reference implementation:
> 
>    http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py

I find the slicing API surprising. IMO, if you do support slicing, then
a) the start and end indices should be the same ones that you also use
   in regular indexing.
b) the result should be of the same kind as the thing being sliced, i.e.
   an odict.

So I would rather expect

>>> d['c':'spam']
odict.odict([('c', 'd'), ('foo', 'bar')])

The slicing operation that you provide should be spelled as
d.items()[1:3], or, if you don't want to pay the cost of creating
the full items list, then add a method such as d.slice_by_index(1,3).
What's the use case for this operation, anyway?

Regards,
Martin

From ncoghlan at gmail.com  Sun Jun 15 09:49:25 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Jun 2008 17:49:25 +1000
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <g32fbo$u0t$1@ger.gmane.org>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>		<222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org>		<e04bdf310806120529h72400909je758ad9e5cac9edd@mail.gmail.com>		<367889A5-6BD0-4106-948D-B5B181D4466D@python.org>		<18513.16097.394123.312491@montanaro-dyndns-org.local>		<485235B4.6030401@gmail.com>		<18514.41043.913466.680598@montanaro-dyndns-org.local>		<89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1>		<1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com>		<ca471dc20806131014g25fcad2du4401e787244601f8@mail.gmail.com>	<1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com>	<4854A9BD.5040905@gmail.com>
	<g32fbo$u0t$1@ger.gmane.org>
Message-ID: <4854C985.5000201@gmail.com>

Georg Brandl wrote:
> Nick Coghlan schrieb:
>> This has become a lot more complicated than what I thought we were 
>> doing: adding PEP 8 compliant aliases for the existing methods without 
>> otherwise changing the threading implementation. As far as I can 
>> recall, that is all that was contained in the 2.x patch I reviewed.
> 
> The move to _attributes instead of __attributes was done quite a while
> earlier, in r57222, by Guido himself.

Well, that would explain why it wasn't in the patch I reviewed then :)

Raymond's right that a bunch of those methods are fairly pointless 
though (relics of the module's Java origins I guess, just like the 
camelCase naming).

Perhaps PEP 371 should be expanded a bit to give some details on the 
cleaned up threading API, as well as a deprecation schedule for the old API?

My suggestion would be that we keep the old API around in 3.0 and just 
give it a vanilla PendingDeprecationWarning, then look at actually 
deprecating it in 3.1 for removal in 3.2 - by that time, the need for 
compatibility with versions prior to 2.6 should be fairly rare.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From 2008a at usenet.alexanderweb.de  Sun Jun 15 09:53:08 2008
From: 2008a at usenet.alexanderweb.de (Alexander Schremmer)
Date: Sun, 15 Jun 2008 09:53:08 +0200
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org><485457D8.2090205@voidspace.org.uk>
	<48548FA0.5020009@acm.org>
	<758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
	<loom.20080615T060333-203@post.gmane.org>
Message-ID: <5cuei5-dd7.ln1@tuxpaddy.alexanderweb.homeip.net>

Armin Ronacher wrote:

> That's true, but by now there are countless of ordered dict
> implementations with a mostly-compatible interface and applications and
> libraries are using them already.

Even worse, most of them are slow, i.e. show a wrong algorithmic
complexity ...

> I have an example implementation here that incorporates the ideas
> from ordereddict, Django's OrderedDict and Babel's odict:
> 
>     http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py

... like your implementation. It is not too hard to get the delitem
O(log n) compared to your O(n), see here:

http://codespeak.net/svn/user/arigo/hack/pyfuse/OrderedDict.py

So many people are implementing this kind of data type but do not really
care about making as fast as it could be ... IMHO yet another reason to
ship a usable implementation with Python.

Kind regards,
Alexander


From ncoghlan at gmail.com  Sun Jun 15 10:03:02 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Jun 2008 18:03:02 +1000
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <BD99F89BF3164159904E42E5D328DC2A@RaymondLaptop1>
References: <loom.20080614T231548-103@post.gmane.org><485457D8.2090205@voidspace.org.uk>	<48548FA0.5020009@acm.org><758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>	<op.ucr078vrhlrjc9@conan>
	<BD99F89BF3164159904E42E5D328DC2A@RaymondLaptop1>
Message-ID: <4854CCB6.9050308@gmail.com>

Raymond Hettinger wrote:
> I don't favor one over the other.  Am just pointing-out that the 
> proposal is a little more complex than simply wishing for an ordered 
> verion of a dictionary and expecting that that wish is self-defining in 
> a way the matches everyone's intuition, use cases, and expectations.

If you have an odict with first-insertion ordering, it's fairly trivial 
to convert it to a dictionary with last-insertion ordering:

class odictlastinsertion(odict):
   def __setitem__(self, k, v):
       self.pop(k, None)
       self[k] = v

As you note, going the other way would be rather difficult, suggesting 
that the version ordered by the first key insertion is the more 
fundamental structure.

A PEP would definitely be needed to thrash out those kind of issues and 
decisions though

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sun Jun 15 10:14:12 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Jun 2008 18:14:12 +1000
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com>
References: <loom.20080614T231548-103@post.gmane.org>
	<09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com>
Message-ID: <4854CF54.7090506@gmail.com>

dbpokorny at gmail.com wrote:
> -1 for ordered dict
> +1 for sorted dict

Build the ordered dict, then sort it in-place afterwards, just like a 
list (alternatively, build a normal dict then feed sorted(orig.items()) 
to the ordered dict constructor).

The point of an ordered dict is that unlike a normal dict, the order the 
keys are returned is well defined at the interface level, rather than 
arbitrary and implementation-defined.

David Wolever's example of a normal dict() with an associated key list 
is an excellent way of thinking about the proposed semantics.

Having a fast ordered dict in the standard library also opens up the 
possibility of eventually using such a thing for keyword arguments at 
some point in the future. How nice would it be to be able to just write:

  t = namedtuple(a=1, b=2, c=3)

instead of:

   c = namedtuple("a b c")
   t = c(a=1, b=2, c=3)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From steve at pearwood.info  Sun Jun 15 10:38:55 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 15 Jun 2008 18:38:55 +1000
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <B27A70C2-E5DD-42B0-8D0D-F038C5A446E7@cs.toronto.edu>
References: <loom.20080614T231548-103@post.gmane.org>
	<B27A70C2-E5DD-42B0-8D0D-F038C5A446E7@cs.toronto.edu>
Message-ID: <200806151838.56078.steve@pearwood.info>

On Sun, 15 Jun 2008 02:59:51 pm David Wolever wrote:

> And, as far as questions about the definition of an ordered
> dictionary, is there any good reason not to simply treat the dict as
> a list?  Something like (with the obvious bits left out):

Yes. 

(1) If you implement the ordered dict as a list of key/value pairs, then 
you get order for free, but searching is slow, and so is deletion. If 
we wanted O(N) searches, we'd just use a list of tuples :)

(2) If you implement it as a hash table plus a list of keys in insertion 
order, then searching the dict is fast, but deletions are slow. Also 
you double (?) the amount of memory needed for the keys.


On the other hand... a tree-based implementation would require more 
work, and many more decisions (what kind of tree?), would lose the O(1) 
behaviour of the hash table, and may end up using just as much memory. 
So I wouldn't discount your suggestion.


-- 
Steven

From lgautier at gmail.com  Sun Jun 15 10:42:32 2008
From: lgautier at gmail.com (laurent)
Date: Sun, 15 Jun 2008 10:42:32 +0200
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <BD99F89BF3164159904E42E5D328DC2A@RaymondLaptop1>
References: <loom.20080614T231548-103@post.gmane.org>
	<485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org>
	<758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
	<op.ucr078vrhlrjc9@conan>
	<BD99F89BF3164159904E42E5D328DC2A@RaymondLaptop1>
Message-ID: <1213519352.6748.20.camel@hot-spring>

On Sun, 2008-06-15 at 00:12 -0700, Raymond Hettinger wrote:
> From: "Cesare Di Mauro" <cesare at pronto.it>
> > The same problem happens with dictionary updates:
> >
> > d = {}
> > d[k1] = v1
> > d[k2] = v2
> > d[k1] = v3
> >
> > The last instruction just replaces the existing entry, so I'm +0 for the first result.
> 
> There's a difference.  With dicts, the third insertion *replaces* the value while leaving data structure unmolested.  Also, the key 
> doesn't update (an equal but identical key doesn't get substituted).
> 
> With an odict that preserves insertion order, you're talking about *deleting* the old entry and *appending* the new one, complete 
> with both the new key and new value.  If that is the desired behavior, then it becomes impossible to update the value of an odict 
> entry while leaving its insertion order unchanged.  What a bummer.
> 
> An alternative behavior is to replace the value, leaving the key in its original position.  But then, you've messed-up the 
> expectation that v2 occurs before v3 eventhough v3 was inserted later.  This is especially weird because you keep k1 which was 
> inserted earlier, not k3 which is equivalent but not necessarily identical.

An other behavior making sense would be to append a new position for the
key. With your example above:

d.index(k1) = [0, 2]
d.index(k2) = [1, ]

Deletion of a key would have to be explicit (and
delete all associated indexes).

An other way to think of such a structure would be as a list, for which
each item would also have an associated key. 
?I think someone mentioned the link with a list during this thread, but
here I am not referring to implementation, but the feature of a
container-like class that would allow to access elements either by
position (index), and that would be a one-to-one association, or key,
and that would be a one-to-possibly-many association.



> Neither behavior is de facto, TheRightWay(tm).  Each has its uses.  Each has its oddities.  Each will have its fans who find that 
> particular way to be the best and most intuitive.
> 
> One other issue arises if you choose the approach where updating a value triggers re-ordering -- the the dict view concept no longer 
> translates very well.  With regular dicts, you can update values while iterating.  Losing that would be a bummer too.
> 
> I don't favor one over the other.  Am just pointing-out that the proposal is a little more complex than simply wishing for an 
> ordered verion of a dictionary and expecting that that wish is self-defining in a way the matches everyone's intuition, use cases, 
> and expectations.
> 
> 
> Raymond 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/lgautier%40gmail.com


From steve at pearwood.info  Sun Jun 15 10:44:13 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 15 Jun 2008 18:44:13 +1000
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <BD99F89BF3164159904E42E5D328DC2A@RaymondLaptop1>
References: <loom.20080614T231548-103@post.gmane.org> <op.ucr078vrhlrjc9@conan>
	<BD99F89BF3164159904E42E5D328DC2A@RaymondLaptop1>
Message-ID: <200806151844.13685.steve@pearwood.info>

On Sun, 15 Jun 2008 05:12:38 pm Raymond Hettinger wrote:

> With an odict that preserves insertion order, you're talking about
> *deleting* the old entry and *appending* the new one, complete with
> both the new key and new value.

I certainly don't consider updating an ordered dictionary entry as a 
deletion followed by an append. In fact, I'd be very surprised, and 
dismayed, if that was the default behaviour.

Conceptually, I would expect the following behaviour:

>>> od = odict()
>>> od[1] = 'spam'  # insert a new key
>>> od[2] = 'parrot'  # insert a new key
>>> od[1] = 'ham'  # modify existing key
>>> od.items()
[(1, 'ham'), (2, 'parrot')]

If I wanted the alternative behaviour, I could easily get it:

>>> od = odict()
>>> od[1] = 'spam'  # insert a new key
>>> od[2] = 'parrot'  # insert a new key
>>> od.pop(1, None); od[1] = 'ham'  # remove and re-insert a key
>>> od.items()
[(2, 'parrot'), (1, 'ham')]


Earlier, Raymond also asked what to do about keys with equal but not 
identical keys. Since I consider setting the value to be an update 
rather than a deletion plus re-insertion, then the behaviour is 
obvious.

>>> od = odict([(1, 'norwegian blue'), (2, 'parrot')])
>>> od[1.0] = 'norwegian red'
>>> od.items()
[(1, 'norwegian red'), (2, 'parrot')]

This is close to the behaviour of regular dicts, and to do differently 
would be very surprising to me. Again, anyone who wants the alternative 
behaviour can get it easily, with a pop and a set.

+1 for an ordered dictionary.

As for a sorted dictionary, I don't care much, so +0. You can always 
sort the keys when you need them.


-- 
Steven

From martin at v.loewis.de  Sun Jun 15 10:48:25 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Jun 2008 10:48:25 +0200
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <5cuei5-dd7.ln1@tuxpaddy.alexanderweb.homeip.net>
References: <loom.20080614T231548-103@post.gmane.org><485457D8.2090205@voidspace.org.uk>	<48548FA0.5020009@acm.org>	<758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>	<loom.20080615T060333-203@post.gmane.org>
	<5cuei5-dd7.ln1@tuxpaddy.alexanderweb.homeip.net>
Message-ID: <4854D759.2040202@v.loewis.de>

> ... like your implementation. It is not too hard to get the delitem
> O(log n) compared to your O(n), see here:
> 
> http://codespeak.net/svn/user/arigo/hack/pyfuse/OrderedDict.py
> 
> So many people are implementing this kind of data type but do not really
> care about making as fast as it could be ... IMHO yet another reason to
> ship a usable implementation with Python.

If you use a linked list instead of Python list, you can even do
deletion in O(1). Of course, memory consumption will be higher.

Regards,
Martin

From steve at pearwood.info  Sun Jun 15 11:03:15 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 15 Jun 2008 19:03:15 +1000
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <1213519352.6748.20.camel@hot-spring>
References: <loom.20080614T231548-103@post.gmane.org>
	<BD99F89BF3164159904E42E5D328DC2A@RaymondLaptop1>
	<1213519352.6748.20.camel@hot-spring>
Message-ID: <200806151903.16005.steve@pearwood.info>

On Sun, 15 Jun 2008 06:42:32 pm laurent wrote:
> An other way to think of such a structure would be as a list, for
> which each item would also have an associated key.
> ?I think someone mentioned the link with a list during this thread,
> but here I am not referring to implementation, but the feature of a
> container-like class that would allow to access elements either by
> position (index), and that would be a one-to-one association, or key,
> and that would be a one-to-possibly-many association.

I think the quickest way to kill this proposal again will be to start 
overloading it with more and more complicated behaviour. Python dicts 
are one-to-one (ignoring edge cases like dict[1] vs dict[1.0]). If you 
want one-to-many, you can subclass dict and get that behaviour.

I think that an ordered dict in the standard library should follow the 
same philosophy: define the simplest, most fundamental behaviour which 
is an ordered dictionary, and then let people sub-class it to make more 
complicated types. So I vote -1 on one-to-many associations, and +1 to 
one-to-one like regular dicts.

As for a API to access items by position rather than by key, I'm neutral 
on it. You can always get the nth key by extracting the keys into a 
list. Provided it doesn't become a point of contention, then I'm +0 on 
giving the ordered dict index-based methods in addition to the 
key-based methods, but if it becomes a sticking point and puts the 
whole concept in jeopardy, then I vote -1 on the index-based API. This 
applies also to slicing.


-- 
Steven

From nd at perlig.de  Sun Jun 15 11:31:54 2008
From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=)
Date: Sun, 15 Jun 2008 11:31:54 +0200
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <ca471dc20806142026i22064b45k7dc73c80e47045c1@mail.gmail.com>
References: <loom.20080614T231548-103@post.gmane.org>
	<200806150157.23985@news.perlig.de>
	<ca471dc20806142026i22064b45k7dc73c80e47045c1@mail.gmail.com>
Message-ID: <200806151131.55036@news.perlig.de>

* Guido van Rossum wrote:

> On Sat, Jun 14, 2008 at 4:57 PM, Andr? Malo <nd at perlig.de> wrote:
> > * Armin Ronacher wrote:
> >> Some reasons why ordered dicts are a useful feature:
> >>
> >>   - in XML/HTML processing it's often desired to keep the attributes
> >> of an tag ordered during processing.  So that input ordering is the
> >> same as the output ordering.
> >>
> >>   - Form data transmitted via HTTP is usually ordered by the position
> >>     of the input/textarea/select field in the HTML document.  That
> >>     information is currently lost in most Python web applications /
> >>     frameworks.
> >>
> >>   - Eaiser transition of code from Ruby/PHP which have sorted
> >>     associative arrays / hashmaps.
> >>
> >>   - Having an ordered dict in the standard library would allow other
> >>     libraries support them.  For example a PHP serializer could return
> >>     odicts rather then dicts which drops the ordering information.
> >>     XML libraries such as etree could add support for it when creating
> >>     elements or return attribute dicts.
> >
> > I find this collection of cases pretty weak as an argument for
> > implementing that in the stdlib. A lot of special purpose types would
> > fit into such reasoning, but do you want to have all of them maintained
> > here?
>
> No, but an ordered dict happens to be a *very* common thing to need,
> for a variety of reasons. So I'm +0.5 on adding this to the
> collections module. However someone needs to contribute working code.
> It would also be useful to verify that it actually fulfills the needs
> of some actual use case. Perhaps looking at how Django uses its
> version would be helpful.

FWIW, I'm working a lot in the contexts described above and I never needed 
ordered dicts so far (what do I have to do in order to need them?). I've 
found myself implementing, for example, mutlivaluedicts instead, several 
times.

nd
-- 
Real programmers confuse Christmas and Halloween because
DEC 25 = OCT 31.  -- Unknown

                                      (found in ssl_engine_mutex.c)

From armin.ronacher at active-4.com  Sun Jun 15 11:39:05 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 15 Jun 2008 09:39:05 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org> <op.ucr078vrhlrjc9@conan>
	<BD99F89BF3164159904E42E5D328DC2A@RaymondLaptop1>
	<200806151844.13685.steve@pearwood.info>
Message-ID: <loom.20080615T093754-59@post.gmane.org>

Steven D'Aprano <steve <at> pearwood.info> writes:

> Conceptually, I would expect the following behaviour:
> 
> >>> od = odict()
> >>> od[1] = 'spam'  # insert a new key
> >>> od[2] = 'parrot'  # insert a new key
> >>> od[1] = 'ham'  # modify existing key
> >>> od.items()
> [(1, 'ham'), (2, 'parrot')]
That behavior is different to any ordered-dict implementation
out there ;-)

Regards,
Armin


From armin.ronacher at active-4.com  Sun Jun 15 12:06:34 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 15 Jun 2008 10:06:34 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org><485457D8.2090205@voidspace.org.uk>
	<48548FA0.5020009@acm.org>
	<758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
	<loom.20080615T060333-203@post.gmane.org>
	<5cuei5-dd7.ln1@tuxpaddy.alexanderweb.homeip.net>
Message-ID: <loom.20080615T100453-459@post.gmane.org>

Hi,

Alexander Schremmer <2008a <at> usenet.alexanderweb.de> writes:

> Even worse, most of them are slow, i.e. show a wrong algorithmic
> complexity ...
I already said that in the initial post.

> > I have an example implementation here that incorporates the ideas
> > from ordereddict, Django's OrderedDict and Babel's odict:
> > 
> >     http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py
> 
> ... like your implementation. It is not too hard to get the delitem
> O(log n) compared to your O(n), see here:
That implementation is intended as example implementation for a possible
API not one you should actually use.

Regards,
Armin


From armin.ronacher at active-4.com  Sun Jun 15 12:13:26 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 15 Jun 2008 10:13:26 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org>
Message-ID: <loom.20080615T101201-207@post.gmane.org>

There are far more responses for that topic than I imagined so I would love
to write a PEP about that topic, incorporating the ideas/questions and
suggestions discussed here.


Regards,
Armin


From armin.ronacher at active-4.com  Sun Jun 15 12:11:39 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 15 Jun 2008 10:11:39 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org>	<200806150157.23985@news.perlig.de>	<ca471dc20806142026i22064b45k7dc73c80e47045c1@mail.gmail.com>
	<loom.20080615T053145-449@post.gmane.org>
	<4854C7AC.4000508@v.loewis.de>
Message-ID: <loom.20080615T100911-940@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:

> 
> > I compared multiple ordered dicts now (including Babel, Django and the
> > C-implementation I mentioned earlier) and implemented a python version
> > of the ordered dict as reference implementation:
> > 
> >    http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py
> 
> I find the slicing API surprising. IMO, if you do support slicing, then
> a) the start and end indices should be the same ones that you also use
>    in regular indexing.
> b) the result should be of the same kind as the thing being sliced, i.e.
>    an odict.
> 
> So I would rather expect
> 
> >>> d['c':'spam']
> odict.odict([('c', 'd'), ('foo', 'bar')])
> 
> The slicing operation that you provide should be spelled as
> d.items()[1:3], or, if you don't want to pay the cost of creating
> the full items list, then add a method such as d.slice_by_index(1,3).
> What's the use case for this operation, anyway?
The use case in my particular case is a ordered dict for messages of a
.po file I want to display page-wise in an application.  However I agree
that it's not useful most of the time so dropping it makes sense.  If an
application really needs slicing it can still subclass it and implement
that.

Furthermore with dict-views in Python 3 it would be possible to implement
an efficient slicing operation on the dict view returned.

Regards,
Armin


From python at rcn.com  Sun Jun 15 12:49:34 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 15 Jun 2008 03:49:34 -0700
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org>
	<loom.20080615T101201-207@post.gmane.org>
Message-ID: <590A66CD9F4648BB8127A1CBAABA93EA@RaymondLaptop1>

From: "Armin Ronacher" <armin.ronacher at active-4.com>
> There are far more responses for that topic than I imagined so I would love
> to write a PEP about that topic, incorporating the ideas/questions and
> suggestions discussed here.

Instead of going straight to a PEP, I recommend opening a new wiki page
on the topic, letting people post sample pure python implementations,
pros/cons of various APIs, showing sample use cases, and analyzing
the O(n) behavior of various implementation strategies.

As a reference point, the collections.namedtuple() recipe was much simpler
but took over six months to mature.  Its development process started
by combining the best of all previously posted attempts at the same
thing, then it was exercised heavily in real apps, then it was posted
on ASPN and underwent significant improvements based on feedback
from a number of expert developers.  Then, it was proposed on python-dev
and improved again based on feedback received there.  Upon writing
the docs and creating examples, more refinements ensued.  Upon applying
it to the standard library, more was learned.  After the alpha, we started
getting user feedback and it got refined even further.


Raymond

From barry at python.org  Sun Jun 15 14:01:40 2008
From: barry at python.org (Barry Warsaw)
Date: Sun, 15 Jun 2008 08:01:40 -0400
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
References: <loom.20080614T231548-103@post.gmane.org><485457D8.2090205@voidspace.org.uk>
	<48548FA0.5020009@acm.org>
	<758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
Message-ID: <B0C795FD-C1E8-4C93-A98F-BE24F1A16859@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 15, 2008, at 1:43 AM, Raymond Hettinger wrote:

> For an insertion order dictionary, there was also a question about
> how to handle duplicate keys.
>
> Given odict([(k1,v1), (k2,v2), (k1,v3)]), what should the  
> odict.items() return?
>
>  [(k1,v3), (k2,v2)]
>  [(k2,v2), (k1,v3)]
>
> The first maintains the original sort order and is consistent with  
> the usual
> idea that d[k]=v simply replaces the value but does not alter the  
> hash table.
> It is a bit weird though because v3 appears earlier than v2 which was
> inserted earlier.  It's especially weird for keys that are equal but  
> not
> identical: d[0]=v1  d[0.0]=v3.  Do you want 0.0 to remain associated
> with v3 or should the items list contain a pair that was not in the  
> original
> insertion list, (0, v3)?
>
> The second one is a bit weird because a key "loses its place" whenever
> the value is updated.

Heh, neither of these would work for the email package's own flavor of  
"ordered" dictionary.  Its .items() will return all three key/val  
pairs, but it's __getitem__ interface would only return the first two,  
and there are additional interfaces exposed for .get_all() (which is  
like .get() but returns a list of values matching the given key).

Okay, so email couldn't use whatever stdlib odict gets added, but I  
think that just shows that this may not be as fundamental a data  
structure as we think it is.  I'd much rather see a package on the  
cheeseshop gain overwhelming popularity, practically forcing us to  
include it in the stdlib.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFUEpHEjvBPtnXfVAQIj9QQAtuvlC5YtcSBsddztqD8kwokSrvKz7Nef
oM0JpxjVBQ7oT0F9MnWEvu9Rf8aTVdXsR/zWTf0yw1jt4HtM50Yu4RGqyjghFJ/Z
+Gz+hAqyCerJE6Y9AlW4UdJbQ47wD/Sp1AbMafHCubbt3Sp1AxKmr1tN84WAFefw
8rkP6LbpP64=
=dwAi
-----END PGP SIGNATURE-----

From stefan_ml at behnel.de  Sun Jun 15 14:40:36 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 15 Jun 2008 14:40:36 +0200
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080614T231548-103@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>
Message-ID: <g332k5$gbu$1@ger.gmane.org>

Armin Ronacher wrote:
>   - in XML/HTML processing it's often desired to keep the attributes of
>     an tag ordered during processing.  So that input ordering is the
>     same as the output ordering.
> 
>   - [...]
>     XML libraries such as etree could add support for it when creating
>     elements or return attribute dicts.

I hope that what you call "etree" here (I assume you mean ElementTree) would
stay compatible to lxml.etree in that case. Meaning: adding or replacing an
attribute removes a possibly existing entry and appends the new value at the end.

Stefan


From jnoller at gmail.com  Sun Jun 15 14:59:23 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Sun, 15 Jun 2008 08:59:23 -0400
Subject: [Python-Dev] PEP 8 and optional underscores
In-Reply-To: <4854C985.5000201@gmail.com>
References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1>
	<485235B4.6030401@gmail.com>
	<18514.41043.913466.680598@montanaro-dyndns-org.local>
	<89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1>
	<1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com>
	<ca471dc20806131014g25fcad2du4401e787244601f8@mail.gmail.com>
	<1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com>
	<4854A9BD.5040905@gmail.com> <g32fbo$u0t$1@ger.gmane.org>
	<4854C985.5000201@gmail.com>
Message-ID: <4222a8490806150559qfe7496er184af3af3043bb37@mail.gmail.com>

On Sun, Jun 15, 2008 at 3:49 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Georg Brandl wrote:
>>
>> Nick Coghlan schrieb:
>>>
>>> This has become a lot more complicated than what I thought we were doing:
>>> adding PEP 8 compliant aliases for the existing methods without otherwise
>>> changing the threading implementation. As far as I can recall, that is all
>>> that was contained in the 2.x patch I reviewed.
>>
>> The move to _attributes instead of __attributes was done quite a while
>> earlier, in r57222, by Guido himself.
>
> Well, that would explain why it wasn't in the patch I reviewed then :)
>
> Raymond's right that a bunch of those methods are fairly pointless though
> (relics of the module's Java origins I guess, just like the camelCase
> naming).
>
> Perhaps PEP 371 should be expanded a bit to give some details on the cleaned
> up threading API, as well as a deprecation schedule for the old API?
>
> My suggestion would be that we keep the old API around in 3.0 and just give
> it a vanilla PendingDeprecationWarning, then look at actually deprecating it
> in 3.1 for removal in 3.2 - by that time, the need for compatibility with
> versions prior to 2.6 should be fairly rare.
>
> Cheers,
> Nick.

Rather than piling onto pep 371, perhaps a secondary PEP covering the
renaming of *both* modules and the cleaning up the new API should be
done, that way pep 371 can simply conform to the latter?

-jesse

From tom at vector-seven.com  Sun Jun 15 15:11:36 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Sun, 15 Jun 2008 23:11:36 +1000
Subject: [Python-Dev] xmlrpclib.{True, False} (was Re:  Assignment to None)
In-Reply-To: <484EA327.2050704@vector-seven.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk>	<484D9D59.903@v.loewis.de>
	<484EA327.2050704@vector-seven.com>
Message-ID: <48551508.5060500@vector-seven.com>

My work on the AST optimizer has led me down the path of attempting to 
replace things like Name("True") with Const(Py_True) nodes. This works 
fine most of the time, with the exception of the xmlrpclib module, where 
True and False are actually redefined:

    True, False = True, False

As I stated in an earlier email, the optimizer tries to replace the 
tuple of Name nodes on the LHS with Py_True and Py_False respectively, 
which has the net effect of removing xmlrpclib.{True, False}. Obviously 
undesirable.

The simplest options I can think of to remedy this:

1. A setattr hack: setattr(__import__(__name__), "True", True)
2. Remove all optimization of Name("True") and Name("False")
3. Skip AST optimization entirely for the LHS of Assignment nodes 
(effectively removing any optimization of the "targets" tuple)

I'm leaning towards #3 at the moment as it seems like it's going to be 
the cleanest approach and makes a lot of sense -- at least on the 
surface. Can anybody think of problems with this approach?

Cheers,
T

Thomas Lee wrote:
> Martin v. L?wis wrote:
>>> The question is, what is the specification for Python.
>>>     
>>
>> Now, that's a more interesting question than the question originally
>> asked (which I interpreted as "why does it work the way it works").
>>
>> The only indication in the specification of that feature I could find
>> was:
>>
>> http://docs.python.org/dev/library/constants.html
>>
>> "Changed in version 2.4: Assignments to None are illegal and raise a
>> SyntaxError."
>>
>> Now, given that this talks about the built-in namespace, this *doesn't*
>> specify that foo.None=1 should also raise a syntax error.
>>
>> So the implementation apparently deviates from the specification.
>>
>> In Python 3, None, True, and False are keywords, so clearly, the
>> intended semantics is also the implemented one (and the language
>> description for 2.x needs to be updated/clarified).
>>
>>   
> Interestingly enough, the semantics of True, False and None are 
> different from one another in 2.6:
>
> True = "blah" and False = 6 are perfectly legal in Python <=2.6.
>
> Funny, I just ran into this. I was trying to figure out why the AST 
> optimization code was breaking test_xmlrpc ... turns out xmlrpclib 
> defines xmlrpclib.True and xmlrpclib.False and the optimizer was 
> trying to resolve them as constants while compiling the module. Ouch.
>
> What happened in 3k? Were the constants in xmlrpclib renamed/removed?
>
> Cheers,
> T
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com


From musiccomposition at gmail.com  Sun Jun 15 15:33:02 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sun, 15 Jun 2008 08:33:02 -0500
Subject: [Python-Dev] xmlrpclib.{True,
	False} (was Re: Assignment to None)
In-Reply-To: <48551508.5060500@vector-seven.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>
	<200806091348.44361.steve@pearwood.info>
	<484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de>
	<484EA327.2050704@vector-seven.com>
	<48551508.5060500@vector-seven.com>
Message-ID: <1afaf6160806150633n76e2f9d2ud83be8d5d30786b@mail.gmail.com>

On Sun, Jun 15, 2008 at 8:11 AM, Thomas Lee <tom at vector-seven.com> wrote:
>
> The simplest options I can think of to remedy this:
>
> 1. A setattr hack: setattr(__import__(__name__), "True", True)
> 2. Remove all optimization of Name("True") and Name("False")
> 3. Skip AST optimization entirely for the LHS of Assignment nodes
> (effectively removing any optimization of the "targets" tuple)

You're working on optimization for the 2.6 branch, correct? In that
case, why don't we take option 3 in 2.x and just reenable it in 3.x
where it's completely forbidden to assign to True or False?

>
> I'm leaning towards #3 at the moment as it seems like it's going to be the
> cleanest approach and makes a lot of sense -- at least on the surface. Can
> anybody think of problems with this approach?
>
> Cheers,
> T



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From tom at vector-seven.com  Sun Jun 15 15:34:14 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Sun, 15 Jun 2008 23:34:14 +1000
Subject: [Python-Dev] xmlrpclib.{True,
 False} (was Re:  Assignment to None)
In-Reply-To: <48551508.5060500@vector-seven.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk>	<484D9D59.903@v.loewis.de>	<484EA327.2050704@vector-seven.com>
	<48551508.5060500@vector-seven.com>
Message-ID: <48551A56.1020704@vector-seven.com>

Option 4 just struck me: only optimize Name nodes if they have a Load 
ctx. This makes even more sense: in a Store context, we almost 
invariably want the name rather than the constant.

Cheers,
T

Thomas Lee wrote:
> My work on the AST optimizer has led me down the path of attempting to 
> replace things like Name("True") with Const(Py_True) nodes. This works 
> fine most of the time, with the exception of the xmlrpclib module, 
> where True and False are actually redefined:
>
>    True, False = True, False
>
> As I stated in an earlier email, the optimizer tries to replace the 
> tuple of Name nodes on the LHS with Py_True and Py_False respectively, 
> which has the net effect of removing xmlrpclib.{True, False}. 
> Obviously undesirable.
>
> The simplest options I can think of to remedy this:
>
> 1. A setattr hack: setattr(__import__(__name__), "True", True)
> 2. Remove all optimization of Name("True") and Name("False")
> 3. Skip AST optimization entirely for the LHS of Assignment nodes 
> (effectively removing any optimization of the "targets" tuple)
>
> I'm leaning towards #3 at the moment as it seems like it's going to be 
> the cleanest approach and makes a lot of sense -- at least on the 
> surface. Can anybody think of problems with this approach?
>
> Cheers,
> T
>
> Thomas Lee wrote:
>> Martin v. L?wis wrote:
>>>> The question is, what is the specification for Python.
>>>>     
>>>
>>> Now, that's a more interesting question than the question originally
>>> asked (which I interpreted as "why does it work the way it works").
>>>
>>> The only indication in the specification of that feature I could find
>>> was:
>>>
>>> http://docs.python.org/dev/library/constants.html
>>>
>>> "Changed in version 2.4: Assignments to None are illegal and raise a
>>> SyntaxError."
>>>
>>> Now, given that this talks about the built-in namespace, this *doesn't*
>>> specify that foo.None=1 should also raise a syntax error.
>>>
>>> So the implementation apparently deviates from the specification.
>>>
>>> In Python 3, None, True, and False are keywords, so clearly, the
>>> intended semantics is also the implemented one (and the language
>>> description for 2.x needs to be updated/clarified).
>>>
>>>   
>> Interestingly enough, the semantics of True, False and None are 
>> different from one another in 2.6:
>>
>> True = "blah" and False = 6 are perfectly legal in Python <=2.6.
>>
>> Funny, I just ran into this. I was trying to figure out why the AST 
>> optimization code was breaking test_xmlrpc ... turns out xmlrpclib 
>> defines xmlrpclib.True and xmlrpclib.False and the optimizer was 
>> trying to resolve them as constants while compiling the module. Ouch.
>>
>> What happened in 3k? Were the constants in xmlrpclib renamed/removed?
>>
>> Cheers,
>> T
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com


From tom at vector-seven.com  Sun Jun 15 15:37:25 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Sun, 15 Jun 2008 23:37:25 +1000
Subject: [Python-Dev] xmlrpclib.{True,
	False} (was Re: Assignment to None)
In-Reply-To: <1afaf6160806150633n76e2f9d2ud83be8d5d30786b@mail.gmail.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	
	<200806091348.44361.steve@pearwood.info>	
	<484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de>	
	<484EA327.2050704@vector-seven.com>	
	<48551508.5060500@vector-seven.com>
	<1afaf6160806150633n76e2f9d2ud83be8d5d30786b@mail.gmail.com>
Message-ID: <48551B15.8070702@vector-seven.com>

Benjamin Peterson wrote:
> On Sun, Jun 15, 2008 at 8:11 AM, Thomas Lee <tom at vector-seven.com> wrote:
>   
>> The simplest options I can think of to remedy this:
>>
>> 1. A setattr hack: setattr(__import__(__name__), "True", True)
>> 2. Remove all optimization of Name("True") and Name("False")
>> 3. Skip AST optimization entirely for the LHS of Assignment nodes
>> (effectively removing any optimization of the "targets" tuple)
>>     
>
> You're working on optimization for the 2.6 branch, correct? In that
> case, why don't we take option 3 in 2.x and just reenable it in 3.x
> where it's completely forbidden to assign to True or False?
>
>   
Sorry, that's correct. This is against 2.6 trunk.

That's the idea -- in 3.x this will be a non-issue.

Cheers,
T


From solipsis at pitrou.net  Sun Jun 15 16:19:19 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 15 Jun 2008 14:19:19 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org>
	<09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com>
Message-ID: <loom.20080615T140045-802@post.gmane.org>

dbpokorny <at> gmail.com <dbpokorny <at> gmail.com> writes:
> 
> If you don't like the fact that your web application framework loses
> the order of its key:value pairs relative to the page, then you should
> bring it up with the developers.

Who will then point up that to retain that order while providing you
with a dict-like API, they need to write some kind of ordered dict
implementation. Then you'll complain that their implementation is sub-optimal
and isn't 100% compatible with the original dict API, and post on python-dev
to ask that a standard odict implementation is considered for the stdlib.

> Ordered dicts, dicts that remember the chronological order of their
> insertion, don't sound generally useful.

They are generally useful in any case where you want to handle key-value
pairs while not confusing a human operator by messing up the original order.
Think e.g. configuration files. A common complaint against ConfigParser is
that writing a configuration file does not preserve the order of the original
file, which is harmless for the computer but very annoying for the human
being who maintains that file.

As for sorted dicts with general O(log(n)) behaviour, you could try to combine
the blist type (http://pypi.python.org/pypi/blist/) with the standard bisect
module and see what that gives.



From g.brandl at gmx.net  Sun Jun 15 16:23:27 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 15 Jun 2008 16:23:27 +0200
Subject: [Python-Dev] xmlrpclib.{True,
	False} (was Re:  Assignment to None)
In-Reply-To: <48551A56.1020704@vector-seven.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk>	<484D9D59.903@v.loewis.de>	<484EA327.2050704@vector-seven.com>	<48551508.5060500@vector-seven.com>
	<48551A56.1020704@vector-seven.com>
Message-ID: <g338l9$be$1@ger.gmane.org>

Remember that it must still be possible to write (in 2.6)

True = 0
assert not True

Georg

Thomas Lee schrieb:
> Option 4 just struck me: only optimize Name nodes if they have a Load 
> ctx. This makes even more sense: in a Store context, we almost 
> invariably want the name rather than the constant.
> 
> Cheers,
> T
> 
> Thomas Lee wrote:
>> My work on the AST optimizer has led me down the path of attempting to 
>> replace things like Name("True") with Const(Py_True) nodes. This works 
>> fine most of the time, with the exception of the xmlrpclib module, 
>> where True and False are actually redefined:
>>
>>    True, False = True, False
>>
>> As I stated in an earlier email, the optimizer tries to replace the 
>> tuple of Name nodes on the LHS with Py_True and Py_False respectively, 
>> which has the net effect of removing xmlrpclib.{True, False}. 
>> Obviously undesirable.
>>
>> The simplest options I can think of to remedy this:
>>
>> 1. A setattr hack: setattr(__import__(__name__), "True", True)
>> 2. Remove all optimization of Name("True") and Name("False")
>> 3. Skip AST optimization entirely for the LHS of Assignment nodes 
>> (effectively removing any optimization of the "targets" tuple)
>>
>> I'm leaning towards #3 at the moment as it seems like it's going to be 
>> the cleanest approach and makes a lot of sense -- at least on the 
>> surface. Can anybody think of problems with this approach?
>>
>> Cheers,
>> T
>>
>> Thomas Lee wrote:
>>> Martin v. L?wis wrote:
>>>>> The question is, what is the specification for Python.
>>>>>     
>>>>
>>>> Now, that's a more interesting question than the question originally
>>>> asked (which I interpreted as "why does it work the way it works").
>>>>
>>>> The only indication in the specification of that feature I could find
>>>> was:
>>>>
>>>> http://docs.python.org/dev/library/constants.html
>>>>
>>>> "Changed in version 2.4: Assignments to None are illegal and raise a
>>>> SyntaxError."
>>>>
>>>> Now, given that this talks about the built-in namespace, this *doesn't*
>>>> specify that foo.None=1 should also raise a syntax error.
>>>>
>>>> So the implementation apparently deviates from the specification.
>>>>
>>>> In Python 3, None, True, and False are keywords, so clearly, the
>>>> intended semantics is also the implemented one (and the language
>>>> description for 2.x needs to be updated/clarified).
>>>>
>>>>   
>>> Interestingly enough, the semantics of True, False and None are 
>>> different from one another in 2.6:
>>>
>>> True = "blah" and False = 6 are perfectly legal in Python <=2.6.
>>>
>>> Funny, I just ran into this. I was trying to figure out why the AST 
>>> optimization code was breaking test_xmlrpc ... turns out xmlrpclib 
>>> defines xmlrpclib.True and xmlrpclib.False and the optimizer was 
>>> trying to resolve them as constants while compiling the module. Ouch.
>>>
>>> What happened in 3k? Were the constants in xmlrpclib renamed/removed?
>>>
>>> Cheers,
>>> T
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: 
>>> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
> 


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From pje at telecommunity.com  Sun Jun 15 16:29:05 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 15 Jun 2008 10:29:05 -0400
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080615T140045-802@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>
	<09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com>
	<loom.20080615T140045-802@post.gmane.org>
Message-ID: <20080615142830.0882A3A402D@sparrow.telecommunity.com>

At 02:19 PM 6/15/2008 +0000, Antoine Pitrou wrote:
> > Ordered dicts, dicts that remember the chronological order of their
> > insertion, don't sound generally useful.
>
>They are generally useful in any case where you want to handle key-value
>pairs while not confusing a human operator by messing up the original order.
>Think e.g. configuration files. A common complaint against ConfigParser is
>that writing a configuration file does not preserve the order of the original
>file, which is harmless for the computer but very annoying for the human
>being who maintains that file.

You don't need an ordered dictionary for that; you need a save 
routine that stream-edits the old file contents.  That way, you don't 
lose comments and spacing either.

As for the other uses for ordered dictionaries, I find it simplest to 
just use a list of key,value pairs, and only transform it to a 
dictionary or dictionary-like structure as needed, using tools like 
the cgi module, the email package, or wsgiref.headers.


From solipsis at pitrou.net  Sun Jun 15 16:34:56 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 15 Jun 2008 14:34:56 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org>
	<09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com>
	<loom.20080615T140045-802@post.gmane.org>
	<20080615142830.0882A3A402D@sparrow.telecommunity.com>
Message-ID: <loom.20080615T143212-944@post.gmane.org>

Phillip J. Eby <pje <at> telecommunity.com> writes:
> 
> As for the other uses for ordered dictionaries, I find it simplest to 
> just use a list of key,value pairs, and only transform it to a 
> dictionary or dictionary-like structure as needed, using tools like 
> the cgi module, the email package, or wsgiref.headers.

What you are saying is that there are already generally useful container
types in the stdlib, but isn't it a good argument in favor of ripping them
out of domain-specific packages and provide them as generic classes in the
collections module?

Someone never using cgi or wsgiref wouldn't know that some of the code there
can be useful for other purposes.



From tom at vector-seven.com  Sun Jun 15 16:40:47 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Mon, 16 Jun 2008 00:40:47 +1000
Subject: [Python-Dev] xmlrpclib.{True,
 False} (was Re:  Assignment to None)
In-Reply-To: <g338l9$be$1@ger.gmane.org>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk>	<484D9D59.903@v.loewis.de>	<484EA327.2050704@vector-seven.com>	<48551508.5060500@vector-seven.com>	<48551A56.1020704@vector-seven.com>
	<g338l9$be$1@ger.gmane.org>
Message-ID: <485529EF.3080209@vector-seven.com>

Georg Brandl wrote:
> Remember that it must still be possible to write (in 2.6)
>
> True = 0
> assert not True

Ah of course. Looks like I should just avoid optimizations of 
Name("True") and Name("False") all together. That's a shame!

Cheers,
T
>
> Georg
>
> Thomas Lee schrieb:
>> Option 4 just struck me: only optimize Name nodes if they have a Load 
>> ctx. This makes even more sense: in a Store context, we almost 
>> invariably want the name rather than the constant.
>>
>> Cheers,
>> T
>>
>> Thomas Lee wrote:
>>> My work on the AST optimizer has led me down the path of attempting 
>>> to replace things like Name("True") with Const(Py_True) nodes. This 
>>> works fine most of the time, with the exception of the xmlrpclib 
>>> module, where True and False are actually redefined:
>>>
>>>    True, False = True, False
>>>
>>> As I stated in an earlier email, the optimizer tries to replace the 
>>> tuple of Name nodes on the LHS with Py_True and Py_False 
>>> respectively, which has the net effect of removing xmlrpclib.{True, 
>>> False}. Obviously undesirable.
>>>
>>> The simplest options I can think of to remedy this:
>>>
>>> 1. A setattr hack: setattr(__import__(__name__), "True", True)
>>> 2. Remove all optimization of Name("True") and Name("False")
>>> 3. Skip AST optimization entirely for the LHS of Assignment nodes 
>>> (effectively removing any optimization of the "targets" tuple)
>>>
>>> I'm leaning towards #3 at the moment as it seems like it's going to 
>>> be the cleanest approach and makes a lot of sense -- at least on the 
>>> surface. Can anybody think of problems with this approach?
>>>
>>> Cheers,
>>> T
>>>
>>> Thomas Lee wrote:
>>>> Martin v. L?wis wrote:
>>>>>> The question is, what is the specification for Python.
>>>>>>     
>>>>>
>>>>> Now, that's a more interesting question than the question originally
>>>>> asked (which I interpreted as "why does it work the way it works").
>>>>>
>>>>> The only indication in the specification of that feature I could find
>>>>> was:
>>>>>
>>>>> http://docs.python.org/dev/library/constants.html
>>>>>
>>>>> "Changed in version 2.4: Assignments to None are illegal and raise a
>>>>> SyntaxError."
>>>>>
>>>>> Now, given that this talks about the built-in namespace, this 
>>>>> *doesn't*
>>>>> specify that foo.None=1 should also raise a syntax error.
>>>>>
>>>>> So the implementation apparently deviates from the specification.
>>>>>
>>>>> In Python 3, None, True, and False are keywords, so clearly, the
>>>>> intended semantics is also the implemented one (and the language
>>>>> description for 2.x needs to be updated/clarified).
>>>>>
>>>>>   
>>>> Interestingly enough, the semantics of True, False and None are 
>>>> different from one another in 2.6:
>>>>
>>>> True = "blah" and False = 6 are perfectly legal in Python <=2.6.
>>>>
>>>> Funny, I just ran into this. I was trying to figure out why the AST 
>>>> optimization code was breaking test_xmlrpc ... turns out xmlrpclib 
>>>> defines xmlrpclib.True and xmlrpclib.False and the optimizer was 
>>>> trying to resolve them as constants while compiling the module. Ouch.
>>>>
>>>> What happened in 3k? Were the constants in xmlrpclib renamed/removed?
>>>>
>>>> Cheers,
>>>> T
>>>>
>>>> _______________________________________________
>>>> Python-Dev mailing list
>>>> Python-Dev at python.org
>>>> http://mail.python.org/mailman/listinfo/python-dev
>>>> Unsubscribe: 
>>>> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com 
>>>>
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: 
>>> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com 
>>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org 
>>
>>
>
>


From g.brandl at gmx.net  Sun Jun 15 16:47:41 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 15 Jun 2008 16:47:41 +0200
Subject: [Python-Dev] xmlrpclib.{True,
	False} (was Re:  Assignment to None)
In-Reply-To: <485529EF.3080209@vector-seven.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk>	<484D9D59.903@v.loewis.de>	<484EA327.2050704@vector-seven.com>	<48551508.5060500@vector-seven.com>	<48551A56.1020704@vector-seven.com>	<g338l9$be$1@ger.gmane.org>
	<485529EF.3080209@vector-seven.com>
Message-ID: <g33a2m$46q$1@ger.gmane.org>

Thomas Lee schrieb:
> Georg Brandl wrote:
>> Remember that it must still be possible to write (in 2.6)
>>
>> True = 0
>> assert not True
> 
> Ah of course. Looks like I should just avoid optimizations of 
> Name("True") and Name("False") all together. That's a shame!

We can of course decide to make assignment to True and False
illegal in 2.7 :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From pje at telecommunity.com  Sun Jun 15 16:52:20 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 15 Jun 2008 10:52:20 -0400
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080615T143212-944@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>
	<09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com>
	<loom.20080615T140045-802@post.gmane.org>
	<20080615142830.0882A3A402D@sparrow.telecommunity.com>
	<loom.20080615T143212-944@post.gmane.org>
Message-ID: <20080615145147.31B213A402D@sparrow.telecommunity.com>

At 02:34 PM 6/15/2008 +0000, Antoine Pitrou wrote:
>Phillip J. Eby <pje <at> telecommunity.com> writes:
> >
> > As for the other uses for ordered dictionaries, I find it simplest to
> > just use a list of key,value pairs, and only transform it to a
> > dictionary or dictionary-like structure as needed, using tools like
> > the cgi module, the email package, or wsgiref.headers.
>
>What you are saying is that there are already generally useful container
>types in the stdlib, but isn't it a good argument in favor of ripping them
>out of domain-specific packages and provide them as generic classes in the
>collections module?
>
>Someone never using cgi or wsgiref wouldn't know that some of the code there
>can be useful for other purposes.

I didn't say I used them for other purposes, or that they were 
generally useful.  Rather, they're specifically useful for the things 
they're useful for.

More often than not, the use case calls for not merely ordering, but 
ordering of *values*, with multiple values for each key.  But the 
precise API desired for manipulating such structures tends to be 
highly app-specific (like email, CGI form values, HTTP headers, 
etc.), so it's actually IMO an argument *against* a general odict type.


From tom at vector-seven.com  Sun Jun 15 16:55:40 2008
From: tom at vector-seven.com (Thomas Lee)
Date: Mon, 16 Jun 2008 00:55:40 +1000
Subject: [Python-Dev] xmlrpclib.{True,
 False} (was Re:  Assignment to None)
In-Reply-To: <g33a2m$46q$1@ger.gmane.org>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk>	<484D9D59.903@v.loewis.de>	<484EA327.2050704@vector-seven.com>	<48551508.5060500@vector-seven.com>	<48551A56.1020704@vector-seven.com>	<g338l9$be$1@ger.gmane.org>	<485529EF.3080209@vector-seven.com>
	<g33a2m$46q$1@ger.gmane.org>
Message-ID: <48552D6C.1080507@vector-seven.com>

Georg Brandl wrote:
>
> We can of course decide to make assignment to True and False
> illegal in 2.7 :)
>
> Georg
>

Great to know that's an option. There's little-to-no chance of this 
making 2.6.

I might just avoid trying to treat True/False as "real" constants until 
there's been a proper discussion about changing these semantics -- just 
to get test_xmlrpc passing.

Cheers,
T


From rhamph at gmail.com  Sun Jun 15 18:45:35 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Sun, 15 Jun 2008 10:45:35 -0600
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <B0C795FD-C1E8-4C93-A98F-BE24F1A16859@python.org>
References: <loom.20080614T231548-103@post.gmane.org>
	<485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org>
	<758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
	<B0C795FD-C1E8-4C93-A98F-BE24F1A16859@python.org>
Message-ID: <aac2c7cb0806150945m49a7dc4ay87527d8024833ed9@mail.gmail.com>

On Sun, Jun 15, 2008 at 6:01 AM, Barry Warsaw <barry at python.org> wrote:
> On Jun 15, 2008, at 1:43 AM, Raymond Hettinger wrote:
>> The second one is a bit weird because a key "loses its place" whenever
>> the value is updated.
>
> Heh, neither of these would work for the email package's own flavor of
> "ordered" dictionary.  Its .items() will return all three key/val pairs, but
> it's __getitem__ interface would only return the first two, and there are
> additional interfaces exposed for .get_all() (which is like .get() but
> returns a list of values matching the given key).
>
> Okay, so email couldn't use whatever stdlib odict gets added, but I think
> that just shows that this may not be as fundamental a data structure as we
> think it is.  I'd much rather see a package on the cheeseshop gain
> overwhelming popularity, practically forcing us to include it in the stdlib.

+1 on this comment, -0 on adding an ordered dict to 2.6/3.0.

An ordered (or sorted) dict seems to be more of a gut reaction.  They
have some data in a dict, they realize they want it ordered/sorted for
some purpose, so the first thing they check is if the dict already
provides it.  It doesn't, but putting together a combination of a dict
and a list is often a fair bit of work - nevermind if you want O(1)
removal - so we hear about it in the mean time.

But my point is that we we need to focus on finding real use cases and
exploring how best to solve them.  Otherwise we might as well throw in
my OrderedSet[1] as-is, despite that it's got no comments and no
ratings.  Even I don't seem to use it.

Everybody has an opinion on the colour of this bikeshed.


[1] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/528878

-- 
Adam Olsen, aka Rhamphoryncus

From p.f.moore at gmail.com  Sun Jun 15 18:54:08 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 15 Jun 2008 17:54:08 +0100
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <590A66CD9F4648BB8127A1CBAABA93EA@RaymondLaptop1>
References: <loom.20080614T231548-103@post.gmane.org>
	<loom.20080615T101201-207@post.gmane.org>
	<590A66CD9F4648BB8127A1CBAABA93EA@RaymondLaptop1>
Message-ID: <79990c6b0806150954v24cde6f9v9196db9976717321@mail.gmail.com>

2008/6/15 Raymond Hettinger <python at rcn.com>:
> From: "Armin Ronacher" <armin.ronacher at active-4.com>
>>
>> There are far more responses for that topic than I imagined so I would
>> love
>> to write a PEP about that topic, incorporating the ideas/questions and
>> suggestions discussed here.
>
> Instead of going straight to a PEP, I recommend opening a new wiki page
> on the topic, letting people post sample pure python implementations,
> pros/cons of various APIs, showing sample use cases, and analyzing
> the O(n) behavior of various implementation strategies.

That sounds reasonable. I'm +1 in principle, but would like to see the
details thrashed out.

BTW, as a meta-question, where is this proposal targeted for? I'd
assume 2.6/3.0 are closed now, as we're hitting betas - so are we
looking at 2.7/3.1? (Just a quick sanity check on the timing...)

Paul.

From martin at v.loewis.de  Sun Jun 15 19:02:26 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Jun 2008 19:02:26 +0200
Subject: [Python-Dev] xmlrpclib.{True,
	False} (was Re: Assignment to None)
In-Reply-To: <48551B15.8070702@vector-seven.com>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>		<200806091348.44361.steve@pearwood.info>		<484CE993.5050506@voidspace.org.uk>
	<484D9D59.903@v.loewis.de>		<484EA327.2050704@vector-seven.com>		<48551508.5060500@vector-seven.com>	<1afaf6160806150633n76e2f9d2ud83be8d5d30786b@mail.gmail.com>
	<48551B15.8070702@vector-seven.com>
Message-ID: <48554B22.5010708@v.loewis.de>

> Sorry, that's correct. This is against 2.6 trunk.
> 
> That's the idea -- in 3.x this will be a non-issue.

It's perhaps even less of an issue than you think: In 3.0,
True and False are *already* compiled into constants. So
any additional special casing would make no difference.

Regards,
Martin

From guido at python.org  Sun Jun 15 19:50:13 2008
From: guido at python.org (Guido van Rossum)
Date: Sun, 15 Jun 2008 10:50:13 -0700
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <4854CCB6.9050308@gmail.com>
References: <loom.20080614T231548-103@post.gmane.org>
	<485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org>
	<758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
	<op.ucr078vrhlrjc9@conan>
	<BD99F89BF3164159904E42E5D328DC2A@RaymondLaptop1>
	<4854CCB6.9050308@gmail.com>
Message-ID: <ca471dc20806151050j209d82f7u6752140614f375ec@mail.gmail.com>

On Sun, Jun 15, 2008 at 1:03 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Raymond Hettinger wrote:
>>
>> I don't favor one over the other.  Am just pointing-out that the proposal
>> is a little more complex than simply wishing for an ordered verion of a
>> dictionary and expecting that that wish is self-defining in a way the
>> matches everyone's intuition, use cases, and expectations.
>
> If you have an odict with first-insertion ordering, it's fairly trivial to
> convert it to a dictionary with last-insertion ordering:
>
> class odictlastinsertion(odict):
>  def __setitem__(self, k, v):
>      self.pop(k, None)
>      self[k] = v
>
> As you note, going the other way would be rather difficult, suggesting that
> the version ordered by the first key insertion is the more fundamental
> structure.
>
> A PEP would definitely be needed to thrash out those kind of issues and
> decisions though

Right.

Though on this particular issue, my gut instinct tells me that
first-insertion-order is more useful (matching your assertion above).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From zooko at zooko.com  Sun Jun 15 21:20:19 2008
From: zooko at zooko.com (zooko)
Date: Sun, 15 Jun 2008 12:20:19 -0700
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <ca471dc20806142026i22064b45k7dc73c80e47045c1@mail.gmail.com>
References: <loom.20080614T231548-103@post.gmane.org>
	<200806150157.23985@news.perlig.de>
	<ca471dc20806142026i22064b45k7dc73c80e47045c1@mail.gmail.com>
Message-ID: <5CA3F464-102F-432C-92B3-98145D71DF68@zooko.com>

On Jun 14, 2008, at 8:26 PM, Guido van Rossum wrote:

> No, but an ordered dict happens to be a *very* common thing to need,
> for a variety of reasons. So I'm +0.5 on adding this to the
> collections module. However someone needs to contribute working code.

Here's an LRU cache that I've used a few times over the years:

http://allmydata.org/trac/pyutil/browser/pyutil/pyutil/cache.py

This is just like a dict ordered by insertion, except:

1.  That it removes the oldest entry if it grows beyond a limit.

2.  That it moves an entry to the head of the queue when has_key() is  
called on that item.

So, it would be easy to change those two behaviors in order to use  
this implementation.  There are actually three implementations in  
that file: one that is asymptotically O(1) for all operations (using  
a double-linked list woven into the values of the dict), and one that  
uses a Python list to hold the order, so it is faster for small  
enough dicts.  The third implementation is an implementation that  
someone else wrote that I included just for comparison purposes --  
the comparison showed that each of mine was better.

Regards,

Zooko


From leif.walsh at gmail.com  Sun Jun 15 22:07:37 2008
From: leif.walsh at gmail.com (Leif Walsh)
Date: Sun, 15 Jun 2008 13:07:37 -0700
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <4854A541.5080101@gmail.com>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<4854A541.5080101@gmail.com>
Message-ID: <cc7430500806151307q61088974pd700f11fd3355310@mail.gmail.com>

-1 to this 'on' statement.

On Sat, Jun 14, 2008 at 10:14 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> It isn't that a Pascal-with-style statement can't be achieved, it's that it
> is pointless syntactic sugar in Python. Just use o =
> long_complicated_object_name instead.

+1 to this reason.

-- 
Cheers,
Leif

From zooko at zooko.com  Mon Jun 16 00:02:12 2008
From: zooko at zooko.com (zooko)
Date: Sun, 15 Jun 2008 15:02:12 -0700
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <5CA3F464-102F-432C-92B3-98145D71DF68@zooko.com>
References: <loom.20080614T231548-103@post.gmane.org>
	<200806150157.23985@news.perlig.de>
	<ca471dc20806142026i22064b45k7dc73c80e47045c1@mail.gmail.com>
	<5CA3F464-102F-432C-92B3-98145D71DF68@zooko.com>
Message-ID: <3F26BFF2-5756-4A7F-A2E7-04CF4A44B716@zooko.com>

On Jun 15, 2008, at 12:20 PM, zooko wrote:

> So, it would be easy to change those two behaviors in order to use  
> this implementation.  There are actually three implementations in  
> that file: one that is asymptotically O(1) for all operations  
> (using a double-linked list woven into the values of the dict), and  
> one that uses a Python list to hold the order, so it is faster for  
> small enough dicts.

P.S.  I didn't mean to fall for the common misunderstanding that hash  
table operations are O(1).  What I should have written is that my  
ordered dict technique *adds* only O(1) time to the time of the dict  
on which it is built.

As to the question of how important or common this data structure is,  
I have to admit that while I implemented this one and used it several  
times (always exclusively for LRU caching), I currently don't use it  
for anything.  Nowadays I try to avoid doing transparent caching  
(such as LRU caches are often used for) in favor of explicit  
management of the resource.

Regards,

Zooko


From aleaxit at gmail.com  Mon Jun 16 01:12:46 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Sun, 15 Jun 2008 16:12:46 -0700
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <op.ucr0100rhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org> <op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan> <4854614B.9070903@canterbury.ac.nz>
	<op.ucr0100rhlrjc9@conan>
Message-ID: <e8a0972d0806151612q1382371dg2ebc8c8913a13ef8@mail.gmail.com>

+1 on updating the FAQ.  Maybe we could even have it notice that a
read-only version of the desired semantic for 'with' is easily hacked
with the *current* semantic of 'with'...:

@contextlib.contextmanager
def readonly(anobj):
    caller_globals = sys._getframe(2).f_globals
    saved_globals = caller_globals.copy()
    caller_globals.update((n, getattr(anobj, n)) for n in dir(anobj))
    yield
    caller_globals.clear()
    caller_globals.update(saved_globals)

and then, of course,

with readonly(someobj): ...

(local variables take precedence, and in particular all assignments
define local variables, as usual -- but you can say e.g. 'zz' to mean
'someobj.zz', if you're sufficiently keen on giving up the advantage
of having many well-separated namespaces;-).


Alex


On Sat, Jun 14, 2008 at 11:16 PM, Cesare Di Mauro <cesare at pronto.it> wrote:
> In data 15 giugno 2008 alle ore 02:24:43, Greg Ewing <greg.ewing at canterbury.ac.nz> ha scritto:
>
>> ...and which should *not* be used in most cases, for
>> the same reason.
>>
>> All those tutorials that start out with 'from something
>> import *' are doing a lot of harm to the impressionable
>> minds of new programmers, IMO.
>
> OK, but nobody have questioned about removing 'from something import *' just to help noobs...
> That's because the instruction *can* be useful in *some* (hopely limited, but existent) contexts.
> It's a matter of programmer choises.
>
> Anyway (and dropping my proposal), I think that the FAQ needs to be changed to advice that the
> 'with' keyword in Python makes a completely different kind of work.
>
> Cesare
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
>

From steve at pearwood.info  Mon Jun 16 02:07:17 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 16 Jun 2008 10:07:17 +1000
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080615T093754-59@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>
	<200806151844.13685.steve@pearwood.info>
	<loom.20080615T093754-59@post.gmane.org>
Message-ID: <200806161007.18697.steve@pearwood.info>

On Sun, 15 Jun 2008 07:39:05 pm Armin Ronacher wrote:
> Steven D'Aprano <steve <at> pearwood.info> writes:
> > Conceptually, I would expect the following behaviour:
> > >>> od = odict()
> > >>> od[1] = 'spam'  # insert a new key
> > >>> od[2] = 'parrot'  # insert a new key
> > >>> od[1] = 'ham'  # modify existing key
> > >>> od.items()
> >
> > [(1, 'ham'), (2, 'parrot')]
>
> That behavior is different to any ordered-dict implementation
> out there ;-)

I beg to differ. It's the same behaviour as in this implementation:

http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py

which I understand was written by you.



-- 
Steven

From greg.ewing at canterbury.ac.nz  Mon Jun 16 03:00:28 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 16 Jun 2008 13:00:28 +1200
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <48548FA0.5020009@acm.org>
References: <loom.20080614T231548-103@post.gmane.org>
	<485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org>
Message-ID: <4855BB2C.30902@canterbury.ac.nz>

Talin wrote:
> In some cases, the term is used to mean a 
> dictionary that remembers the order of insertions, and in other cases it 
> is used to mean a sorted dict,
> 
> I would be more in favor of the idea if we could come up with a less 
> ambiguous naming scheme.

Perhaps "indexed list" or maybe "keyed list" would be
a better term for the first variety.

And "sorted dict" seems like a good enough term for
the other one.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Mon Jun 16 03:08:55 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 16 Jun 2008 13:08:55 +1200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <op.ucr0100rhlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org> <op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan> <4854614B.9070903@canterbury.ac.nz>
	<op.ucr0100rhlrjc9@conan>
Message-ID: <4855BD27.1080605@canterbury.ac.nz>

Cesare Di Mauro wrote:

> OK, but nobody have questioned about removing 'from something import *' just to help noobs...

I'm not advocating removing it from the language, far
from it. I agree there are legitimate uses for it in
rare cases.

I just wish that people wouldn't use it in tutorials,
where it's the first thing the newcomer will see, and
get the impression that it's the normal thing to do.

-- 
Greg

From apt.shansen at gmail.com  Mon Jun 16 06:16:59 2008
From: apt.shansen at gmail.com (Stephen Hansen)
Date: Sun, 15 Jun 2008 21:16:59 -0700
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <aac2c7cb0806150945m49a7dc4ay87527d8024833ed9@mail.gmail.com>
References: <loom.20080614T231548-103@post.gmane.org>
	<485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org>
	<758DCE76527549B7879CC378065C3EC3@RaymondLaptop1>
	<B0C795FD-C1E8-4C93-A98F-BE24F1A16859@python.org>
	<aac2c7cb0806150945m49a7dc4ay87527d8024833ed9@mail.gmail.com>
Message-ID: <7a9c25c20806152116u5cbc7370te17b2c493f3e8a7f@mail.gmail.com>

> But my point is that we we need to focus on finding real use cases and
> exploring how best to solve them.  Otherwise we might as well throw in
> my OrderedSet[1] as-is, despite that it's got no comments and no
> ratings.  Even I don't seem to use it.
>

I'm mostly lurking on these threads, so as to be semi-prepared when new
versions come out.. in fifty years, since we *just* migrated from 2.3 to 2.4
on our product, so. :)

Anyways, we've had an OrderedDictionary sort of implementation in our
library for eons. The product is commercial and a mix between a desktop
application and a web one, with an application server and cross-platform
availability... so there's a slightly bizarre and wide range of uses that
we've found for putting ordered dictionaries to. If some various use-cases
and our reasoning helps, here it is. If not, ignore :)

- One is that the system is modular, with various parts able to be activated
or deactivated in configuration. The order of module load is important, as
we've never quite bothered to put in automatic dependency checking -- that's
just overboard for us. Further, the modules can't really be referred to each
other via "import" or in code, but instead need to go through a layer of
indirection through a name-- so-- the system maintains an ordered dict of
modules, a la sys.modules, with the order determining load when it goes
through to initialize itself.

- There's several more places with a similar pattern; a mapping between
"Component Name" and "Module" for generic systems. A processing host which
is meant to be able to load and run any kind of service or task.

- There's several places where a user is configuring a slightly complex set
of actions-- he gives these actions a name, which is shown to the user, and
then we have the configuration options itself we use if that is chosen. Its
just natural/easy to store that in an ordered dict after we pull it out of
the DB, as we want its order to be whatever the user chooses in their setup,
and the key->value association is clear.

- In a modular application with a generic outer interface(meaning the main
app can't even fathom what all it might be asked to load), there's things
like a "Windows" menu item that's easily stored internally as a mapping
between window names and the window object itself, so the menu can be
readily re-generated at will and the window found to switch to.

- In fact, we use a lot of OrderedDictionaries as a part of that UI to
data/configuration mapping, come to think of it. We know the order of
"fields" that someone can search on in the database in advance, and have
them written into the searchUI code as an ordered dict because it just works
nicely. The keys() become a drop-down list, the value a structure
identifying to the central server what field it is they're searching on.

- Fiiinally (sorta), we find passing ordered dictionaries to our Genshi web
templating layer very lovely for making easy-designable web templates for
the web client. We even let customers edit them sometimes!

Basically, after looking at all of these, my impressions of an "ordered
dictionary" for the various use cases we use are:

- The ordered dictionary is used almost exclusively in situations where we
are getting the order implicitly from some other source. Be it a SQL query
(with its own ORDER BY statements), a configuration option, the order of
lines in a file, an auto-sequenced table, or hard-coded data.... Thus, we've
always found "insertion order" to be important.

- Much to my surprise, we actually aren't ever using an ordered dictionary
in a situation where the value ends up being modified after the dictionary
is loaded.

- The only time we use dictionaries where we are updating them after the
fact and their order is -expected- to change is when we are using a *sorted*
dictionary.

- As such, I'd be quite surprised if I was updating the value of an ordered
dictionary and it were to change its order. Meaning:

  >>> d = odict()
  >>> d["hello"] = 1
  >>> d["there"] = 2
  >>> d["hello"] = 3
  >>> d.keys()
  ['hello', 'there']

And not: ['there', 'hello']

An ordered dictionary that does not simply preserve initial insertion order
strikes me as a *sorted* dictionary-- sorting on insertion time. I'd expect
a sorted dictionary to shift itself around as appropriate. I'd not expect an
ordered dictionary to change the order without some explicit action.

To me, "ordered dictionary" is in fact a *preordered* dictionary. The order
is defined before the data in, and the dictionary's job is to just preserve
it.

Anyways. No idea if that'll help the discussion, but a couple people kept
talking about use cases :)

--Stephen
P.S. Our implementation, incidentally, is essentially the same as one
mentioned above though its subclassing from UserDict (historical reasons for
now). We just track the keys in _keys, and if someone's setting something in
the dictionary not in keys, we append. It seemed the most natural/obvious
way to do it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080615/c6e89f7e/attachment-0001.htm>

From dbpokorny at gmail.com  Mon Jun 16 08:26:35 2008
From: dbpokorny at gmail.com (dbpokorny at gmail.com)
Date: Sun, 15 Jun 2008 23:26:35 -0700 (PDT)
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080614T231548-103@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>
Message-ID: <5edc7d74-5507-43d4-8f12-1410b0dc5c3b@z16g2000prn.googlegroups.com>

It is possible to get both ordered dict and sorted dict semantics in
the same type if you replace (key, value) pairs for dictionary entries
with (key,value,order) triples. The third component is a value that
indicates the place of the entry relative to the other entries. To get
an ordered dict, __setitem__ assigns 1+ max(order) to the new entry's
order. To get a sorted dict, order = key. To get a dict sorted by some
key function, order = keyfunc(key), etc...

David

From armin.ronacher at active-4.com  Mon Jun 16 11:05:47 2008
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Mon, 16 Jun 2008 09:05:47 +0000 (UTC)
Subject: [Python-Dev] Proposal: add odict to collections
References: <loom.20080614T231548-103@post.gmane.org>
	<loom.20080615T101201-207@post.gmane.org>
Message-ID: <loom.20080616T090353-853@post.gmane.org>

Armin Ronacher <armin.ronacher <at> active-4.com> writes:

> 
> There are far more responses for that topic than I imagined so I would love
> to write a PEP about that topic, incorporating the ideas/questions and
> suggestions discussed here.

There is now a PEP for the ordered dict:

  - PEP: http://www.python.org/dev/peps/pep-0372/
  - Thread on comp.lang.python:
    http://thread.gmane.org/gmane.comp.python.general/577894

Regards,
Armin


From ncoghlan at gmail.com  Mon Jun 16 11:29:13 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 16 Jun 2008 19:29:13 +1000
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <loom.20080616T090353-853@post.gmane.org>
References: <loom.20080614T231548-103@post.gmane.org>	<loom.20080615T101201-207@post.gmane.org>
	<loom.20080616T090353-853@post.gmane.org>
Message-ID: <48563269.3050609@gmail.com>

Armin Ronacher wrote:
> Armin Ronacher <armin.ronacher <at> active-4.com> writes:
> 
>> There are far more responses for that topic than I imagined so I would love
>> to write a PEP about that topic, incorporating the ideas/questions and
>> suggestions discussed here.
> 
> There is now a PEP for the ordered dict:
> 
>   - PEP: http://www.python.org/dev/peps/pep-0372/
>   - Thread on comp.lang.python:
>     http://thread.gmane.org/gmane.comp.python.general/577894

Another use case: in 2.6, RawConfigParser accepts a dict_type argument 
that allows an application to set the type of dictionary used 
internally. The motivation for this addition was expressly to allow 
users to provide an ordered dictionary [1].

Cheers,
Nick.

[1] http://bugs.python.org/issue1371075

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From rasky at develer.com  Mon Jun 16 18:18:23 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Mon, 16 Jun 2008 18:18:23 +0200
Subject: [Python-Dev] Troubles with Roundup
Message-ID: <4856924F.6080302@develer.com>

Hello,

I'm trying to login into the tracker but it gives me "invalid login" 
even after multiple password resets. I can't submit a proper bugreport 
because... I can't login :)

Who can I privately contact to avoid spamming this list?

Thanks!
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com

From stephen at xemacs.org  Mon Jun 16 20:45:19 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 17 Jun 2008 03:45:19 +0900
Subject: [Python-Dev] Proposal: add odict to collections
In-Reply-To: <5edc7d74-5507-43d4-8f12-1410b0dc5c3b@z16g2000prn.googlegroups.com>
References: <loom.20080614T231548-103@post.gmane.org>
	<5edc7d74-5507-43d4-8f12-1410b0dc5c3b@z16g2000prn.googlegroups.com>
Message-ID: <87bq21i5hc.fsf@uwakimon.sk.tsukuba.ac.jp>

dbpokorny at gmail.com writes:

 > It is possible to get both ordered dict and sorted dict semantics in
 > the same type if you replace (key, value) pairs for dictionary entries
 > with (key,value,order) triples.

Roundup uses something like this concept for its value choice menus.
I don't actually think it's used, though, except as a way to allow
admin user input that inserts an item at a position.

From brett at python.org  Mon Jun 16 21:07:38 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 16 Jun 2008 12:07:38 -0700
Subject: [Python-Dev] Troubles with Roundup
In-Reply-To: <4856924F.6080302@develer.com>
References: <4856924F.6080302@develer.com>
Message-ID: <bbaeab100806161207r42ed3adfx76441843b09c01a7@mail.gmail.com>

On Mon, Jun 16, 2008 at 9:18 AM, Giovanni Bajo <rasky at develer.com> wrote:
> Hello,
>
> I'm trying to login into the tracker but it gives me "invalid login" even
> after multiple password resets. I can't submit a proper bugreport because...
> I can't login :)
>
> Who can I privately contact to avoid spamming this list?
>

The tracker-discuss mailing list is the best place. Or file a bug on
the meta-tracker.

-Brett

From barry at python.org  Mon Jun 16 22:19:59 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 16 Jun 2008 16:19:59 -0400
Subject: [Python-Dev] Blow your mind
Message-ID: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

A colleague just forward this to me and it blew my fscking mind to  
smithereens.  It also brings back a lot of memories.  Enjoy!

- -Barry

http://www.vimeo.com/1093745

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFbK8XEjvBPtnXfVAQLPhwP/XQitqKSHHHUsYyrqN77G2FfIK0adyGNI
/NUY0DGJQUKrL0Wy+LQQzXUqY0B9a2uk6hxyyA7qWYzd57/l7JCGCgWdEJPhOyxb
Y8LW1N9z2SqdngaZHrQz1j5rRj5x2BFZpwt9/c/fqhbBhXIUrLnX/CZ342jQzneL
WrKzvqemGlY=
=JSCL
-----END PGP SIGNATURE-----

From simgidacav at gmail.com  Mon Jun 16 08:42:18 2008
From: simgidacav at gmail.com (Simgi Dacav)
Date: Mon, 16 Jun 2008 08:42:18 +0200
Subject: [Python-Dev] Let me introduce myself! :)
Message-ID: <20080616084218.9150832e.simgidacav@gmail.com>

Hi there!

I'm Giovanni Simoni, known as Dacav. I'm a student of Computer Science at the
University of Trento (Italy), a GNU/Linux user and, since I've tryied it
for the first time, I've appreciated the expressive power of Python, so I'd
like to delve into it.

I subscribed this mailing list after reading the introduction to development
(http://www.python.org/dev/intro/). I feel that it wolud be a good mean to
improve my skillness with Python, even if I'm probably not enough experienced
to bring a real contribute, for the moment.

In the meanwhile I inform you that you have one more lurker! :)

Thanks for your efforts, and regards
Giovanni

- -- 
Giovanni [Dacav] Simoni
Linux User #414144

Molti pensano che l'apprendere a fare le cose, diventando uno
scienziato, faccia sparire il mistero. Ci? che io sento ? che il
mistero motiva e nutre la scienza. La scoperta approfondisce il mistero.
                    -- Manfred Eigen

From musiccomposition at gmail.com  Mon Jun 16 22:24:10 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 16 Jun 2008 15:24:10 -0500
Subject: [Python-Dev] Blow your mind
In-Reply-To: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org>
References: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org>
Message-ID: <1afaf6160806161324y1c82cf5fu3b262a07d9a79ac7@mail.gmail.com>

On Mon, Jun 16, 2008 at 3:19 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> A colleague just forward this to me and it blew my fscking mind to
> smithereens.  It also brings back a lot of memories.  Enjoy!

Darn! I'm not on there yet. Anyway, it's nice to see Python
development is exploding like fireworks now!



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From mal at egenix.com  Mon Jun 16 22:27:40 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 16 Jun 2008 22:27:40 +0200
Subject: [Python-Dev] xmlrpclib.{True,
 False} (was Re:  Assignment to None)
In-Reply-To: <g33a2m$46q$1@ger.gmane.org>
References: <d2155e360806081924x77efc81fs64a498c665150d5d@mail.gmail.com>	<200806091348.44361.steve@pearwood.info>	<484CE993.5050506@voidspace.org.uk>	<484D9D59.903@v.loewis.de>	<484EA327.2050704@vector-seven.com>	<48551508.5060500@vector-seven.com>	<48551A56.1020704@vector-seven.com>	<g338l9$be$1@ger.gmane.org>	<485529EF.3080209@vector-seven.com>
	<g33a2m$46q$1@ger.gmane.org>
Message-ID: <4856CCBC.5040003@egenix.com>

On 2008-06-15 16:47, Georg Brandl wrote:
> Thomas Lee schrieb:
>> Georg Brandl wrote:
>>> Remember that it must still be possible to write (in 2.6)
>>>
>>> True = 0
>>> assert not True
>>
>> Ah of course. Looks like I should just avoid optimizations of 
>> Name("True") and Name("False") all together. That's a shame!
> 
> We can of course decide to make assignment to True and False
> illegal in 2.7 :)

Raising a run-time exception would be fine, but not a SyntaxError at
compile time - this would effectively make it impossible to keep
code compatible to Python 2.1.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 16 2008)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-07-07: EuroPython 2008, Vilnius, Lithuania            20 days to go

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611

From brett at python.org  Tue Jun 17 00:11:43 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 16 Jun 2008 15:11:43 -0700
Subject: [Python-Dev] Blow your mind
In-Reply-To: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org>
References: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org>
Message-ID: <bbaeab100806161511k459e32c8k53e504e8e8224e5f@mail.gmail.com>

On Mon, Jun 16, 2008 at 1:19 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> A colleague just forward this to me and it blew my fscking mind to
> smithereens.  It also brings back a lot of memories.  Enjoy!
>

In case anyone cares to download the video, it's easy with Safari.
Just go to the page and under Window -> Activity there is a list of
all files downloaded for the page. Just double-click the multi-MB FLV
file and Safari will download the Flash video file directly.

-Brett

From curt at hagenlocher.org  Tue Jun 17 01:08:53 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Mon, 16 Jun 2008 16:08:53 -0700
Subject: [Python-Dev] Epoch and Platform
Message-ID: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>

The documentation for the time module says that "the epoch is the point
where the time starts. On January 1st of that year, at 0 hours, the ``time
since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the
epoch is, look at gmtime(0)."  This confirms that the epoch is
platform-specific.  As such, the only legal uses of the timestamp should be

1) comparing with another timestamp to determine elapsed time in seconds
2) passing to another standard Python library function which expects a
timestamp
3) as a source of randomness.

However, the following files in the standard library have hardcoded the
assumption that the Python epoch will always be the same as the Unix epoch:
In gzip.py, method GzipFile._write_gzip_header
In tarfile.py, method _Stream._init_write_gz
In uuid.py, function uuid1

Additionally, the following files in the standard library have hardcoded the
assumption that the Python epoch will cause timestamps to fall within the
range of a 32-bit unsigned integer value:
In imputil.py, function _compile
In py_compile.py, function compile

So there's some kind of a potential discrepancy here, albeit a minor one.
This discrepancy can be resolved in one of at least three ways:

1) The documentation for the time module is wrong, and the epoch for Python
(at least versions 2.x) should be the Unix epoch.
2) These library functions are slightly wrong and should be modified by
subtracing an "epoch offset" before doing other calculations. This offset
can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) -
time.timezone".
3) These library files should be considered part of the platform-specific
implementation, and an alternate platform should provide its own version of
these files if necessary.

Any thoughts on this?

>From the perspective of implementing IronPython, I'd prefer that the answer
is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as
possible, while respecting CLR-specific norms for functionality which is
left up to individual implementations.

--
Curt Hagenlocher
curt at hagenlocher.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080616/2e763e34/attachment-0001.htm>

From guido at python.org  Tue Jun 17 01:38:46 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Jun 2008 16:38:46 -0700
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
Message-ID: <ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>

ISTR that we force the epoch to be 1970 on all major platforms -- or
perhaps it happens to be 1970 even on Windows when using MS's C
runtime.

On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
> The documentation for the time module says that "the epoch is the point
> where the time starts. On January 1st of that year, at 0 hours, the ``time
> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the
> epoch is, look at gmtime(0)."  This confirms that the epoch is
> platform-specific.  As such, the only legal uses of the timestamp should be
>
> 1) comparing with another timestamp to determine elapsed time in seconds
> 2) passing to another standard Python library function which expects a
> timestamp
> 3) as a source of randomness.
>
> However, the following files in the standard library have hardcoded the
> assumption that the Python epoch will always be the same as the Unix epoch:
> In gzip.py, method GzipFile._write_gzip_header
> In tarfile.py, method _Stream._init_write_gz
> In uuid.py, function uuid1
>
> Additionally, the following files in the standard library have hardcoded the
> assumption that the Python epoch will cause timestamps to fall within the
> range of a 32-bit unsigned integer value:
> In imputil.py, function _compile
> In py_compile.py, function compile
>
> So there's some kind of a potential discrepancy here, albeit a minor one.
> This discrepancy can be resolved in one of at least three ways:
>
> 1) The documentation for the time module is wrong, and the epoch for Python
> (at least versions 2.x) should be the Unix epoch.
> 2) These library functions are slightly wrong and should be modified by
> subtracing an "epoch offset" before doing other calculations. This offset
> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) -
> time.timezone".
> 3) These library files should be considered part of the platform-specific
> implementation, and an alternate platform should provide its own version of
> these files if necessary.
>
> Any thoughts on this?
>
> From the perspective of implementing IronPython, I'd prefer that the answer
> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as
> possible, while respecting CLR-specific norms for functionality which is
> left up to individual implementations.
>
> --
> Curt Hagenlocher
> curt at hagenlocher.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From curt at hagenlocher.org  Tue Jun 17 01:51:19 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Mon, 16 Jun 2008 16:51:19 -0700
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
Message-ID: <d2155e360806161651v7792f37cicbeb93abbb13dcf3@mail.gmail.com>

>From what I remember, the Microsoft CLIB has been consistent with the
Unix epoch since the bad old days of 16-bit.  I believe that the
Macintosh CLIB used to be based on January 1, 1904 -- but it's been a
long time since I did any Mac development and I'm sure it would have
changed with OS X.

On Mon, Jun 16, 2008 at 4:38 PM, Guido van Rossum <guido at python.org> wrote:
> ISTR that we force the epoch to be 1970 on all major platforms -- or
> perhaps it happens to be 1970 even on Windows when using MS's C
> runtime.
>
> On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>> The documentation for the time module says that "the epoch is the point
>> where the time starts. On January 1st of that year, at 0 hours, the ``time
>> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the
>> epoch is, look at gmtime(0)."  This confirms that the epoch is
>> platform-specific.  As such, the only legal uses of the timestamp should be
>>
>> 1) comparing with another timestamp to determine elapsed time in seconds
>> 2) passing to another standard Python library function which expects a
>> timestamp
>> 3) as a source of randomness.
>>
>> However, the following files in the standard library have hardcoded the
>> assumption that the Python epoch will always be the same as the Unix epoch:
>> In gzip.py, method GzipFile._write_gzip_header
>> In tarfile.py, method _Stream._init_write_gz
>> In uuid.py, function uuid1
>>
>> Additionally, the following files in the standard library have hardcoded the
>> assumption that the Python epoch will cause timestamps to fall within the
>> range of a 32-bit unsigned integer value:
>> In imputil.py, function _compile
>> In py_compile.py, function compile
>>
>> So there's some kind of a potential discrepancy here, albeit a minor one.
>> This discrepancy can be resolved in one of at least three ways:
>>
>> 1) The documentation for the time module is wrong, and the epoch for Python
>> (at least versions 2.x) should be the Unix epoch.
>> 2) These library functions are slightly wrong and should be modified by
>> subtracing an "epoch offset" before doing other calculations. This offset
>> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) -
>> time.timezone".
>> 3) These library files should be considered part of the platform-specific
>> implementation, and an alternate platform should provide its own version of
>> these files if necessary.
>>
>> Any thoughts on this?
>>
>> From the perspective of implementing IronPython, I'd prefer that the answer
>> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as
>> possible, while respecting CLR-specific norms for functionality which is
>> left up to individual implementations.
>>
>> --
>> Curt Hagenlocher
>> curt at hagenlocher.org
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

From bonelake at gmail.com  Tue Jun 17 16:32:35 2008
From: bonelake at gmail.com (Brad Miller)
Date: Tue, 17 Jun 2008 09:32:35 -0500
Subject: [Python-Dev] Error building 3.0 on OS-X
Message-ID: <e45086040806170732v1708c98fs5604b0cf4b15338a@mail.gmail.com>

I have been updating the 3.0 source using bzr and after my latest bzr
pull I get an error on make install.  I'm running  10.5.3.

./configure --enable-framework
make

make install

...

running install_egg_info
Writing /Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/lib-dynload/Python-3.0a5_-py3.0.egg-info
ln -fs "../../../Python"
"/Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/config/libpython3.0.a"
cd Mac && make installmacsubtree DESTDIR=""
DYLD_FRAMEWORK_PATH=/Users/bmiller/src/python/3.0: ../python.exe
./scripts/cachersrc.py -v
/Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/plat-mac
/Library/Frameworks/Python.framework/Versions/3.0/Mac/Tools
../python.exe: can't open file './scripts/cachersrc.py': [Errno 2] No
such file or directory
make[1]: *** [installmacsubtree] Error 2
make: *** [frameworkinstallmaclib] Error 2


Any hints?

Thanks,

Brad

From p.f.moore at gmail.com  Tue Jun 17 17:54:34 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 17 Jun 2008 16:54:34 +0100
Subject: [Python-Dev] Blow your mind
In-Reply-To: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org>
References: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org>
Message-ID: <79990c6b0806170854n55cbb72bxe788d11c644c8a10@mail.gmail.com>

2008/6/16 Barry Warsaw <barry at python.org>:
> A colleague just forward this to me and it blew my fscking mind to
> smithereens.  It also brings back a lot of memories.  Enjoy!

Wow!

One thing that surprised me was that I never saw Tim appear...
Paul.

From curt at hagenlocher.org  Tue Jun 17 18:26:55 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Tue, 17 Jun 2008 09:26:55 -0700
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
Message-ID: <d2155e360806170926p414a6dc8nb414eafbd96986b8@mail.gmail.com>

Any chance of an Official Pronouncement on this topic?  It would help
us greatly -- even if only to figure out who'll be paying for the next
round of beer.

On Mon, Jun 16, 2008 at 4:38 PM, Guido van Rossum <guido at python.org> wrote:
> ISTR that we force the epoch to be 1970 on all major platforms -- or
> perhaps it happens to be 1970 even on Windows when using MS's C
> runtime.
>
> On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>> The documentation for the time module says that "the epoch is the point
>> where the time starts. On January 1st of that year, at 0 hours, the ``time
>> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the
>> epoch is, look at gmtime(0)."  This confirms that the epoch is
>> platform-specific.  As such, the only legal uses of the timestamp should be
>>
>> 1) comparing with another timestamp to determine elapsed time in seconds
>> 2) passing to another standard Python library function which expects a
>> timestamp
>> 3) as a source of randomness.
>>
>> However, the following files in the standard library have hardcoded the
>> assumption that the Python epoch will always be the same as the Unix epoch:
>> In gzip.py, method GzipFile._write_gzip_header
>> In tarfile.py, method _Stream._init_write_gz
>> In uuid.py, function uuid1
>>
>> Additionally, the following files in the standard library have hardcoded the
>> assumption that the Python epoch will cause timestamps to fall within the
>> range of a 32-bit unsigned integer value:
>> In imputil.py, function _compile
>> In py_compile.py, function compile
>>
>> So there's some kind of a potential discrepancy here, albeit a minor one.
>> This discrepancy can be resolved in one of at least three ways:
>>
>> 1) The documentation for the time module is wrong, and the epoch for Python
>> (at least versions 2.x) should be the Unix epoch.
>> 2) These library functions are slightly wrong and should be modified by
>> subtracing an "epoch offset" before doing other calculations. This offset
>> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) -
>> time.timezone".
>> 3) These library files should be considered part of the platform-specific
>> implementation, and an alternate platform should provide its own version of
>> these files if necessary.
>>
>> Any thoughts on this?
>>
>> From the perspective of implementing IronPython, I'd prefer that the answer
>> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as
>> possible, while respecting CLR-specific norms for functionality which is
>> left up to individual implementations.
>>
>> --
>> Curt Hagenlocher
>> curt at hagenlocher.org
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

From guido at python.org  Tue Jun 17 19:01:09 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Jun 2008 10:01:09 -0700
Subject: [Python-Dev] Blow your mind
In-Reply-To: <79990c6b0806170854n55cbb72bxe788d11c644c8a10@mail.gmail.com>
References: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org>
	<79990c6b0806170854n55cbb72bxe788d11c644c8a10@mail.gmail.com>
Message-ID: <ca471dc20806171001y6bc881fhaf9f9cbe8ab9bf2c@mail.gmail.com>

On Tue, Jun 17, 2008 at 8:54 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2008/6/16 Barry Warsaw <barry at python.org>:
>> A colleague just forward this to me and it blew my fscking mind to
>> smithereens.  It also brings back a lot of memories.  Enjoy!
>
> Wow!
>
> One thing that surprised me was that I never saw Tim appear...
> Paul.

He wasn't able to contribute while we were at CNRI (I did the checkins
for him) and once we were at SourceForge there were so many people
that it was hard to find anyone in particular.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Jun 17 19:03:42 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Jun 2008 10:03:42 -0700
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <d2155e360806170926p414a6dc8nb414eafbd96986b8@mail.gmail.com>
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
	<d2155e360806170926p414a6dc8nb414eafbd96986b8@mail.gmail.com>
Message-ID: <ca471dc20806171003v7cbf1638t44fd0f714092f54@mail.gmail.com>

Can you explain why you are so anxious to get this resolved (apart
from the beer :-) ?

On Tue, Jun 17, 2008 at 9:26 AM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
> Any chance of an Official Pronouncement on this topic?  It would help
> us greatly -- even if only to figure out who'll be paying for the next
> round of beer.
>
> On Mon, Jun 16, 2008 at 4:38 PM, Guido van Rossum <guido at python.org> wrote:
>> ISTR that we force the epoch to be 1970 on all major platforms -- or
>> perhaps it happens to be 1970 even on Windows when using MS's C
>> runtime.
>>
>> On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>>> The documentation for the time module says that "the epoch is the point
>>> where the time starts. On January 1st of that year, at 0 hours, the ``time
>>> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the
>>> epoch is, look at gmtime(0)."  This confirms that the epoch is
>>> platform-specific.  As such, the only legal uses of the timestamp should be
>>>
>>> 1) comparing with another timestamp to determine elapsed time in seconds
>>> 2) passing to another standard Python library function which expects a
>>> timestamp
>>> 3) as a source of randomness.
>>>
>>> However, the following files in the standard library have hardcoded the
>>> assumption that the Python epoch will always be the same as the Unix epoch:
>>> In gzip.py, method GzipFile._write_gzip_header
>>> In tarfile.py, method _Stream._init_write_gz
>>> In uuid.py, function uuid1
>>>
>>> Additionally, the following files in the standard library have hardcoded the
>>> assumption that the Python epoch will cause timestamps to fall within the
>>> range of a 32-bit unsigned integer value:
>>> In imputil.py, function _compile
>>> In py_compile.py, function compile
>>>
>>> So there's some kind of a potential discrepancy here, albeit a minor one.
>>> This discrepancy can be resolved in one of at least three ways:
>>>
>>> 1) The documentation for the time module is wrong, and the epoch for Python
>>> (at least versions 2.x) should be the Unix epoch.
>>> 2) These library functions are slightly wrong and should be modified by
>>> subtracing an "epoch offset" before doing other calculations. This offset
>>> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) -
>>> time.timezone".
>>> 3) These library files should be considered part of the platform-specific
>>> implementation, and an alternate platform should provide its own version of
>>> these files if necessary.
>>>
>>> Any thoughts on this?
>>>
>>> From the perspective of implementing IronPython, I'd prefer that the answer
>>> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as
>>> possible, while respecting CLR-specific norms for functionality which is
>>> left up to individual implementations.
>>>
>>> --
>>> Curt Hagenlocher
>>> curt at hagenlocher.org
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From agthorr at barsoom.org  Tue Jun 17 19:24:57 2008
From: agthorr at barsoom.org (Daniel Stutzbach)
Date: Tue, 17 Jun 2008 12:24:57 -0500
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
Message-ID: <eae285400806171024mb4910c4te56b14b32c6f68ef@mail.gmail.com>

On Mon, Jun 16, 2008 at 6:38 PM, Guido van Rossum <guido at python.org> wrote:
> ISTR that we force the epoch to be 1970 on all major platforms -- or
> perhaps it happens to be 1970 even on Windows when using MS's C
> runtime.

I can confirm that Python under Windows does indeed use the Unix epoch:

Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.gmtime(0)
(1970, 1, 1, 0, 0, 0, 3, 1, 0)

-- 
Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC

From list8a.forest at tibit.com  Tue Jun 17 19:21:39 2008
From: list8a.forest at tibit.com (Forest)
Date: Tue, 17 Jun 2008 10:21:39 -0700 (PDT)
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <mailman.49.1213696814.25164.python-dev@python.org>
References: <mailman.49.1213696814.25164.python-dev@python.org>
Message-ID: <6289.75.55.199.5.1213723299.squirrel@webmail.sonic.net>

ISTR using a Microsoft C compiler in the early/mid 1990s whose runtime
library used an unusual epoch.  I don't recall any others straying from
the Unix way, but then again, I haven't been looking for such quirks.

Guido wrote:
>
> ISTR that we force the epoch to be 1970 on all major platforms -- or
> perhaps it happens to be 1970 even on Windows when using MS's C
> runtime.
>
> On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher <curt at hagenlocher.org>
> wrote:
>> The documentation for the time module says that "the epoch is the point
>> where the time starts. On January 1st of that year, at 0 hours, the
>> ``time
>> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what
>> the
>> epoch is, look at gmtime(0)."  This confirms that the epoch is
>> platform-specific.  As such, the only legal uses of the timestamp should
>> be
>>
>> 1) comparing with another timestamp to determine elapsed time in seconds
>> 2) passing to another standard Python library function which expects a
>> timestamp
>> 3) as a source of randomness.
>>
>> However, the following files in the standard library have hardcoded the
>> assumption that the Python epoch will always be the same as the Unix
>> epoch:
>> In gzip.py, method GzipFile._write_gzip_header
>> In tarfile.py, method _Stream._init_write_gz
>> In uuid.py, function uuid1
>>
>> Additionally, the following files in the standard library have hardcoded
>> the
>> assumption that the Python epoch will cause timestamps to fall within
>> the
>> range of a 32-bit unsigned integer value:
>> In imputil.py, function _compile
>> In py_compile.py, function compile
>>
>> So there's some kind of a potential discrepancy here, albeit a minor
>> one.
>> This discrepancy can be resolved in one of at least three ways:
>>
>> 1) The documentation for the time module is wrong, and the epoch for
>> Python
>> (at least versions 2.x) should be the Unix epoch.
>> 2) These library functions are slightly wrong and should be modified by
>> subtracing an "epoch offset" before doing other calculations. This
>> offset
>> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) -
>> time.timezone".
>> 3) These library files should be considered part of the
>> platform-specific
>> implementation, and an alternate platform should provide its own version
>> of
>> these files if necessary.
>>
>> Any thoughts on this?
>>
>> From the perspective of implementing IronPython, I'd prefer that the
>> answer
>> is 1 or 2 -- but mainly I just want to be as compatible with "the spec"
>> as
>> possible, while respecting CLR-specific norms for functionality which is
>> left up to individual implementations.
>>
>> --
>> Curt Hagenlocher
>> curt at hagenlocher.org



From curt at hagenlocher.org  Tue Jun 17 19:40:23 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Tue, 17 Jun 2008 10:40:23 -0700
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <ca471dc20806171003v7cbf1638t44fd0f714092f54@mail.gmail.com>
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
	<d2155e360806170926p414a6dc8nb414eafbd96986b8@mail.gmail.com>
	<ca471dc20806171003v7cbf1638t44fd0f714092f54@mail.gmail.com>
Message-ID: <d2155e360806171040p2e865f8dxc0b8982c29c75221@mail.gmail.com>

I don't *feel* anxious, but my doctor *has* been trying to persuade me
to switch to decaf...

There's no real urgency.  The reason this came up is because I just
implemented zlib, which automatically enabled the gzip unit tests.
The gzip tests are failing because the current timestamp can't be
written as a 32-bit value.  In order to checkin my changes, I can't
have any failing tests -- so my choices are

1) change the IronPython epoch so that the timestamp works for gzip and tarlib
2) change gzip and tarlib to work with a "less standard" epoch, or
3) disable the failing test(s)

...and I'd rather not resort to #3, if possible.

On Tue, Jun 17, 2008 at 10:03 AM, Guido van Rossum <guido at python.org> wrote:
> Can you explain why you are so anxious to get this resolved (apart
> from the beer :-) ?
>
> On Tue, Jun 17, 2008 at 9:26 AM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>> Any chance of an Official Pronouncement on this topic?  It would help
>> us greatly -- even if only to figure out who'll be paying for the next
>> round of beer.
>>
>> On Mon, Jun 16, 2008 at 4:38 PM, Guido van Rossum <guido at python.org> wrote:
>>> ISTR that we force the epoch to be 1970 on all major platforms -- or
>>> perhaps it happens to be 1970 even on Windows when using MS's C
>>> runtime.
>>>
>>> On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>>>> The documentation for the time module says that "the epoch is the point
>>>> where the time starts. On January 1st of that year, at 0 hours, the ``time
>>>> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the
>>>> epoch is, look at gmtime(0)."  This confirms that the epoch is
>>>> platform-specific.  As such, the only legal uses of the timestamp should be
>>>>
>>>> 1) comparing with another timestamp to determine elapsed time in seconds
>>>> 2) passing to another standard Python library function which expects a
>>>> timestamp
>>>> 3) as a source of randomness.
>>>>
>>>> However, the following files in the standard library have hardcoded the
>>>> assumption that the Python epoch will always be the same as the Unix epoch:
>>>> In gzip.py, method GzipFile._write_gzip_header
>>>> In tarfile.py, method _Stream._init_write_gz
>>>> In uuid.py, function uuid1
>>>>
>>>> Additionally, the following files in the standard library have hardcoded the
>>>> assumption that the Python epoch will cause timestamps to fall within the
>>>> range of a 32-bit unsigned integer value:
>>>> In imputil.py, function _compile
>>>> In py_compile.py, function compile
>>>>
>>>> So there's some kind of a potential discrepancy here, albeit a minor one.
>>>> This discrepancy can be resolved in one of at least three ways:
>>>>
>>>> 1) The documentation for the time module is wrong, and the epoch for Python
>>>> (at least versions 2.x) should be the Unix epoch.
>>>> 2) These library functions are slightly wrong and should be modified by
>>>> subtracing an "epoch offset" before doing other calculations. This offset
>>>> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) -
>>>> time.timezone".
>>>> 3) These library files should be considered part of the platform-specific
>>>> implementation, and an alternate platform should provide its own version of
>>>> these files if necessary.
>>>>
>>>> Any thoughts on this?
>>>>
>>>> From the perspective of implementing IronPython, I'd prefer that the answer
>>>> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as
>>>> possible, while respecting CLR-specific norms for functionality which is
>>>> left up to individual implementations.
>>>>
>>>> --
>>>> Curt Hagenlocher
>>>> curt at hagenlocher.org
>>>> _______________________________________________
>>>> Python-Dev mailing list
>>>> Python-Dev at python.org
>>>> http://mail.python.org/mailman/listinfo/python-dev
>>>> Unsubscribe:
>>>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>>>
>>
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

From millbr02 at luther.edu  Tue Jun 17 19:49:20 2008
From: millbr02 at luther.edu (Brad Miller)
Date: Tue, 17 Jun 2008 12:49:20 -0500
Subject: [Python-Dev] Error building python3.0 under OS X
Message-ID: <e45086040806171049r123a2a46u591033482fb48b5a@mail.gmail.com>

I have been updating the 3.0 source using bzr and after my latest bzr
pull I get an error on make install.  I'm running  10.5.3.

./configure --enable-framework
make

make install

...

running install_egg_info
Writing /Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/lib-dynload/Python-3.0a5_-py3.0.egg-info
ln -fs "../../../Python"
"/Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/config/libpython3.0.a"
cd Mac && make installmacsubtree DESTDIR=""
DYLD_FRAMEWORK_PATH=/Users/bmiller/src/python/3.0: ../python.exe
./scripts/cachersrc.py -v
/Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/plat-mac
/Library/Frameworks/Python.framework/Versions/3.0/Mac/Tools
../python.exe: can't open file './scripts/cachersrc.py': [Errno 2] No
such file or directory
make[1]: *** [installmacsubtree] Error 2
make: *** [frameworkinstallmaclib] Error 2


Any hints?

Thanks,

Brad

From guido at python.org  Tue Jun 17 19:56:13 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Jun 2008 10:56:13 -0700
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <d2155e360806171040p2e865f8dxc0b8982c29c75221@mail.gmail.com>
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
	<d2155e360806170926p414a6dc8nb414eafbd96986b8@mail.gmail.com>
	<ca471dc20806171003v7cbf1638t44fd0f714092f54@mail.gmail.com>
	<d2155e360806171040p2e865f8dxc0b8982c29c75221@mail.gmail.com>
Message-ID: <ca471dc20806171056n3348791al72c28500312553d1@mail.gmail.com>

On Tue, Jun 17, 2008 at 10:40 AM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
> I don't *feel* anxious, but my doctor *has* been trying to persuade me
> to switch to decaf...
>
> There's no real urgency.  The reason this came up is because I just
> implemented zlib, which automatically enabled the gzip unit tests.
> The gzip tests are failing because the current timestamp can't be
> written as a 32-bit value.

Why is that? Is it because your epoch is different? If so, I would
much prefer the epoch to be 1970. (Maybe this is the resolution you're
seeking?)

> In order to checkin my changes, I can't
> have any failing tests -- so my choices are
>
> 1) change the IronPython epoch so that the timestamp works for gzip and tarlib
> 2) change gzip and tarlib to work with a "less standard" epoch, or
> 3) disable the failing test(s)
>
> ...and I'd rather not resort to #3, if possible.
>
> On Tue, Jun 17, 2008 at 10:03 AM, Guido van Rossum <guido at python.org> wrote:
>> Can you explain why you are so anxious to get this resolved (apart
>> from the beer :-) ?
>>
>> On Tue, Jun 17, 2008 at 9:26 AM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>>> Any chance of an Official Pronouncement on this topic?  It would help
>>> us greatly -- even if only to figure out who'll be paying for the next
>>> round of beer.
>>>
>>> On Mon, Jun 16, 2008 at 4:38 PM, Guido van Rossum <guido at python.org> wrote:
>>>> ISTR that we force the epoch to be 1970 on all major platforms -- or
>>>> perhaps it happens to be 1970 even on Windows when using MS's C
>>>> runtime.
>>>>
>>>> On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>>>>> The documentation for the time module says that "the epoch is the point
>>>>> where the time starts. On January 1st of that year, at 0 hours, the ``time
>>>>> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the
>>>>> epoch is, look at gmtime(0)."  This confirms that the epoch is
>>>>> platform-specific.  As such, the only legal uses of the timestamp should be
>>>>>
>>>>> 1) comparing with another timestamp to determine elapsed time in seconds
>>>>> 2) passing to another standard Python library function which expects a
>>>>> timestamp
>>>>> 3) as a source of randomness.
>>>>>
>>>>> However, the following files in the standard library have hardcoded the
>>>>> assumption that the Python epoch will always be the same as the Unix epoch:
>>>>> In gzip.py, method GzipFile._write_gzip_header
>>>>> In tarfile.py, method _Stream._init_write_gz
>>>>> In uuid.py, function uuid1
>>>>>
>>>>> Additionally, the following files in the standard library have hardcoded the
>>>>> assumption that the Python epoch will cause timestamps to fall within the
>>>>> range of a 32-bit unsigned integer value:
>>>>> In imputil.py, function _compile
>>>>> In py_compile.py, function compile
>>>>>
>>>>> So there's some kind of a potential discrepancy here, albeit a minor one.
>>>>> This discrepancy can be resolved in one of at least three ways:
>>>>>
>>>>> 1) The documentation for the time module is wrong, and the epoch for Python
>>>>> (at least versions 2.x) should be the Unix epoch.
>>>>> 2) These library functions are slightly wrong and should be modified by
>>>>> subtracing an "epoch offset" before doing other calculations. This offset
>>>>> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) -
>>>>> time.timezone".
>>>>> 3) These library files should be considered part of the platform-specific
>>>>> implementation, and an alternate platform should provide its own version of
>>>>> these files if necessary.
>>>>>
>>>>> Any thoughts on this?
>>>>>
>>>>> From the perspective of implementing IronPython, I'd prefer that the answer
>>>>> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as
>>>>> possible, while respecting CLR-specific norms for functionality which is
>>>>> left up to individual implementations.
>>>>>
>>>>> --
>>>>> Curt Hagenlocher
>>>>> curt at hagenlocher.org
>>>>> _______________________________________________
>>>>> Python-Dev mailing list
>>>>> Python-Dev at python.org
>>>>> http://mail.python.org/mailman/listinfo/python-dev
>>>>> Unsubscribe:
>>>>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>>>>
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From curt at hagenlocher.org  Tue Jun 17 20:09:13 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Tue, 17 Jun 2008 11:09:13 -0700
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <ca471dc20806171056n3348791al72c28500312553d1@mail.gmail.com>
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
	<d2155e360806170926p414a6dc8nb414eafbd96986b8@mail.gmail.com>
	<ca471dc20806171003v7cbf1638t44fd0f714092f54@mail.gmail.com>
	<d2155e360806171040p2e865f8dxc0b8982c29c75221@mail.gmail.com>
	<ca471dc20806171056n3348791al72c28500312553d1@mail.gmail.com>
Message-ID: <d2155e360806171109m38c905d8o68ed57dad0a10bf5@mail.gmail.com>

On Tue, Jun 17, 2008 at 10:56 AM, Guido van Rossum <guido at python.org> wrote:
> On Tue, Jun 17, 2008 at 10:40 AM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>>
>> There's no real urgency.  The reason this came up is because I just
>> implemented zlib, which automatically enabled the gzip unit tests.
>> The gzip tests are failing because the current timestamp can't be
>> written as a 32-bit value.
>
> Why is that? Is it because your epoch is different? If so, I would
> much prefer the epoch to be 1970. (Maybe this is the resolution you're
> seeking?)

Yes!  Except that I was hoping for something a little stronger, like
"the epoch must be 1970" or "the epoch can be anything you want, but
you're utterly retarded if you pick something other than 1970".  But
I'll definitely settle for "much prefer". :)

Thanks,
-Curt

--
Curt Hagenlocher
curt at hagenlocher.org

From guido at python.org  Tue Jun 17 20:34:45 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Jun 2008 11:34:45 -0700
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <d2155e360806171109m38c905d8o68ed57dad0a10bf5@mail.gmail.com>
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
	<d2155e360806170926p414a6dc8nb414eafbd96986b8@mail.gmail.com>
	<ca471dc20806171003v7cbf1638t44fd0f714092f54@mail.gmail.com>
	<d2155e360806171040p2e865f8dxc0b8982c29c75221@mail.gmail.com>
	<ca471dc20806171056n3348791al72c28500312553d1@mail.gmail.com>
	<d2155e360806171109m38c905d8o68ed57dad0a10bf5@mail.gmail.com>
Message-ID: <ca471dc20806171134p7c2433ch62cf5fa1df852b3a@mail.gmail.com>

On Tue, Jun 17, 2008 at 11:09 AM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
> On Tue, Jun 17, 2008 at 10:56 AM, Guido van Rossum <guido at python.org> wrote:
>> On Tue, Jun 17, 2008 at 10:40 AM, Curt Hagenlocher <curt at hagenlocher.org> wrote:
>>>
>>> There's no real urgency.  The reason this came up is because I just
>>> implemented zlib, which automatically enabled the gzip unit tests.
>>> The gzip tests are failing because the current timestamp can't be
>>> written as a 32-bit value.
>>
>> Why is that? Is it because your epoch is different? If so, I would
>> much prefer the epoch to be 1970. (Maybe this is the resolution you're
>> seeking?)
>
> Yes!  Except that I was hoping for something a little stronger, like
> "the epoch must be 1970" or "the epoch can be anything you want, but
> you're utterly retarded if you pick something other than 1970".  But
> I'll definitely settle for "much prefer". :)

I could go with the "utterly retarded" wording. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From amauryfa at gmail.com  Tue Jun 17 21:44:45 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 17 Jun 2008 21:44:45 +0200
Subject: [Python-Dev] Merging changes before beta
Message-ID: <e27efe130806171244t4b4689ect9ee8fa70014c282f@mail.gmail.com>

Hello,

I am in the process of merging changes from trunk to the py3k branch.
(If somebody else is already doing it, please tell me quickly!)

IMO it would be better to have the beta versions in sync;
the 2.6->3.0 migration should not be a regression!

I will integrate the changes that are either simple, or that I can understand,
or have unit tests.

The changes I will skip are (for the moment):

- r64114 (gregory.p.smith): a bunch of checks for integer overflows,
contributed by Google.
This change has many conflicts, I will do it later.

- r63828 (mark.hammond): Fix bdist_wininst --user-access-control for win2k
For this it is necessary to recompile the various wininst-*.exe,
and not all my compilers have the necessary headers or libraries.

- r63955 (ronald.oussoren): MacOS X: Enable 4-way universal builds
I may try the merge, but without any way to test the result.
OTOH, I could go and trust the buildbots.

- 64062,64068-64069,64080 (josiah.carlson): Asynchat
I may give a try, but it will be tricky. And I don't even know how
asynchat works
(I just remember it was hard to get it right for python 3.0)

- 63207,63210,63218,63403,63469,63537,63597-63598,63601,63604,63617-63618,
63714,63718,63742,63745,63799 (jesus.cea, gregory.p.smith): bsddb module
This is a important list of changes, and may require heavy adaptations
to the code.
issue2887 has been filed for this task.

Do you think these tasks should be considered as Release Blockers?
Beta1 is close...

-- 
Amaury Forgeot d'Arc

From barry at python.org  Tue Jun 17 21:59:46 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Jun 2008 15:59:46 -0400
Subject: [Python-Dev] [Python-3000] Merging changes before beta
In-Reply-To: <e27efe130806171244t4b4689ect9ee8fa70014c282f@mail.gmail.com>
References: <e27efe130806171244t4b4689ect9ee8fa70014c282f@mail.gmail.com>
Message-ID: <C0DD0C67-D3D1-4404-A6F0-7BD0B94D8C96@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 17, 2008, at 3:44 PM, Amaury Forgeot d'Arc wrote:

> I am in the process of merging changes from trunk to the py3k branch.
> (If somebody else is already doing it, please tell me quickly!)

BTW, I will be hanging out on #python-dev @ freenode from now until  
tomorrow night, so if you have questions feel free to ping me.  My  
userid is 'barry' (so creative <wink>) and I keep generally UTC-0400  
time.

> The changes I will skip are (for the moment):
>
> - r64114 (gregory.p.smith): a bunch of checks for integer overflows,
> contributed by Google.
> This change has many conflicts, I will do it later.

Not a blocker.

> - r63828 (mark.hammond): Fix bdist_wininst --user-access-control for  
> win2k
> For this it is necessary to recompile the various wininst-*.exe,
> and not all my compilers have the necessary headers or libraries.

Not a blocker.

> - r63955 (ronald.oussoren): MacOS X: Enable 4-way universal builds
> I may try the merge, but without any way to test the result.
> OTOH, I could go and trust the buildbots.

Only if you're prepared to back it out if they go red.  Not a blocker.

> - 64062,64068-64069,64080 (josiah.carlson): Asynchat
> I may give a try, but it will be tricky. And I don't even know how
> asynchat works
> (I just remember it was hard to get it right for python 3.0)

Hmm.  I don't know the details of these.

> -  
> 63207,63210,63218,63403,63469,63537,63597 
> -63598,63601,63604,63617-63618,
> 63714,63718,63742,63745,63799 (jesus.cea, gregory.p.smith): bsddb  
> module
> This is a important list of changes, and may require heavy adaptations
> to the code.
> issue2887 has been filed for this task.

Not a blocker.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFgXs3EjvBPtnXfVAQJdiQP8CIahHlmiwep5ljm0gakwYzEI7LJ1e7+0
5HXNqClrhntIP+wjR8BPR9oNTp04D0Y35v8ZZcZDh0t/2LtboP6fySO4eVmYcoma
p5/Ld6xPoaSRfy15jVb9xKx4AsegKNZlOdJWpZs9+jkYK5x5mcflns0Nn4KoRKwS
r3ExyMj5YIQ=
=liS1
-----END PGP SIGNATURE-----

From tnelson at onresolve.com  Wed Jun 18 07:48:40 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 18 Jun 2008 06:48:40 +0100
Subject: [Python-Dev] Progress on switching Windows build to Berkeley DB
	4.7.25...
Message-ID: <6167796BFEB5D0438720AC212E89A6B00787275B@exchange.onresolve.com>

Hi all,

Jesus, apologies that this has taken so long for me to get back to, I've been completely and utterly swamped with client work the past few weeks.  However, thanks to a couple of hours spare at Detroit airport yesterday, I was finally able to make some progress on updating the Windows Berkeley DB build to 4.7.25.  I've checked in the work I've done so far to branches/tnelson-trunk-bsddb-47-upgrade.  One thing I wanted to double check with you is the following change:

Modified: python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py
==============================================================================
--- python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py   (original)
+++ python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py   Wed Jun 18 06:13:44 2008
@@ -94,7 +94,7 @@
         # The timeout is necessary in BDB 4.5, since DB_EVENT_REP_STARTUPDONE
         # is not generated if the master has no new transactions.
         # This is solved in BDB 4.6 (#15542).
-        timeout = time.time()+2
+        timeout = time.time()+10
         while (time.time()<timeout) and not (self.confirmed_master and self.client_startupdone) :
             time.sleep(0.02)
         if db.version() >= (4,6) :

Basically, when using +2, the test failed every so often when running the entire test_bsddb3 suite.  I picked 10 arbitrarily; it improves things, but it's still not 100%, I still encounter the following failure every so often:

======================================================================
ERROR: test01_basic_replication (bsddb.test.test_replication.DBReplicationManager)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "s:\src\svn+ssh\pythondev at svn.python.org\python\branches\tnelson-trunk-bsddb-47-upgrade\lib\bsddb\test\test_replication.py", line 101, in setUp
    self.assertTrue(time.time()<timeout)
AssertionError

Can you comment on this?

Apart from this small issue, the other 311 tests pass on x86 and x64 with flying colours, so nice work, whatever you've been doing ;-)


Regards,

        Trent.


From martin at v.loewis.de  Wed Jun 18 09:24:25 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 18 Jun 2008 09:24:25 +0200
Subject: [Python-Dev] [Python-checkins] r64360 -
	python/trunk/Lib/lib2to3/refactor.py
In-Reply-To: <4858B166.5070300@gmail.com>
References: <20080617230919.B2EA11E4004@bag.python.org>
	<48589636.202@v.loewis.de> <4858B166.5070300@gmail.com>
Message-ID: <4858B829.2060002@v.loewis.de>

>> Please never commit to python/trunk/Lib/lib2to3.
>> Instead, commit to sandbox/trunk/2to3, and merge
>> that using svnmerge.
> 
> Could you explain that policy? Why isn't 2to3 maintained in the
> trunk but in the sandbox? Or maybe more general: Where can I find
> more information on the svn directory layout?

It started off as a sandbox project, also with the intent of releasing
it separately. At PyCon 2008, I restructured it introducing lib2to3,
which I then copied into the trunk, and from there into the 3k branch.
The official sources still remain in the sandbox:
a) for historical reasons, because that's where the main 2to3
   contributors commit to (who might not even have a trunk checkout,
   only a 2.5 installation and a 3.0 checkout)
b) because the 2to3 sources contain a few files that don't really
   "belong" into the trunk (although a place for them could probably
   be found), such as find_pattern utility.
c) to continue allowing stand-alone releases; IIUC, Collin Winter
   wants to change this package into a general Python rewriting
   library, which might be useful even in a 2.5 context.

I don't think there can be much said about the layout of the repository:
the top-level directory contains various projects (including python),
and sandbox contains "stuff".

Within the Python tree, there are really not that many exceptions
to the rule "trunk is where all new features go", apart from the 3k
branch, of course. One class of exceptions is libraries that need
to stay backwards compatible, see PEP 291. The other class of
exceptions is libraries whose official sources are outside trunk,
namely (possibly incomplete):
- lib2to3
- xml.etree, and Modules/_elementtree.c
- bsddb (recently)
- _multiprocessing ???
- Modules/zlib
Those have special commit constraints, and the maintainer should be
contacted if you don't know what the constraints are. A few other
packages are also separately released but don't have such constraints:
- ctypes
- sqlite
For these packages, the maintainers will pickup and changes
themselves and merge them into whatever external repository they
may use.

Regards,
Martin

From pete.forman at westerngeco.com  Wed Jun 18 11:50:59 2008
From: pete.forman at westerngeco.com (Pete Forman)
Date: Wed, 18 Jun 2008 10:50:59 +0100
Subject: [Python-Dev] Epoch and Platform
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
	<d2155e360806170926p414a6dc8nb414eafbd96986b8@mail.gmail.com>
	<ca471dc20806171003v7cbf1638t44fd0f714092f54@mail.gmail.com>
	<d2155e360806171040p2e865f8dxc0b8982c29c75221@mail.gmail.com>
	<ca471dc20806171056n3348791al72c28500312553d1@mail.gmail.com>
	<d2155e360806171109m38c905d8o68ed57dad0a10bf5@mail.gmail.com>
	<ca471dc20806171134p7c2433ch62cf5fa1df852b3a@mail.gmail.com>
Message-ID: <y753oyv0.fsf@wgmail2.gatwick.eur.slb.com>

"Guido van Rossum" <guido at python.org> writes:
 > On Tue, Jun 17, 2008 at 11:09 AM, Curt Hagenlocher
 > <curt at hagenlocher.org> wrote:
>> On Tue, Jun 17, 2008 at 10:56 AM, Guido van Rossum
>> <guido at python.org> wrote:
>>> On Tue, Jun 17, 2008 at 10:40 AM, Curt Hagenlocher
>>> <curt at hagenlocher.org> wrote:
>>>>
>>>> There's no real urgency.  The reason this came up is because I
>>>> just implemented zlib, which automatically enabled the gzip unit
>>>> tests.  The gzip tests are failing because the current timestamp
>>>> can't be written as a 32-bit value.
>>>
>>> Why is that? Is it because your epoch is different? If so, I would
>>> much prefer the epoch to be 1970. (Maybe this is the resolution
>>> you're seeking?)
>>
>> Yes!  Except that I was hoping for something a little stronger, like
>> "the epoch must be 1970" or "the epoch can be anything you want, but
>> you're utterly retarded if you pick something other than 1970".  But
>> I'll definitely settle for "much prefer". :)
>
 > I could go with the "utterly retarded" wording. :-)

The POSIX (UNIX) rules for handling time are laid out clearly.  Not
everyone is happy with how leap seconds work but at least they can be
dealt with by those who need to.  Most people will use local time or
UTC and other time scales are possible with a bit of effort.  If you
were to allow an epoch earlier than 1970 then ambiguity is more likely
to sneak in.

Another reason not to allow an epoch of 1900 is that Microsoft
introduced a bug in its leap year algorithm in Excel.  The better
(less bad) way of using that epoch is to propagate that bug.

According to the Pythonic way you should not have to look before you
leap.  ;-)

+1 for strengthening the definition of epoch in the time module to
1970-01-01T00:00:00Z for all platforms.
-- 
Pete Forman                -./\.-  Disclaimer: This post is originated
WesternGeco                  -./\.-   by myself and does not represent
pete.forman at westerngeco.com    -./\.-   the opinion of Schlumberger or
http://petef.22web.net           -./\.-   WesternGeco.


From barry at python.org  Wed Jun 18 13:45:47 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 18 Jun 2008 07:45:47 -0400
Subject: [Python-Dev] Another run at beta one
Message-ID: <F7467050-4DEC-4663-AD2F-4C4739FCA948@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Well, I'm going to try again tonight to release beta one for Python  
2.6 and 3.0.  I'm planning on starting the release at 6pm US/Eastern  
or 2200 UTC.

Right now the Python 2.6 buildbots look pretty good.  We have /one/  
green buildbot for Python 3.0.  That's better than last week at least,  
but it would be great to get a few more.  I know folks have been  
working hard at this, and I really appreciate it.

I think we're looking fairly good on the release critical bugs.  I'll  
clear 3089 and 3009 tonight, and check locally on 3088 over the day.

If you're looking for a way to help out, see if you can turn any of  
the other 3.0 buildbots green.  I will be hanging out on #python-dev  
all day, but ping my nick (barry) if you need my attention.

Thanks, and let's get it done today!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFj1bHEjvBPtnXfVAQLrvAP/ftfpN50HbeimRoK2WhX4RcuN9G81IfbJ
4+CXdjH7WLC/wwKvNp1Q6qc3vHWTqPgA7jF7yowU/a/3FretkzUPGNmtjoWvfQzM
BL1ktG6fh298+KRJTSTXGeNSEzjwBg2GryyQJV5xmwsUMiwvrI4OE3w8O6eE5AkS
gFmU86oZSFw=
=SiNl
-----END PGP SIGNATURE-----

From jnoller at gmail.com  Wed Jun 18 13:54:17 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 18 Jun 2008 07:54:17 -0400
Subject: [Python-Dev] [Python-3000] Another run at beta one
In-Reply-To: <F7467050-4DEC-4663-AD2F-4C4739FCA948@python.org>
References: <F7467050-4DEC-4663-AD2F-4C4739FCA948@python.org>
Message-ID: <4222a8490806180454i5671c8f6sd9d2423961eec940@mail.gmail.com>

On Wed, Jun 18, 2008 at 7:45 AM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Well, I'm going to try again tonight to release beta one for Python 2.6 and
> 3.0.  I'm planning on starting the release at 6pm US/Eastern or 2200 UTC.
>
> Right now the Python 2.6 buildbots look pretty good.  We have /one/ green
> buildbot for Python 3.0.  That's better than last week at least, but it
> would be great to get a few more.  I know folks have been working hard at
> this, and I really appreciate it.
>
> I think we're looking fairly good on the release critical bugs.  I'll clear
> 3089 and 3009 tonight, and check locally on 3088 over the day.
>
> If you're looking for a way to help out, see if you can turn any of the
> other 3.0 buildbots green.  I will be hanging out on #python-dev all day,
> but ping my nick (barry) if you need my attention.
>
> Thanks, and let's get it done today!
> - -Barry
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
>
> iQCVAwUBSFj1bHEjvBPtnXfVAQLrvAP/ftfpN50HbeimRoK2WhX4RcuN9G81IfbJ
> 4+CXdjH7WLC/wwKvNp1Q6qc3vHWTqPgA7jF7yowU/a/3FretkzUPGNmtjoWvfQzM
> BL1ktG6fh298+KRJTSTXGeNSEzjwBg2GryyQJV5xmwsUMiwvrI4OE3w8O6eE5AkS
> gFmU86oZSFw=
> =SiNl
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/jnoller%40gmail.com
>

Please let me know about 3088 - I've disabled the more unreliable
tests until I can give the entire suite some TLC. I'm still trying to
work around/identify the "Function not Implemented" error in issue3111
 - this happens to be failing the tests on a few of the linux boxes. I
suspected it was due to a combination of chroot + lack of /dev/shm on
some of the build bots, ergo a patch put in by Ben last night to skip
the tests on machines lacking the device, but it doesn't seem to be
skipping/the device exists.

-jesse

From jnoller at gmail.com  Wed Jun 18 16:30:50 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 18 Jun 2008 10:30:50 -0400
Subject: [Python-Dev] [Python-3000] Another run at beta one
In-Reply-To: <4222a8490806180454i5671c8f6sd9d2423961eec940@mail.gmail.com>
References: <F7467050-4DEC-4663-AD2F-4C4739FCA948@python.org>
	<4222a8490806180454i5671c8f6sd9d2423961eec940@mail.gmail.com>
Message-ID: <4222a8490806180730i616ec5fdyfec8478a80ec6814@mail.gmail.com>

On Wed, Jun 18, 2008 at 7:54 AM, Jesse Noller <jnoller at gmail.com> wrote:
> On Wed, Jun 18, 2008 at 7:45 AM, Barry Warsaw <barry at python.org> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Well, I'm going to try again tonight to release beta one for Python 2.6 and
>> 3.0.  I'm planning on starting the release at 6pm US/Eastern or 2200 UTC.
>>
>> Right now the Python 2.6 buildbots look pretty good.  We have /one/ green
>> buildbot for Python 3.0.  That's better than last week at least, but it
>> would be great to get a few more.  I know folks have been working hard at
>> this, and I really appreciate it.
>>
>> I think we're looking fairly good on the release critical bugs.  I'll clear
>> 3089 and 3009 tonight, and check locally on 3088 over the day.
>>
>> If you're looking for a way to help out, see if you can turn any of the
>> other 3.0 buildbots green.  I will be hanging out on #python-dev all day,
>> but ping my nick (barry) if you need my attention.
>>
>> Thanks, and let's get it done today!
>> - -Barry
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.9 (Darwin)
>>
>> iQCVAwUBSFj1bHEjvBPtnXfVAQLrvAP/ftfpN50HbeimRoK2WhX4RcuN9G81IfbJ
>> 4+CXdjH7WLC/wwKvNp1Q6qc3vHWTqPgA7jF7yowU/a/3FretkzUPGNmtjoWvfQzM
>> BL1ktG6fh298+KRJTSTXGeNSEzjwBg2GryyQJV5xmwsUMiwvrI4OE3w8O6eE5AkS
>> gFmU86oZSFw=
>> =SiNl
>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> Python-3000 mailing list
>> Python-3000 at python.org
>> http://mail.python.org/mailman/listinfo/python-3000
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-3000/jnoller%40gmail.com
>>

Update: Most of the trunk bots went green post r64375, so I merged the
change over to py3k. The change aggressively skips the MP package
tests if RLock instantiation throws an OSError.

-jesse

From guido at python.org  Wed Jun 18 18:21:46 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Jun 2008 09:21:46 -0700
Subject: [Python-Dev] Epoch and Platform
In-Reply-To: <y753oyv0.fsf@wgmail2.gatwick.eur.slb.com>
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
	<d2155e360806170926p414a6dc8nb414eafbd96986b8@mail.gmail.com>
	<ca471dc20806171003v7cbf1638t44fd0f714092f54@mail.gmail.com>
	<d2155e360806171040p2e865f8dxc0b8982c29c75221@mail.gmail.com>
	<ca471dc20806171056n3348791al72c28500312553d1@mail.gmail.com>
	<d2155e360806171109m38c905d8o68ed57dad0a10bf5@mail.gmail.com>
	<ca471dc20806171134p7c2433ch62cf5fa1df852b3a@mail.gmail.com>
	<y753oyv0.fsf@wgmail2.gatwick.eur.slb.com>
Message-ID: <ca471dc20806180921m5854cfebld1eb74aa6753542b@mail.gmail.com>

Leap seconds? Leap seconds aren't supposed to exist when looking at
POSIX timestamps. They don't exist when I look at my watch either, and
that's what I expect from any time-keeping device except those for use
by astronomers.

On Wed, Jun 18, 2008 at 2:50 AM, Pete Forman
<pete.forman at westerngeco.com> wrote:
> "Guido van Rossum" <guido at python.org> writes:
>  > On Tue, Jun 17, 2008 at 11:09 AM, Curt Hagenlocher
>  > <curt at hagenlocher.org> wrote:
>>> On Tue, Jun 17, 2008 at 10:56 AM, Guido van Rossum
>>> <guido at python.org> wrote:
>>>> On Tue, Jun 17, 2008 at 10:40 AM, Curt Hagenlocher
>>>> <curt at hagenlocher.org> wrote:
>>>>>
>>>>> There's no real urgency.  The reason this came up is because I
>>>>> just implemented zlib, which automatically enabled the gzip unit
>>>>> tests.  The gzip tests are failing because the current timestamp
>>>>> can't be written as a 32-bit value.
>>>>
>>>> Why is that? Is it because your epoch is different? If so, I would
>>>> much prefer the epoch to be 1970. (Maybe this is the resolution
>>>> you're seeking?)
>>>
>>> Yes!  Except that I was hoping for something a little stronger, like
>>> "the epoch must be 1970" or "the epoch can be anything you want, but
>>> you're utterly retarded if you pick something other than 1970".  But
>>> I'll definitely settle for "much prefer". :)
>>
>  > I could go with the "utterly retarded" wording. :-)
>
> The POSIX (UNIX) rules for handling time are laid out clearly.  Not
> everyone is happy with how leap seconds work but at least they can be
> dealt with by those who need to.  Most people will use local time or
> UTC and other time scales are possible with a bit of effort.  If you
> were to allow an epoch earlier than 1970 then ambiguity is more likely
> to sneak in.
>
> Another reason not to allow an epoch of 1900 is that Microsoft
> introduced a bug in its leap year algorithm in Excel.  The better
> (less bad) way of using that epoch is to propagate that bug.
>
> According to the Pythonic way you should not have to look before you
> leap.  ;-)
>
> +1 for strengthening the definition of epoch in the time module to
> 1970-01-01T00:00:00Z for all platforms.
> --
> Pete Forman                -./\.-  Disclaimer: This post is originated
> WesternGeco                  -./\.-   by myself and does not represent
> pete.forman at westerngeco.com    -./\.-   the opinion of Schlumberger or
> http://petef.22web.net           -./\.-   WesternGeco.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tnelson at onresolve.com  Wed Jun 18 18:20:14 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Wed, 18 Jun 2008 17:20:14 +0100
Subject: [Python-Dev] test_multiprocessing: test_listener_client flakiness
Message-ID: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com>

I gave my Windows buildbots a little bit of TLC last night.  This little chestnut in test_multiprocessing.py around line 1346 is causing my buildbots to wedge more often than not:

    def test_listener_client(self):
        for family in self.connection.families:
            l = self.connection.Listener(family=family)
            p = self.Process(target=self._test, args=(l.address,))
            p.set_daemon(True)
            p.start()
            conn = l.accept()
            self.assertEqual(conn.recv(), 'hello')
            p.join()
            l.close()

The wedging will be a result of that accept() call.  Not knowing anything about the module or the test suite, I can only assume that there's a race condition introduced between when the subprocess attempts to connect to the listener, versus when the l.accept() call is actually entered.  (On the basis that a race condition would explain why sometimes it wedges and sometimes it doesn't.)

Just FYI, the error in the buildbot log (http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/810/step-test/0) when this occurs is as follows:

test_multiprocessing

command timed out: 1200 seconds without output
SIGKILL failed to kill process
using fake rc=-1
program finished with exit code -1
remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process
]

(The fact it can't be killed cleanly is a bug in Twisted's signalProcess('KILL') method, which doesn't work against Python processes that have entered accept() calls on Windows (which present the 'wedged' behaviour and have to be forcibly killed with OpenProcess/TerminateProcess).)

    Trent.

From amauryfa at gmail.com  Wed Jun 18 18:35:26 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 18 Jun 2008 18:35:26 +0200
Subject: [Python-Dev] test_multiprocessing: test_listener_client
	flakiness
In-Reply-To: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com>
References: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com>
Message-ID: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>

Hello,

2008/6/18 Trent Nelson <tnelson at onresolve.com>:
> I gave my Windows buildbots a little bit of TLC last night.  This little chestnut in test_multiprocessing.py around line 1346 is causing my buildbots to wedge more often than not:
>
>    def test_listener_client(self):
>        for family in self.connection.families:
>            l = self.connection.Listener(family=family)
>            p = self.Process(target=self._test, args=(l.address,))
>            p.set_daemon(True)
>            p.start()
>            conn = l.accept()
>            self.assertEqual(conn.recv(), 'hello')
>            p.join()
>            l.close()
>
> The wedging will be a result of that accept() call.  Not knowing anything about the module or the test suite, I can only assume that there's a race condition introduced between when the subprocess attempts to connect to the listener, versus when the l.accept() call is actually entered.  (On the basis that a race condition would explain why sometimes it wedges and sometimes it doesn't.)
>
> Just FYI, the error in the buildbot log (http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/810/step-test/0) when this occurs is as follows:
>
> test_multiprocessing
>
> command timed out: 1200 seconds without output
> SIGKILL failed to kill process
> using fake rc=-1
> program finished with exit code -1
> remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
> Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process
> ]
>
> (The fact it can't be killed cleanly is a bug in Twisted's signalProcess('KILL') method, which doesn't work against Python processes that have entered accept() calls on Windows (which present the 'wedged' behaviour and have to be forcibly killed with OpenProcess/TerminateProcess).)

I just found the cause of the problem ten minutes ago:
It seems that when a socket listens on the address "127.0.0.1" or
"localhost", another process cannot connect to it using the machine's
name (even from the same machine).
The best seems to listen with the empty address "".

Index: Lib/multiprocessing/connection.py
===================================================================
--- Lib/multiprocessing/connection.py   (revision 64374)
+++ Lib/multiprocessing/connection.py   (working copy)
@@ -49,7 +49,7 @@
     Return an arbitrary free address for the given family
     '''
     if family == 'AF_INET':
-        return ('localhost', 0)
+        return ('', 0)
     elif family == 'AF_UNIX':
         return tempfile.mktemp(prefix='listener-', dir=get_temp_dir())
     elif family == 'AF_PIPE':

And the test started to pass for me.
Can you please check this in if it works; I don't have svn access for
the moment.

-- 
Amaury Forgeot d'Arc

From jnoller at gmail.com  Wed Jun 18 18:39:30 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 18 Jun 2008 12:39:30 -0400
Subject: [Python-Dev] test_multiprocessing: test_listener_client
	flakiness
In-Reply-To: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
References: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com>
	<e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
Message-ID: <4222a8490806180939xc1e5340g560f35c2b639321e@mail.gmail.com>

On Wed, Jun 18, 2008 at 12:35 PM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> Hello,
>
> 2008/6/18 Trent Nelson <tnelson at onresolve.com>:
>> I gave my Windows buildbots a little bit of TLC last night.  This little chestnut in test_multiprocessing.py around line 1346 is causing my buildbots to wedge more often than not:
>>
>>    def test_listener_client(self):
>>        for family in self.connection.families:
>>            l = self.connection.Listener(family=family)
>>            p = self.Process(target=self._test, args=(l.address,))
>>            p.set_daemon(True)
>>            p.start()
>>            conn = l.accept()
>>            self.assertEqual(conn.recv(), 'hello')
>>            p.join()
>>            l.close()
>>
>> The wedging will be a result of that accept() call.  Not knowing anything about the module or the test suite, I can only assume that there's a race condition introduced between when the subprocess attempts to connect to the listener, versus when the l.accept() call is actually entered.  (On the basis that a race condition would explain why sometimes it wedges and sometimes it doesn't.)
>>
>> Just FYI, the error in the buildbot log (http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/810/step-test/0) when this occurs is as follows:
>>
>> test_multiprocessing
>>
>> command timed out: 1200 seconds without output
>> SIGKILL failed to kill process
>> using fake rc=-1
>> program finished with exit code -1
>> remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
>> Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process
>> ]
>>
>> (The fact it can't be killed cleanly is a bug in Twisted's signalProcess('KILL') method, which doesn't work against Python processes that have entered accept() calls on Windows (which present the 'wedged' behaviour and have to be forcibly killed with OpenProcess/TerminateProcess).)
>
> I just found the cause of the problem ten minutes ago:
> It seems that when a socket listens on the address "127.0.0.1" or
> "localhost", another process cannot connect to it using the machine's
> name (even from the same machine).
> The best seems to listen with the empty address "".
>
> Index: Lib/multiprocessing/connection.py
> ===================================================================
> --- Lib/multiprocessing/connection.py   (revision 64374)
> +++ Lib/multiprocessing/connection.py   (working copy)
> @@ -49,7 +49,7 @@
>     Return an arbitrary free address for the given family
>     '''
>     if family == 'AF_INET':
> -        return ('localhost', 0)
> +        return ('', 0)
>     elif family == 'AF_UNIX':
>         return tempfile.mktemp(prefix='listener-', dir=get_temp_dir())
>     elif family == 'AF_PIPE':
>
> And the test started to pass for me.
> Can you please check this in if it works; I don't have svn access for
> the moment.
>
> --
> Amaury Forgeot d'Arc


I am testing the patch locally now.

From solipsis at pitrou.net  Wed Jun 18 18:40:02 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Jun 2008 16:40:02 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?test=5Fmultiprocessing=3Atest=5Flistener?=
	=?utf-8?q?=5Fclient=09flakiness?=
References: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com>
	<e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
Message-ID: <loom.20080618T163852-134@post.gmane.org>

Amaury Forgeot d'Arc <amauryfa <at> gmail.com> writes:
>      if family == 'AF_INET':
> -        return ('localhost', 0)
> +        return ('', 0)
>      elif family == 'AF_UNIX':
>          return tempfile.mktemp(prefix='listener-', dir=get_temp_dir())
>      elif family == 'AF_PIPE':

Doesn't it have important security implications?




From exarkun at divmod.com  Wed Jun 18 18:41:36 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 18 Jun 2008 12:41:36 -0400
Subject: [Python-Dev] test_multiprocessing: test_listener_client
 flakiness
In-Reply-To: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
Message-ID: <20080618164136.4714.382689990.divmod.quotient.10414@ohm>

On Wed, 18 Jun 2008 18:35:26 +0200, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
> [snip]
>
>I just found the cause of the problem ten minutes ago:
>It seems that when a socket listens on the address "127.0.0.1" or
>"localhost", another process cannot connect to it using the machine's
>name (even from the same machine).

That's because when you try to connect to A:B you won't connect to a
server listening on X:B - somewhat by design.  Changing the test to
listen on A:B and X:B might fix it, but so would changing it to connect
to the same address it listens to.

>The best seems to listen with the empty address "".

This will cause it to listen on all available interfaces, including
public ones.  It's rather unlikely that someone from the internet
will connect to the port while the test suite is running and use it
to do terrible things to you, but it's not /impossible/.

I'd suggest changing the client to connect to "127.0.0.1" or "localhost",
instead.

Jean-Paul

From amauryfa at gmail.com  Wed Jun 18 18:48:20 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 18 Jun 2008 18:48:20 +0200
Subject: [Python-Dev] test_multiprocessing: test_listener_client
	flakiness
In-Reply-To: <20080618164136.4714.382689990.divmod.quotient.10414@ohm>
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
Message-ID: <e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>

Hello,

2008/6/18 Jean-Paul Calderone <exarkun at divmod.com>:
> On Wed, 18 Jun 2008 18:35:26 +0200, Amaury Forgeot d'Arc
> <amauryfa at gmail.com> wrote:
>>
>> [snip]
>>
>> I just found the cause of the problem ten minutes ago:
>> It seems that when a socket listens on the address "127.0.0.1" or
>> "localhost", another process cannot connect to it using the machine's
>> name (even from the same machine).
>
> That's because when you try to connect to A:B you won't connect to a
> server listening on X:B - somewhat by design.  Changing the test to
> listen on A:B and X:B might fix it, but so would changing it to connect
> to the same address it listens to.
>
>> The best seems to listen with the empty address "".
>
> This will cause it to listen on all available interfaces, including
> public ones.  It's rather unlikely that someone from the internet
> will connect to the port while the test suite is running and use it
> to do terrible things to you, but it's not /impossible/.
>
> I'd suggest changing the client to connect to "127.0.0.1" or "localhost",
> instead.

OK, I am not a tcp expert.
How about the following patch?
It seems to me that it is wrong for the server to lie about the
address it listens to.

Index: ../../Lib/multiprocessing/connection.py
===================================================================
--- ../../Lib/multiprocessing/connection.py     (revision 64378)
+++ ../../Lib/multiprocessing/connection.py     (working copy)
@@ -216,8 +216,6 @@
         self._socket.bind(address)
         self._socket.listen(backlog)
         address = self._socket.getsockname()
-        if type(address) is tuple:
-            address = (socket.getfqdn(address[0]),) + address[1:]
         self._address = address
         self._family = family
         self._last_accepted = None



-- 
Amaury Forgeot d'Arc

From g.brandl at gmx.net  Wed Jun 18 18:56:47 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 18 Jun 2008 18:56:47 +0200
Subject: [Python-Dev] Opcode frequency
Message-ID: <g3beok$p0f$1@ger.gmane.org>

Hi,

Maciej Fijalkowski did an opcode analysis for PyPy,
it also shows the relative frequency of opcodes following a
specifc one:

http://codespeak.net/svn/user/fijal/opcodes.txt

Might it make sense to add more PREDICT()ions based
on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR?

Georg


From pete.forman at westerngeco.com  Wed Jun 18 18:58:18 2008
From: pete.forman at westerngeco.com (Pete Forman)
Date: Wed, 18 Jun 2008 17:58:18 +0100
Subject: [Python-Dev] Epoch and Platform
References: <d2155e360806161608hac44936h23855581362eb130@mail.gmail.com>
	<ca471dc20806161638h45728126he6995359ce4f4350@mail.gmail.com>
	<d2155e360806170926p414a6dc8nb414eafbd96986b8@mail.gmail.com>
	<ca471dc20806171003v7cbf1638t44fd0f714092f54@mail.gmail.com>
	<d2155e360806171040p2e865f8dxc0b8982c29c75221@mail.gmail.com>
	<ca471dc20806171056n3348791al72c28500312553d1@mail.gmail.com>
	<d2155e360806171109m38c905d8o68ed57dad0a10bf5@mail.gmail.com>
	<ca471dc20806171134p7c2433ch62cf5fa1df852b3a@mail.gmail.com>
	<y753oyv0.fsf@wgmail2.gatwick.eur.slb.com>
	<ca471dc20806180921m5854cfebld1eb74aa6753542b@mail.gmail.com>
Message-ID: <ve06he8l.fsf@wgmail2.gatwick.eur.slb.com>

"Guido van Rossum" <guido at python.org> writes:

 > Leap seconds?  Leap seconds aren't supposed to exist when looking
 > at POSIX timestamps.  They don't exist when I look at my watch
 > either, and that's what I expect from any time-keeping device
 > except those for use by astronomers.

That's pretty much what I was saying.  The POSIX rule for Seconds
Since the Epoch explicitly excludes leap seconds.  Astronomers and the
like then know what further adjustments are required for the their
needs.  The rest of us get along with the assumption that there are
86400 seconds in every day.
-- 
Pete Forman                -./\.-  Disclaimer: This post is originated
WesternGeco                  -./\.-   by myself and does not represent
pete.forman at westerngeco.com    -./\.-   the opinion of Schlumberger or
http://petef.22web.net           -./\.-   WesternGeco.


From solipsis at pitrou.net  Wed Jun 18 19:51:56 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Jun 2008 17:51:56 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?test=5Fmultiprocessing=3Atest=5Flistener?=
	=?utf-8?q?=5Fclient=09flakiness?=
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>
Message-ID: <loom.20080618T174848-255@post.gmane.org>

Amaury Forgeot d'Arc <amauryfa <at> gmail.com> writes:
> 
> OK, I am not a tcp expert.
> How about the following patch?
> It seems to me that it is wrong for the server to lie about the
> address it listens to.

Maybe I'm missing something but... if the client part defaults to connecting
to "localhost" in TCP mode, why doesn't the server part also default to
listening on "localhost" in TCP mode? Having different address detection
algorithms on both ends sounds like a recipe for bugs like this.

In any case, perhaps one should ask guidance to the original author before
making important changes like this one.





From jnoller at gmail.com  Wed Jun 18 19:55:12 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 18 Jun 2008 13:55:12 -0400
Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness
In-Reply-To: <loom.20080618T174848-255@post.gmane.org>
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>
	<loom.20080618T174848-255@post.gmane.org>
Message-ID: <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>

On Wed, Jun 18, 2008 at 1:51 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Amaury Forgeot d'Arc <amauryfa <at> gmail.com> writes:
>>
>> OK, I am not a tcp expert.
>> How about the following patch?
>> It seems to me that it is wrong for the server to lie about the
>> address it listens to.
>
> Maybe I'm missing something but... if the client part defaults to connecting
> to "localhost" in TCP mode, why doesn't the server part also default to
> listening on "localhost" in TCP mode? Having different address detection
> algorithms on both ends sounds like a recipe for bugs like this.
>
> In any case, perhaps one should ask guidance to the original author before
> making important changes like this one.

The server component *is* listening to localhost by default. The
client in the code I pasted has the server's .address attribute passed
into it to tell the client where to connect to.

-jesse

From jnoller at gmail.com  Wed Jun 18 19:59:08 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 18 Jun 2008 13:59:08 -0400
Subject: [Python-Dev] test_multiprocessing: test_listener_client
	flakiness
In-Reply-To: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com>
References: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com>
Message-ID: <4222a8490806181059m407ba440l3cfd0689d7e2d2c5@mail.gmail.com>

On Wed, Jun 18, 2008 at 12:20 PM, Trent Nelson <tnelson at onresolve.com> wrote:
> I gave my Windows buildbots a little bit of TLC last night.  This little chestnut in test_multiprocessing.py around line 1346 is causing my buildbots to wedge more often than not:
>
>    def test_listener_client(self):
>        for family in self.connection.families:
>            l = self.connection.Listener(family=family)
>            p = self.Process(target=self._test, args=(l.address,))
>            p.set_daemon(True)
>            p.start()
>            conn = l.accept()
>            self.assertEqual(conn.recv(), 'hello')
>            p.join()
>            l.close()
>
> The wedging will be a result of that accept() call.  Not knowing anything about the module or the test suite, I can only assume that there's a race condition introduced between when the subprocess attempts to connect to the listener, versus when the l.accept() call is actually entered.  (On the basis that a race condition would explain why sometimes it wedges and sometimes it doesn't.)
>
> Just FYI, the error in the buildbot log (http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/810/step-test/0) when this occurs is as follows:
>
> test_multiprocessing
>
> command timed out: 1200 seconds without output
> SIGKILL failed to kill process
> using fake rc=-1
> program finished with exit code -1
> remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
> Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process
> ]
>
> (The fact it can't be killed cleanly is a bug in Twisted's signalProcess('KILL') method, which doesn't work against Python processes that have entered accept() calls on Windows (which present the 'wedged' behaviour and have to be forcibly killed with OpenProcess/TerminateProcess).)
>
>    Trent.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com
>

Trent, I commented out the test causing the issue for now (the entire
suite is being revisited post beta-1)
See r64378
-jesse

From solipsis at pitrou.net  Wed Jun 18 20:04:45 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Jun 2008 18:04:45 +0000 (UTC)
Subject: [Python-Dev] Opcode frequency
References: <g3beok$p0f$1@ger.gmane.org>
Message-ID: <loom.20080618T175215-871@post.gmane.org>


Hi,

> Maciej Fijalkowski did an opcode analysis for PyPy,
> it also shows the relative frequency of opcodes following a
> specifc one:
> 
> http://codespeak.net/svn/user/fijal/opcodes.txt

Nice, but we have to be careful here: what is the tested workload?
For example, I find it hard to believe that CALL_FUNCTION_VAR_KW would 
always (99%) be followed by STORE_FAST.

I'm not even sure we're talking about the same VM/compiler. What are
CALL_LIKELY_BUILTIN and LOOKUP_METHOD? :-)

> Might it make sense to add more PREDICT()ions based
> on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR?

This particular one sounds fine, yes.

Another approach is to create opcodes which better reflect actual usage.
For example part of the following patch (*) was to create opcodes fusing
JUMP_IF_{TRUE,FALSE} and POP_TOP. This is because in most cases, you want
to pop the comparison result regardless of its value (true or false), which
nowadays produces the following kind of code:

    JUMP_IF_FALSE <branch_for_false>
    POP_TOP
    [... do stuff and then ...]
    JUMP_FORWARD <branch_for_false_after_pop_top>
branch_for_false:
    POP_TOP
branch_for_false_after_pop_top:
    [etc.]

With a JUMP_IF_FALSE which also pops the pop of the stack, this gives:

    JUMP_IF_FALSE <branch_for_false>
    [... do stuff and then ...]
branch_for_false:
    [etc.]

Prediction doesn't help you in this case.

(*) http://bugs.python.org/issue2459

Regards

Antoine.



From solipsis at pitrou.net  Wed Jun 18 20:09:51 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Jun 2008 18:09:51 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?test=5Fmultiprocessing=3Atest=5Flistener?=
	=?utf-8?q?=5Fclient_flakiness?=
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>
	<loom.20080618T174848-255@post.gmane.org>
	<4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>
Message-ID: <loom.20080618T180853-875@post.gmane.org>

Jesse Noller <jnoller <at> gmail.com> writes:
> 
> The server component *is* listening to localhost by default. The
> client in the code I pasted has the server's .address attribute passed
> into it to tell the client where to connect to.

So I take it that the .address attribute is different from "localhost",
but why? Is there any point in trying to be clever here?





From jnoller at gmail.com  Wed Jun 18 20:32:35 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 18 Jun 2008 14:32:35 -0400
Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness
In-Reply-To: <loom.20080618T180853-875@post.gmane.org>
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>
	<loom.20080618T174848-255@post.gmane.org>
	<4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>
	<loom.20080618T180853-875@post.gmane.org>
Message-ID: <4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com>

On Wed, Jun 18, 2008 at 2:09 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Jesse Noller <jnoller <at> gmail.com> writes:
>>
>> The server component *is* listening to localhost by default. The
>> client in the code I pasted has the server's .address attribute passed
>> into it to tell the client where to connect to.
>
> So I take it that the .address attribute is different from "localhost",
> but why? Is there any point in trying to be clever here?

No, it is *not* different than localhost - it *is* localhost as that
is the object.address of the server.

From g.brandl at gmx.net  Wed Jun 18 21:45:07 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 18 Jun 2008 21:45:07 +0200
Subject: [Python-Dev] Opcode frequency
In-Reply-To: <loom.20080618T175215-871@post.gmane.org>
References: <g3beok$p0f$1@ger.gmane.org>
	<loom.20080618T175215-871@post.gmane.org>
Message-ID: <g3bok8$tko$1@ger.gmane.org>

Antoine Pitrou schrieb:
> Hi,
> 
>> Maciej Fijalkowski did an opcode analysis for PyPy,
>> it also shows the relative frequency of opcodes following a
>> specifc one:
>> 
>> http://codespeak.net/svn/user/fijal/opcodes.txt
> 
> Nice, but we have to be careful here: what is the tested workload?
> For example, I find it hard to believe that CALL_FUNCTION_VAR_KW would 
> always (99%) be followed by STORE_FAST.

I'm sorry, I should have given a reference to the original message:
http://codespeak.net/pipermail/pypy-dev/2008q2/004666.html

> I'm not even sure we're talking about the same VM/compiler. What are
> CALL_LIKELY_BUILTIN and LOOKUP_METHOD? :-)

He used the PyPy compiler, of course.

>> Might it make sense to add more PREDICT()ions based
>> on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR?
> 
> This particular one sounds fine, yes.
> 
> Another approach is to create opcodes which better reflect actual usage.
> For example part of the following patch (*) was to create opcodes fusing
> JUMP_IF_{TRUE,FALSE} and POP_TOP. This is because in most cases, you want
> to pop the comparison result regardless of its value (true or false), which
> nowadays produces the following kind of code:
> 
>     JUMP_IF_FALSE <branch_for_false>
>     POP_TOP
>     [... do stuff and then ...]
>     JUMP_FORWARD <branch_for_false_after_pop_top>
> branch_for_false:
>     POP_TOP
> branch_for_false_after_pop_top:
>     [etc.]
> 
> With a JUMP_IF_FALSE which also pops the pop of the stack, this gives:
> 
>     JUMP_IF_FALSE <branch_for_false>
>     [... do stuff and then ...]
> branch_for_false:
>     [etc.]
> 
> Prediction doesn't help you in this case.

Agreed; is there a way for a JUMP to end up without popping the top
after jumping? It would have to do a DUP first then.

Georg


From solipsis at pitrou.net  Wed Jun 18 23:12:15 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Jun 2008 21:12:15 +0000 (UTC)
Subject: [Python-Dev] Opcode frequency
References: <g3beok$p0f$1@ger.gmane.org>
	<loom.20080618T175215-871@post.gmane.org>
	<g3bok8$tko$1@ger.gmane.org>
Message-ID: <loom.20080618T211058-841@post.gmane.org>

Georg Brandl <g.brandl <at> gmx.net> writes:
> Agreed; is there a way for a JUMP to end up without popping the top
> after jumping? It would have to do a DUP first then.

For example:

>>> def f(x, y):
...  return x and y
... 
>>> import dis
>>> dis.dis(f)
  2           0 LOAD_FAST                0 (x)
              3 JUMP_IF_FALSE            4 (to 10)
              6 POP_TOP             
              7 LOAD_FAST                1 (y)
        >>   10 RETURN_VALUE        




From cfbolz at gmx.de  Wed Jun 18 23:20:50 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 18 Jun 2008 23:20:50 +0200
Subject: [Python-Dev] Opcode frequency
In-Reply-To: <g3bok8$tko$1@ger.gmane.org>
References: <g3beok$p0f$1@ger.gmane.org>	<loom.20080618T175215-871@post.gmane.org>
	<g3bok8$tko$1@ger.gmane.org>
Message-ID: <48597C32.6030607@gmx.de>

Georg Brandl wrote:
> Antoine Pitrou schrieb:
>>> Maciej Fijalkowski did an opcode analysis for PyPy,
>>> it also shows the relative frequency of opcodes following a
>>> specifc one:
>>>
>>> http://codespeak.net/svn/user/fijal/opcodes.txt
>>
>> Nice, but we have to be careful here: what is the tested workload?
>> For example, I find it hard to believe that CALL_FUNCTION_VAR_KW would
>> always (99%) be followed by STORE_FAST.
> 
> I'm sorry, I should have given a reference to the original message:
> http://codespeak.net/pipermail/pypy-dev/2008q2/004666.html
> 
>> I'm not even sure we're talking about the same VM/compiler. What are
>> CALL_LIKELY_BUILTIN and LOOKUP_METHOD? :-)
> 
> He used the PyPy compiler, of course.

Just for the reference, PyPy uses exactly CPython's opcodes. Then there
are some completely optional optimizations which add opcodes (like
CALL_LIKELY_BUILTIN and LOOKUP_METHOD) that can be enabled at compile time.

Cheers,

Carl Friedrich Bolz

From cfbolz at gmx.de  Wed Jun 18 23:20:50 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 18 Jun 2008 23:20:50 +0200
Subject: [Python-Dev] Opcode frequency
In-Reply-To: <g3bok8$tko$1@ger.gmane.org>
References: <g3beok$p0f$1@ger.gmane.org>	<loom.20080618T175215-871@post.gmane.org>
	<g3bok8$tko$1@ger.gmane.org>
Message-ID: <48597C32.6030607@gmx.de>

Georg Brandl wrote:
> Antoine Pitrou schrieb:
>>> Maciej Fijalkowski did an opcode analysis for PyPy,
>>> it also shows the relative frequency of opcodes following a
>>> specifc one:
>>>
>>> http://codespeak.net/svn/user/fijal/opcodes.txt
>>
>> Nice, but we have to be careful here: what is the tested workload?
>> For example, I find it hard to believe that CALL_FUNCTION_VAR_KW would
>> always (99%) be followed by STORE_FAST.
> 
> I'm sorry, I should have given a reference to the original message:
> http://codespeak.net/pipermail/pypy-dev/2008q2/004666.html
> 
>> I'm not even sure we're talking about the same VM/compiler. What are
>> CALL_LIKELY_BUILTIN and LOOKUP_METHOD? :-)
> 
> He used the PyPy compiler, of course.

Just for the reference, PyPy uses exactly CPython's opcodes. Then there
are some completely optional optimizations which add opcodes (like
CALL_LIKELY_BUILTIN and LOOKUP_METHOD) that can be enabled at compile time.

Cheers,

Carl Friedrich Bolz


From amauryfa at gmail.com  Wed Jun 18 23:36:22 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 18 Jun 2008 23:36:22 +0200
Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness
In-Reply-To: <4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com>
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>
	<loom.20080618T174848-255@post.gmane.org>
	<4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>
	<loom.20080618T180853-875@post.gmane.org>
	<4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com>
Message-ID: <e27efe130806181436l114498cdg57154d47bb98f488@mail.gmail.com>

2008/6/18 Jesse Noller <jnoller at gmail.com>:
> On Wed, Jun 18, 2008 at 2:09 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Jesse Noller <jnoller <at> gmail.com> writes:
>>>
>>> The server component *is* listening to localhost by default. The
>>> client in the code I pasted has the server's .address attribute passed
>>> into it to tell the client where to connect to.
>>
>> So I take it that the .address attribute is different from "localhost",
>> but why? Is there any point in trying to be clever here?
>
> No, it is *not* different than localhost - it *is* localhost as that
> is the object.address of the server.

Well, on my win2k machine getfqdn('127.0.0.1') returns the actual name
of the machine.
This is the name that was stored in server.address.
Hence my patch that simply remove the call to getfqdn.

-- 
Amaury Forgeot d'Arc

From ncoghlan at gmail.com  Thu Jun 19 01:03:08 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 19 Jun 2008 09:03:08 +1000
Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness
In-Reply-To: <e27efe130806181436l114498cdg57154d47bb98f488@mail.gmail.com>
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>	<loom.20080618T174848-255@post.gmane.org>	<4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>	<loom.20080618T180853-875@post.gmane.org>	<4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com>
	<e27efe130806181436l114498cdg57154d47bb98f488@mail.gmail.com>
Message-ID: <4859942C.9010009@gmail.com>

Amaury Forgeot d'Arc wrote:
> 2008/6/18 Jesse Noller <jnoller at gmail.com>:
>> On Wed, Jun 18, 2008 at 2:09 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>> Jesse Noller <jnoller <at> gmail.com> writes:
>>>> The server component *is* listening to localhost by default. The
>>>> client in the code I pasted has the server's .address attribute passed
>>>> into it to tell the client where to connect to.
>>> So I take it that the .address attribute is different from "localhost",
>>> but why? Is there any point in trying to be clever here?
>> No, it is *not* different than localhost - it *is* localhost as that
>> is the object.address of the server.
> 
> Well, on my win2k machine getfqdn('127.0.0.1') returns the actual name
> of the machine.
> This is the name that was stored in server.address.
> Hence my patch that simply remove the call to getfqdn.

Point of interest: I was involved in developing an application where we 
gave up on trying to persuade getfqdn() to return reliable results on 
Windows machines - regardless of which local network interface you use 
to do the lookup, Windows 2k3 is likely to return an answer of 
'computername.computerdomain' without bothering to consult the local DNS 
server (annoyingly, it used to do this right on previous Windows versions).

So I'm inclined to consider any code which expects a useful answer out 
of getfqdn() to be non-portable at best and outright broken at worst. We 
worked around the problem by only doing forward DNS lookups and always 
working with addresses in the IP domain.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From python at rcn.com  Thu Jun 19 01:03:19 2008
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 18 Jun 2008 16:03:19 -0700
Subject: [Python-Dev] Opcode frequency
References: <g3beok$p0f$1@ger.gmane.org>
Message-ID: <B556EA862D604758944E9DF7B83FB864@RaymondLaptop1>

> Maciej Fijalkowski did an opcode analysis for PyPy,
> it also shows the relative frequency of opcodes following a
> specifc one:
> 
> http://codespeak.net/svn/user/fijal/opcodes.txt
> 
> Might it make sense to add more PREDICT()ions based
> on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR?

This particular one might be okay.  What code generates it?
Are there other possible successors to BUILD_SLICE?
What code generates those?

There were lots of other opcode pairings that were previously
studied and rejected.  You're going over well traveled ground.

Also, opcode analysis is a somewhat hazardous undertaking.
Dynamic analysis takes into account which codes tend to
occur more often inside loops but it is *very* sensitive
to who wrote the app and their coding style.  These results
always have to be taken with a grain of salt. 


Raymond

From barry at python.org  Thu Jun 19 01:38:19 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 18 Jun 2008 19:38:19 -0400
Subject: [Python-Dev] releasing betas tonight
Message-ID: <8AE8BB5B-447B-4D31-854F-671AE902E4FF@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

There are enough green buildbots and I think all the showstoppers are  
resolvable, so I'm going ahead and releasing beta 1 tonight.  Please  
refrain from any commits unless I ask you to check something in.

We've got a major thunderstorm blowing through right now, but when it  
clears I'll be on irc.freenode.net #python-dev.  I'll start making the  
release as soon as I get on, and feel free to ask any questions.

- -B

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFmca3EjvBPtnXfVAQICfAP8DHTOoQap9MnXJrUY0a3rurgRerMDNxSe
1dmOlhSVGmbQncPSZWP5x0xf1gC5QOCUAofGYfh2xY5ng6BbCQlzMX3Iogy1KvHQ
Fb1HQ4EPUO5PGEQipgvHlkN38OFDqevoyvaRBIOGOppIVjL0v8P66eXNkkLq4BvT
RztyojPrhxE=
=jZ8g
-----END PGP SIGNATURE-----

From barry at python.org  Thu Jun 19 03:02:05 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 18 Jun 2008 21:02:05 -0400
Subject: [Python-Dev] [Python-3000] Another run at beta one
In-Reply-To: <4222a8490806180454i5671c8f6sd9d2423961eec940@mail.gmail.com>
References: <F7467050-4DEC-4663-AD2F-4C4739FCA948@python.org>
	<4222a8490806180454i5671c8f6sd9d2423961eec940@mail.gmail.com>
Message-ID: <9E5AEF05-6169-42A5-8CC8-EEC8750EE00B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 18, 2008, at 7:54 AM, Jesse Noller wrote:

> Please let me know about 3088 - I've disabled the more unreliable
> tests until I can give the entire suite some TLC. I'm still trying to
> work around/identify the "Function not Implemented" error in issue3111
> - this happens to be failing the tests on a few of the linux boxes. I
> suspected it was due to a combination of chroot + lack of /dev/shm on
> some of the build bots, ergo a patch put in by Ben last night to skip
> the tests on machines lacking the device, but it doesn't seem to be
> skipping/the device exists.

We're going to have to deal with 3088 after the beta release.  I'm not  
going to let it hold us up this time.  I think we're going to have a  
tougher time than we think getting the 3.0 release stabilized for  
final though.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFmwDXEjvBPtnXfVAQIFjAP+LGwB1Z3eBLaF/ZEez5TgkpMWdhHZwmWA
zefJnlORHRN4TiawGDNkGdbs+B3O0An8vuCkXTnM97sdKqXYIUZ+0C8qdU5g9OaA
UlX2Wagl1UTC8HHMhmPVlo7AnIzcR0mUOlu7T9SgtjKeEbcrSB4vCT9Cqy7rH7r8
jsHzlmzjswI=
=r2K6
-----END PGP SIGNATURE-----

From barry at python.org  Thu Jun 19 04:46:06 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 18 Jun 2008 22:46:06 -0400
Subject: [Python-Dev] RELEASED Python 2.6b1 and 3.0b1
Message-ID: <1972109D-735D-4485-82F4-9BC8F2984967@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team and the Python community, I  
am happy to announce the first beta releases of Python 2.6 and Python  
3.0.

Please note that these are beta releases, and as such are not suitable  
for production environments.  We continue to strive for a high degree  
of quality, and these releases are intended to freeze the feature set  
for Python 2.6 and 3.0.

 From now until the planned final releases in September 2008, we will  
be fixing known problems and stabilizing these new Python versions.   
You can help by downloading and testing them, providing feedback and  
hopefully helping to fix bugs.  You can also use these releases to  
determine how changes in 2.6 and 3.0 might impact you.  If you find  
things broken or incorrect, please submit bug reports at

   http://bugs.python.org

For more information and downloadable distributions, see the Python  
2.6 website:

   http://www.python.org/download/releases/2.6/

and the Python 3.0 web site:

   http://www.python.org/download/releases/3.0/

See PEP 361 for release schedule details:

    http://www.python.org/dev/peps/pep-0361/

Enjoy,
- -Barry

Barry Warsaw
barry at python.org
Python 2.6/3.0 Release Manager
(on behalf of the entire python-dev team)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFnIbnEjvBPtnXfVAQKCMgP+L9Vpk9vQgYw01KShtF9HyvCKE0guSy86
0Q/IRbkXGahXg910D4nVLY9NORaOSq/VP8Kx+fOU8egUDHvVF9MuKkY75ZHYoixR
mieeV7BIpYHYKYhva48FVoF4p0+sUeNvfeOAP0nnV5Pi2icGqCpzizfSHZloXJjc
Y8M4F5vVrOM=
=Oo15
-----END PGP SIGNATURE-----

From cesare at pronto.it  Thu Jun 19 06:58:07 2008
From: cesare at pronto.it (Cesare Di Mauro)
Date: Thu, 19 Jun 2008 06:58:07 +0200
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <e8a0972d0806151612q1382371dg2ebc8c8913a13ef8@mail.gmail.com>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucp6isfyhlrjc9@conan>
	<48536B7E.30103@v.loewis.de> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org> <op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan> <4854614B.9070903@canterbury.ac.nz>
	<op.ucr0100rhlrjc9@conan>
	<e8a0972d0806151612q1382371dg2ebc8c8913a13ef8@mail.gmail.com>
Message-ID: <op.uczb25jahlrjc9@conan>

Very very, interesting. Thanks. :)

Somebody thinks that Python is unsuitable to implement a DSL: IMO your example prove the contrary. :D

Cesare

In data 16 giugno 2008 alle ore 01:12:46, Alex Martelli <aleaxit at gmail.com> ha scritto:

> +1 on updating the FAQ.  Maybe we could even have it notice that a
> read-only version of the desired semantic for 'with' is easily hacked
> with the *current* semantic of 'with'...:
>
> @contextlib.contextmanager
> def readonly(anobj):
>     caller_globals = sys._getframe(2).f_globals
>     saved_globals = caller_globals.copy()
>     caller_globals.update((n, getattr(anobj, n)) for n in dir(anobj))
>     yield
>     caller_globals.clear()
>     caller_globals.update(saved_globals)
>
> and then, of course,
>
> with readonly(someobj): ...
>
> (local variables take precedence, and in particular all assignments
> define local variables, as usual -- but you can say e.g. 'zz' to mean
> 'someobj.zz', if you're sufficiently keen on giving up the advantage
> of having many well-separated namespaces;-).
>
>
> Alex


From tnelson at onresolve.com  Thu Jun 19 07:39:55 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Thu, 19 Jun 2008 06:39:55 +0100
Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness
In-Reply-To: <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>
	<loom.20080618T174848-255@post.gmane.org>
	<4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>
Message-ID: <6167796BFEB5D0438720AC212E89A6B00787275C@exchange.onresolve.com>

> The server component *is* listening to localhost by default. The
> client in the code I pasted has the server's .address attribute passed
> into it to tell the client where to connect to.

Interestingly enough, I can't coerce test_multiprocessing.test_listener_client to wedge on my XP laptop.  When I run it on my Windows Server 2008 box (same box the buildbots run on), it wedges every time.  So, I suspect this isn't actually a race like I first thought, but something quirky with 2008.  Will dig around for a bit longer and see what I can find...

        Trent.

From aleaxit at gmail.com  Thu Jun 19 07:55:32 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 18 Jun 2008 22:55:32 -0700
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <op.uczb25jahlrjc9@conan>
References: <4852F7EC.6040403@voidspace.org.uk> <op.ucq66l2nhlrjc9@conan>
	<g316es$687$1@ger.gmane.org> <op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan> <4854614B.9070903@canterbury.ac.nz>
	<op.ucr0100rhlrjc9@conan>
	<e8a0972d0806151612q1382371dg2ebc8c8913a13ef8@mail.gmail.com>
	<op.uczb25jahlrjc9@conan>
Message-ID: <e8a0972d0806182255r22eabe50j975141db8d421ce4@mail.gmail.com>

On Wed, Jun 18, 2008 at 9:58 PM, Cesare Di Mauro <cesare at pronto.it> wrote:
> Very very, interesting. Thanks. :)
>
> Somebody thinks that Python is unsuitable to implement a DSL: IMO your example prove the contrary. :D

As long as you're willing to do the "DSL" within the strictures of
Python syntax, it's OK - not quite as flexible as LISP or Scheme or
even Ruby, but better than most;-).  I did design and implement "DSL"s
in Python (mostly specialized class trees with suitable metaclasses,
decorators &c) in many jobs in my recent past, I'll admit -- it never
feels as free-flowing as Scheme did back when I used THAT, but, close
enough to make my jobs successful!-)


Alex

From tnelson at onresolve.com  Thu Jun 19 08:02:46 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Thu, 19 Jun 2008 07:02:46 +0100
Subject: [Python-Dev] test_multiprocessing: test_listener_client
 flakiness
In-Reply-To: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
References: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com>
	<e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
Message-ID: <6167796BFEB5D0438720AC212E89A6B00787275D@exchange.onresolve.com>

> I just found the cause of the problem ten minutes ago:
> It seems that when a socket listens on the address "127.0.0.1" or
> "localhost", another process cannot connect to it using the machine's
> name (even from the same machine).
> The best seems to listen with the empty address "".

That doesn't seem right.  If that were the case, none of the hundreds of network-oriented tests would work either.  These all bind against ('localhost', 0) by default which seems to Do The Right Thing on all the platforms we have buildbots for.

        Trent.


From tnelson at onresolve.com  Thu Jun 19 08:07:01 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Thu, 19 Jun 2008 07:07:01 +0100
Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness
In-Reply-To: <e27efe130806181436l114498cdg57154d47bb98f488@mail.gmail.com>
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>
	<loom.20080618T174848-255@post.gmane.org>
	<4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>
	<loom.20080618T180853-875@post.gmane.org>
	<4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com>
	<e27efe130806181436l114498cdg57154d47bb98f488@mail.gmail.com>
Message-ID: <6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com>

> Well, on my win2k machine getfqdn('127.0.0.1') returns the actual name
> of the machine. This is the name that was stored in server.address.
> Hence my patch that simply remove the call to getfqdn.

+1 to ditching getfqdn, following patch fixes the issue on my buildbot server:

Index: connection.py
===================================================================
--- connection.py       (revision 64369)
+++ connection.py       (working copy)
@@ -215,10 +215,7 @@
         self._socket = socket.socket(getattr(socket, family))
         self._socket.bind(address)
         self._socket.listen(backlog)
-        address = self._socket.getsockname()
-        if type(address) is tuple:
-            address = (socket.getfqdn(address[0]),) + address[1:]
-        self._address = address
+        self._address = self._socket.getsockname()
         self._family = family
         self._last_accepted = None



        Trent.

From p.f.moore at gmail.com  Thu Jun 19 10:43:25 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 19 Jun 2008 09:43:25 +0100
Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6b1 and 3.0b1
In-Reply-To: <1972109D-735D-4485-82F4-9BC8F2984967@python.org>
References: <1972109D-735D-4485-82F4-9BC8F2984967@python.org>
Message-ID: <79990c6b0806190143n58e51a22ubafd3fe343291dfb@mail.gmail.com>

On 19/06/2008, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On behalf of the Python development team and the Python community, I am
> happy to announce the first beta releases of Python 2.6 and Python 3.0.

Any ETA for Windows builds? The web pages still point to the alphas.
(I'd like to see the Windows builds more closely integrated with the
releases now we're in beta stage...)

Paul.

From barry at python.org  Thu Jun 19 13:53:10 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 19 Jun 2008 07:53:10 -0400
Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6b1 and 3.0b1
In-Reply-To: <79990c6b0806190143n58e51a22ubafd3fe343291dfb@mail.gmail.com>
References: <1972109D-735D-4485-82F4-9BC8F2984967@python.org>
	<79990c6b0806190143n58e51a22ubafd3fe343291dfb@mail.gmail.com>
Message-ID: <5A74EFE6-0915-4EF9-9096-3CE75F32F00E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 19, 2008, at 4:43 AM, Paul Moore wrote:

> On 19/06/2008, Barry Warsaw <barry at python.org> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On behalf of the Python development team and the Python community,  
>> I am
>> happy to announce the first beta releases of Python 2.6 and Python  
>> 3.0.
>
> Any ETA for Windows builds? The web pages still point to the alphas.
> (I'd like to see the Windows builds more closely integrated with the
> releases now we're in beta stage...)

Martin usually fills these in pretty quickly.  I think the current  
situation works fine for the betas but we'll make sure the final  
release (and candidates) are better coordinated.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSFpIpnEjvBPtnXfVAQJyWQP9FSH8Ipg93UDM3nmH3UtN+i61YGsQPd0O
ypHlnz4yHpxeRkJm1zkppHHI0hKMou6JOeUf05QCnPzrAdsG/mkuv5aoBrBt3dDd
UncHLoQOvXEhGrrPzexmHKv3ehxUXPQOzkiWBWVv9e69GYH4e4HcqV6s2Ya2733T
zC/EyOgkyMg=
=5wM5
-----END PGP SIGNATURE-----

From ctb at msu.edu  Thu Jun 19 13:54:51 2008
From: ctb at msu.edu (C. Titus Brown)
Date: Thu, 19 Jun 2008 04:54:51 -0700
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <e8a0972d0806182255r22eabe50j975141db8d421ce4@mail.gmail.com>
References: <op.ucq66l2nhlrjc9@conan> <g316es$687$1@ger.gmane.org>
	<op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan> <4854614B.9070903@canterbury.ac.nz>
	<op.ucr0100rhlrjc9@conan>
	<e8a0972d0806151612q1382371dg2ebc8c8913a13ef8@mail.gmail.com>
	<op.uczb25jahlrjc9@conan>
	<e8a0972d0806182255r22eabe50j975141db8d421ce4@mail.gmail.com>
Message-ID: <20080619115450.GC23599@caltech.edu>

On Wed, Jun 18, 2008 at 10:55:32PM -0700, Alex Martelli wrote:
-> On Wed, Jun 18, 2008 at 9:58 PM, Cesare Di Mauro <cesare at pronto.it> wrote:
-> > Very very, interesting. Thanks. :)
-> >
-> > Somebody thinks that Python is unsuitable to implement a DSL: IMO your example prove the contrary. :D
-> 
-> As long as you're willing to do the "DSL" within the strictures of
-> Python syntax, it's OK - not quite as flexible as LISP or Scheme or
-> even Ruby, but better than most;-).  I did design and implement "DSL"s
-> in Python (mostly specialized class trees with suitable metaclasses,
-> decorators &c) in many jobs in my recent past, I'll admit -- it never
-> feels as free-flowing as Scheme did back when I used THAT, but, close
-> enough to make my jobs successful!-)

It's pretty easy to put a simple syntax remapper around Python, too;
this is what twill does.  So, for example, this:

--
setlocal username <your username>
setlocal password <your password>

go http://www.slashdot.org/
formvalue 1 unickname $username
formvalue 1 upasswd $password
submit

code 200     # make sure form submission is correct!
--

translates very directly to

---
setlocal('username', '...')
setlocal('password', '...')

go('http://www.slashdot.org/')
formvalue('1', 'unickname', username)
formvalue('1', 'upasswd', password)
submit()

code('200')
---

which is DSL-ish enough for me...

More generally, I've never understood why some people insist that
certain features make Ruby better for DSLs -- are code blocks really
that important to DSLs?  Or is it just the lack of parens??

--titus
-- 
C. Titus Brown, ctb at msu.edu

From jnoller at gmail.com  Thu Jun 19 14:57:17 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 19 Jun 2008 08:57:17 -0400
Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness
In-Reply-To: <6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com>
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>
	<loom.20080618T174848-255@post.gmane.org>
	<4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>
	<loom.20080618T180853-875@post.gmane.org>
	<4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com>
	<e27efe130806181436l114498cdg57154d47bb98f488@mail.gmail.com>
	<6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com>
Message-ID: <4222a8490806190557s66a519ccj7b7bd80e3db68c9b@mail.gmail.com>

Thanks Trent, I'll apply the diff and run the tests in a tight loop
after re-enabling the client listener tests. I appreciate you tracking
this down

On Thu, Jun 19, 2008 at 2:07 AM, Trent Nelson <tnelson at onresolve.com> wrote:
>> Well, on my win2k machine getfqdn('127.0.0.1') returns the actual name
>> of the machine. This is the name that was stored in server.address.
>> Hence my patch that simply remove the call to getfqdn.
>
> +1 to ditching getfqdn, following patch fixes the issue on my buildbot server:
>
> Index: connection.py
> ===================================================================
> --- connection.py       (revision 64369)
> +++ connection.py       (working copy)
> @@ -215,10 +215,7 @@
>         self._socket = socket.socket(getattr(socket, family))
>         self._socket.bind(address)
>         self._socket.listen(backlog)
> -        address = self._socket.getsockname()
> -        if type(address) is tuple:
> -            address = (socket.getfqdn(address[0]),) + address[1:]
> -        self._address = address
> +        self._address = self._socket.getsockname()
>         self._family = family
>         self._last_accepted = None
>
>
>
>        Trent.
>

From janssen at parc.com  Thu Jun 19 19:06:22 2008
From: janssen at parc.com (Bill Janssen)
Date: Thu, 19 Jun 2008 10:06:22 PDT
Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness
In-Reply-To: <6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com>
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>
	<loom.20080618T174848-255@post.gmane.org>
	<4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>
	<loom.20080618T180853-875@post.gmane.org>
	<4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com>
	<e27efe130806181436l114498cdg57154d47bb98f488@mail.gmail.com>
	<6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com>
Message-ID: <08Jun19.100622pdt."58698"@synergy1.parc.xerox.com>

This is a common problem.  Binding to '127.0.0.1' will bind to *only*
that address; binding to "" will bind to *all* addresses the machine
is known by.  If the client uses a different way of getting the
address than the literal '127.0.0.1' (like a call to getfqdn(), which
has pretty indeterminate results), you should use the "" form when
calling bind.

If you know you want to talk to '127.0.0.1', just use the tuple
('127.0.0.1', <PORT>) as the address.

Bill


> > Well, on my win2k machine getfqdn('127.0.0.1') returns the actual name
> > of the machine. This is the name that was stored in server.address.
> > Hence my patch that simply remove the call to getfqdn.
> 
> +1 to ditching getfqdn, following patch fixes the issue on my buildbot server:
> 
> Index: connection.py
> ===================================================================
> --- connection.py       (revision 64369)
> +++ connection.py       (working copy)
> @@ -215,10 +215,7 @@
>          self._socket = socket.socket(getattr(socket, family))
>          self._socket.bind(address)
>          self._socket.listen(backlog)
> -        address = self._socket.getsockname()
> -        if type(address) is tuple:
> -            address = (socket.getfqdn(address[0]),) + address[1:]
> -        self._address = address
> +        self._address = self._socket.getsockname()
>          self._family = family
>          self._last_accepted = None
> 
> 
> 
>         Trent.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com


From curt at hagenlocher.org  Thu Jun 19 19:19:34 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Thu, 19 Jun 2008 10:19:34 -0700
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
	statement?
In-Reply-To: <20080619115450.GC23599@caltech.edu>
References: <op.ucq66l2nhlrjc9@conan> <op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan> <4854614B.9070903@canterbury.ac.nz>
	<op.ucr0100rhlrjc9@conan>
	<e8a0972d0806151612q1382371dg2ebc8c8913a13ef8@mail.gmail.com>
	<op.uczb25jahlrjc9@conan>
	<e8a0972d0806182255r22eabe50j975141db8d421ce4@mail.gmail.com>
	<20080619115450.GC23599@caltech.edu>
Message-ID: <d2155e360806191019w6e668844j99dfc3f9b7af047d@mail.gmail.com>

On Thu, Jun 19, 2008 at 4:54 AM, C. Titus Brown <ctb at msu.edu> wrote:
>
> More generally, I've never understood why some people insist that
> certain features make Ruby better for DSLs -- are code blocks really
> that important to DSLs?  Or is it just the lack of parens??

Code blocks let Ruby-based DSLs "do" flow control, while the lack of
parens make ordinary method names look like keywords.  These things
are a mixed bag -- on the one hand, they can certainly make the
resulting DSL a lot easier to read.  On the other, they seem to
encourage a lot of over-the-top cleverness in terms of creating
"fluent interfaces".  I fail to see much value in being able to write
code that says "7.seconds.ago".

--
Curt Hagenlocher
curt at hagenlocher.org

From pje at telecommunity.com  Thu Jun 19 19:21:23 2008
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 19 Jun 2008 13:21:23 -0400
Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with"
 statement?
In-Reply-To: <20080619115450.GC23599@caltech.edu>
References: <op.ucq66l2nhlrjc9@conan> <g316es$687$1@ger.gmane.org>
	<op.ucq8r6s8hlrjc9@conan>
	<ac2200130806141309r181dfd27xe2f4fd0d2e84aa20@mail.gmail.com>
	<op.ucq9fbiuhlrjc9@conan> <4854614B.9070903@canterbury.ac.nz>
	<op.ucr0100rhlrjc9@conan>
	<e8a0972d0806151612q1382371dg2ebc8c8913a13ef8@mail.gmail.com>
	<op.uczb25jahlrjc9@conan>
	<e8a0972d0806182255r22eabe50j975141db8d421ce4@mail.gmail.com>
	<20080619115450.GC23599@caltech.edu>
Message-ID: <20080619172047.DE1143A4112@sparrow.telecommunity.com>

At 04:54 AM 6/19/2008 -0700, C. Titus Brown wrote:
>More generally, I've never understood why some people insist that
>certain features make Ruby better for DSLs -- are code blocks really
>that important to DSLs?  Or is it just the lack of parens??

Comparison to JavaScript suggests that it's the blocks that make the 
difference.  Even the fact that you have to spell your blocks with 
"function(){...}" doesn't stop you from making a usable DSL, it's 
just not always as pretty as you'd like.

The lack of parens and simpler syntax for blocks in Ruby just makes 
it easier to make *nice-looking* DSLs.


From fijall at gmail.com  Thu Jun 19 20:54:26 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 19 Jun 2008 20:54:26 +0200
Subject: [Python-Dev] Opcode frequency
In-Reply-To: <B556EA862D604758944E9DF7B83FB864@RaymondLaptop1>
References: <g3beok$p0f$1@ger.gmane.org>
	<B556EA862D604758944E9DF7B83FB864@RaymondLaptop1>
Message-ID: <693bc9ab0806191154s60c84c43pfb7e18003b827501@mail.gmail.com>

On Thu, Jun 19, 2008 at 1:03 AM, Raymond Hettinger <python at rcn.com> wrote:
>> Maciej Fijalkowski did an opcode analysis for PyPy,
>> it also shows the relative frequency of opcodes following a
>> specifc one:
>>
>> http://codespeak.net/svn/user/fijal/opcodes.txt
>>
>> Might it make sense to add more PREDICT()ions based
>> on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR?
>
> This particular one might be okay.  What code generates it?
> Are there other possible successors to BUILD_SLICE?
> What code generates those?
>
> There were lots of other opcode pairings that were previously
> studied and rejected.  You're going over well traveled ground.
>
> Also, opcode analysis is a somewhat hazardous undertaking.
> Dynamic analysis takes into account which codes tend to
> occur more often inside loops but it is *very* sensitive
> to who wrote the app and their coding style.  These results
> always have to be taken with a grain of salt.
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
>

I think I should mention the code it comes from - it's pypy's
translation toolchain and no, it's not trying to provide any data on
"standard" python usage. For example CALL_LIKELY_BUILTIN comes from
isinstance calls, which is not something completely natural to be
called that often.

It of course depends on coding style, this is our coding style (ie pypy team).

From amk at amk.ca  Thu Jun 19 22:30:30 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 19 Jun 2008 16:30:30 -0400
Subject: [Python-Dev] [Python-checkins] r64407
	-	python/trunk/Doc/library/multiprocessing.rst
In-Reply-To: <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com>
References: <20080619194843.0A12B1E4002@bag.python.org>
	<1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com>
Message-ID: <20080619203030.GA10666@amk-desktop.matrixgroup.net>

On Thu, Jun 19, 2008 at 03:04:11PM -0500, Benjamin Peterson wrote:
> I don't think the whole introduction had to go. I think it helped give
> some idea of how multiprocessing works before jumping straight to the
> API reference.

I don't think overview material like that should be buried inside the
documentation for one particular module; it should go in a chapter
introduction.  

My plan is to move multiprocessing from the 'Optional OS Services'
chapter to the 'Interprocess Communication' chapter, and then expand
the introduction of the chapter to discuss the GIL and related issues.
Presumably it was put in the 'Optional OS Services' chapter because
that's where the threading and thread modules are.

--amk

From musiccomposition at gmail.com  Thu Jun 19 22:32:18 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 19 Jun 2008 15:32:18 -0500
Subject: [Python-Dev] [Python-checkins] r64407 -
	python/trunk/Doc/library/multiprocessing.rst
In-Reply-To: <20080619203030.GA10666@amk-desktop.matrixgroup.net>
References: <20080619194843.0A12B1E4002@bag.python.org>
	<1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com>
	<20080619203030.GA10666@amk-desktop.matrixgroup.net>
Message-ID: <1afaf6160806191332mcfa1a50v72f537e11e8369eb@mail.gmail.com>

On Thu, Jun 19, 2008 at 3:30 PM, A.M. Kuchling <amk at amk.ca> wrote:
> My plan is to move multiprocessing from the 'Optional OS Services'
> chapter to the 'Interprocess Communication' chapter, and then expand
> the introduction of the chapter to discuss the GIL and related issues.
> Presumably it was put in the 'Optional OS Services' chapter because
> that's where the threading and thread modules are.

Ok. Fair enough. Thanks for explaining.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From jnoller at gmail.com  Thu Jun 19 23:16:38 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 19 Jun 2008 17:16:38 -0400
Subject: [Python-Dev] [Python-checkins] r64407 -
	python/trunk/Doc/library/multiprocessing.rst
In-Reply-To: <20080619203030.GA10666@amk-desktop.matrixgroup.net>
References: <20080619194843.0A12B1E4002@bag.python.org>
	<1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com>
	<20080619203030.GA10666@amk-desktop.matrixgroup.net>
Message-ID: <4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com>

Where would that chapter end up (source-wise) I think a few of us
might have additional things to add ;)

On Thu, Jun 19, 2008 at 4:30 PM, A.M. Kuchling <amk at amk.ca> wrote:
> On Thu, Jun 19, 2008 at 03:04:11PM -0500, Benjamin Peterson wrote:
>> I don't think the whole introduction had to go. I think it helped give
>> some idea of how multiprocessing works before jumping straight to the
>> API reference.
>
> I don't think overview material like that should be buried inside the
> documentation for one particular module; it should go in a chapter
> introduction.
>
> My plan is to move multiprocessing from the 'Optional OS Services'
> chapter to the 'Interprocess Communication' chapter, and then expand
> the introduction of the chapter to discuss the GIL and related issues.
> Presumably it was put in the 'Optional OS Services' chapter because
> that's where the threading and thread modules are.
>
> --amk
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com
>

From alexandre at peadrop.com  Thu Jun 19 23:23:12 2008
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Thu, 19 Jun 2008 17:23:12 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
Message-ID: <acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>

On Sun, Jun 1, 2008 at 12:28 AM, Adam Olsen <rhamph at gmail.com> wrote:
> On Sat, May 31, 2008 at 10:11 PM, Alexandre Vassalotti
> <alexandre at peadrop.com> wrote:
>> Would anyone mind if I did add a public C API for gc.disable() and
>> gc.enable()? I would like to use it as an optimization for the pickle
>> module (I found out that I get a good 2x speedup just by disabling the
>> GC while loading large pickles). Of course, I could simply import the
>> gc module and call the functions there, but that seems overkill to me.
>> I included the patch below for review.
>
> I'd rather see it fixed.  It behaves quadratically if you load enough
> to trigger full collection a few times.
>

Do you have any idea how this behavior could be fixed? I am not a GC
expert, but I could try to fix this.

-- Alexandre

From rhamph at gmail.com  Thu Jun 19 23:35:55 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 19 Jun 2008 15:35:55 -0600
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
Message-ID: <aac2c7cb0806191435xf95cb1eicc4bcbf5f7e63f9a@mail.gmail.com>

On Thu, Jun 19, 2008 at 3:23 PM, Alexandre Vassalotti
<alexandre at peadrop.com> wrote:
> On Sun, Jun 1, 2008 at 12:28 AM, Adam Olsen <rhamph at gmail.com> wrote:
>> On Sat, May 31, 2008 at 10:11 PM, Alexandre Vassalotti
>> <alexandre at peadrop.com> wrote:
>>> Would anyone mind if I did add a public C API for gc.disable() and
>>> gc.enable()? I would like to use it as an optimization for the pickle
>>> module (I found out that I get a good 2x speedup just by disabling the
>>> GC while loading large pickles). Of course, I could simply import the
>>> gc module and call the functions there, but that seems overkill to me.
>>> I included the patch below for review.
>>
>> I'd rather see it fixed.  It behaves quadratically if you load enough
>> to trigger full collection a few times.
>>
>
> Do you have any idea how this behavior could be fixed? I am not a GC
> expert, but I could try to fix this.

Not sure.  For long-running programs we actually want a quadratic
cost, but spread out over a long period of time.  It's the bursts of
allocation that shouldn't be quadratic.

So maybe balance it by the total number of allocations that have
happened.  If the total number of allocations is less than 10 times
the allocations that weren't promptly deleted, assume it's a burst.
If it's more than 10 times, assume it's over a long period.


-- 
Adam Olsen, aka Rhamphoryncus

From dbpokorny at gmail.com  Fri Jun 20 00:59:13 2008
From: dbpokorny at gmail.com (David Pokorny)
Date: Thu, 19 Jun 2008 15:59:13 -0700
Subject: [Python-Dev] Status of Issue 2331 - Backport parameter annotations
Message-ID: <cad407ed0806191559s2ec37894mbae60bc6c775e7c0@mail.gmail.com>

Hi,

I am curious if there is any news on this issue. My understanding is
that since this is a new feature, nothing will be committed until
after 2.6 comes out, but it would be *really nice* if I could use
annotations in 2.x.

[warning - mini rant]
Reason is, I am using 3.0a for now for this project, but I am sorry to
say that using parentheses for raising exceptions drives me up the
wall, and I don't like using them for the former print statement
either. I suspect this is partly due to the fact that I'm using 2.5
for work, so I don't have the option to mentally "switch over" to the
Python 3 way.
/rant

Does anyone have an annotations patch floating around against the
trunk / 2.6b, or failing that, does anyone have opinions on what my
chances are / things to look out for if I fiddle around with the
annotations patch from the py3k branch? (My understanding is that this
is revision 53170).

Cheers,
David

From guido at python.org  Fri Jun 20 01:16:12 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 19 Jun 2008 16:16:12 -0700
Subject: [Python-Dev] Status of Issue 2331 - Backport parameter
	annotations
In-Reply-To: <cad407ed0806191559s2ec37894mbae60bc6c775e7c0@mail.gmail.com>
References: <cad407ed0806191559s2ec37894mbae60bc6c775e7c0@mail.gmail.com>
Message-ID: <ca471dc20806191616v2baa73ccn9b719bddbe5846cc@mail.gmail.com>

On Thu, Jun 19, 2008 at 3:59 PM, David Pokorny <dbpokorny at gmail.com> wrote:
> I am curious if there is any news on this issue. My understanding is
> that since this is a new feature, nothing will be committed until
> after 2.6 comes out, but it would be *really nice* if I could use
> annotations in 2.x.
>
> [warning - mini rant]
> Reason is, I am using 3.0a for now for this project, but I am sorry to
> say that using parentheses for raising exceptions drives me up the
> wall, and I don't like using them for the former print statement
> either. I suspect this is partly due to the fact that I'm using 2.5
> for work, so I don't have the option to mentally "switch over" to the
> Python 3 way.
> /rant
>
> Does anyone have an annotations patch floating around against the
> trunk / 2.6b, or failing that, does anyone have opinions on what my
> chances are / things to look out for if I fiddle around with the
> annotations patch from the py3k branch? (My understanding is that this
> is revision 53170).

I give this approximately 0% chance of happening. It's a really
complex change to the syntax and code generator. Your motivation is
also suspect: 2.6 is supposed to be a stepping stone towards 3.0, not
a safe haven for people who don't like certain 3.0 features. Just get
used to print(). Parentheses for exceptions has been allowed since
approximately Python 2.0, so you have even less of an excuse there.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Fri Jun 20 02:36:59 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 20 Jun 2008 12:36:59 +1200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
Message-ID: <485AFBAB.2060404@canterbury.ac.nz>

Alexandre Vassalotti wrote:

> Do you have any idea how this behavior could be fixed? I am not a GC
> expert, but I could try to fix this.

Perhaps after making a GC pass you could look at the
number of objects reclaimed during that pass, and if
it's less than some fraction of the objects in existence,
increase the threshold for performing GC by some
factor.

You would also want to do the opposite, so that a
GC pass which reclaims a large proportion of objects
would reduce the threshold back down again.

-- 
Greg

From dbpokorny at gmail.com  Fri Jun 20 03:05:27 2008
From: dbpokorny at gmail.com (dbpokorny at gmail.com)
Date: Thu, 19 Jun 2008 18:05:27 -0700 (PDT)
Subject: [Python-Dev] Status of Issue 2331 - Backport parameter
	annotations
In-Reply-To: <ca471dc20806191616v2baa73ccn9b719bddbe5846cc@mail.gmail.com>
References: <cad407ed0806191559s2ec37894mbae60bc6c775e7c0@mail.gmail.com> 
	<ca471dc20806191616v2baa73ccn9b719bddbe5846cc@mail.gmail.com>
Message-ID: <f1161fd1-cb61-4a0b-bcdd-6610563c8704@y22g2000prd.googlegroups.com>

On Jun 19, 4:16 pm, "Guido van Rossum" <gu... at python.org> wrote:
> complex change to the syntax and code generator. Your motivation is
> also suspect: 2.6 is supposed to be a stepping stone towards 3.0, not
> a safe haven for people who don't like certain 3.0 features. Just get

Guilty as charged :) I tried out 3.0, decided I didn't like certain
features I found annoying, and now I want to get the features (well
only one feature right now) I like into 2.x so I can go back...I have
some energy to do this, and I'm willing to learn, and I asked this
list because I hoped someone would take pity on me and lend some help.
What is my sentence, exile to outer PHP-iberia?

At any rate, I am still interested if anyone has a working patch for
this against the trunk, or any pointers for adapting 53170, words of
experience when changing the grammar, additions to PEP 306, etc... any
help would be greatly appreciated!

Cheers,
David

From tjreedy at udel.edu  Fri Jun 20 04:11:48 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 19 Jun 2008 22:11:48 -0400
Subject: [Python-Dev] Status of Issue 2331 - Backport parameter
	annotations
In-Reply-To: <cad407ed0806191559s2ec37894mbae60bc6c775e7c0@mail.gmail.com>
References: <cad407ed0806191559s2ec37894mbae60bc6c775e7c0@mail.gmail.com>
Message-ID: <g3f3kv$25r$1@ger.gmane.org>



David Pokorny wrote:

> [warning - mini rant]
> Reason is, I am using 3.0a for now for this project, but I am sorry to
> say that using parentheses for raising exceptions drives me up the
> wall,

[warning - a bit of a counter-rant ;-]
Does it bother you that you need ()s to make instances elsewhere?
That you have to type int('123') instead of int, '123'?
The exception exception is an anomaly that would be gone in 2.6 (or 
possibly not until 2.7), if it had not been put off, with other 
removals, to 3.0.
[/rant]

 > and I don't like using them for the former print statement either.

That I sympathize with, but I prefer file = xxx to >>xxx and otherwise 
consider it an acceptible price for consistency with input() and getting 
rid of lot of things I do not like, from old-style classes to the 
exception anomaly ;-)

> I suspect this is partly due to the fact that I'm using 2.5
> for work, so I don't have the option to mentally "switch over" to the
> Python 3 way.

Whereas I do, which, I admit, makes it easier.

> /rant


From amk at amk.ca  Fri Jun 20 04:16:34 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 19 Jun 2008 22:16:34 -0400
Subject: [Python-Dev] sum() in math module not duplicated?
Message-ID: <20080620021634.GA3480@amk.local>

In the comments before the implementation of sum()
in mathmodule.c, note 4 says:

   Note 4: A similar implementation is in Modules/cmathmodule.c.
   Be sure to update both when making changes.

I can't find a sum implementation or similar code in cmathmodule.c.
Did someone intend to port the sum function to cmath, or was it copied
and then taken out?  I'm wondering if this note should simply be
removed.

--amk

From python at rcn.com  Fri Jun 20 04:41:55 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 19 Jun 2008 19:41:55 -0700
Subject: [Python-Dev] sum() in math module not duplicated?
References: <20080620021634.GA3480@amk.local>
Message-ID: <BE943D8D02254C70A804B8C382E877A6@RaymondLaptop1>

We're still working on the implementation details for math.sum().
When it's finished, the cmath equilvalent will be added.

Raymond

----- Original Message ----- 
From: "A.M. Kuchling" <amk at amk.ca>
To: <python-dev at python.org>
Sent: Thursday, June 19, 2008 7:16 PM
Subject: [Python-Dev] sum() in math module not duplicated?


> In the comments before the implementation of sum()
> in mathmodule.c, note 4 says:
> 
>   Note 4: A similar implementation is in Modules/cmathmodule.c.
>   Be sure to update both when making changes.
> 
> I can't find a sum implementation or similar code in cmathmodule.c.
> Did someone intend to port the sum function to cmath, or was it copied
> and then taken out?  I'm wondering if this note should simply be
> removed.
> 
> --amk
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com

From tony at PageDNA.com  Fri Jun 20 04:38:52 2008
From: tony at PageDNA.com (Tony Lownds)
Date: Thu, 19 Jun 2008 19:38:52 -0700
Subject: [Python-Dev] Status of Issue 2331 - Backport parameter
	annotations
In-Reply-To: <f1161fd1-cb61-4a0b-bcdd-6610563c8704@y22g2000prd.googlegroups.com>
References: <cad407ed0806191559s2ec37894mbae60bc6c775e7c0@mail.gmail.com>
	<ca471dc20806191616v2baa73ccn9b719bddbe5846cc@mail.gmail.com>
	<f1161fd1-cb61-4a0b-bcdd-6610563c8704@y22g2000prd.googlegroups.com>
Message-ID: <A6B48427-3D8C-4CBC-B47C-DAF7511F348C@PageDNA.com>

> At any rate, I am still interested if anyone has a working patch for
> this against the trunk, or any pointers for adapting 53170, words of
> experience when changing the grammar, additions to PEP 306, etc... any
> help would be greatly appreciated!

David,

I can help. I don't have a patch against the trunk but my first  
revisions of the patch
for annotations did handle things like tuple parameters which are  
relevant to 2.6.

-Tony

From tnelson at onresolve.com  Fri Jun 20 07:02:02 2008
From: tnelson at onresolve.com (Trent Nelson)
Date: Fri, 20 Jun 2008 06:02:02 +0100
Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness
In-Reply-To: <08Jun19.100622pdt."58698"@synergy1.parc.xerox.com>
References: <e27efe130806180935s6e59007bh1e1c9fed1b1661a2@mail.gmail.com>
	<20080618164136.4714.382689990.divmod.quotient.10414@ohm>
	<e27efe130806180948r4a44a025w9124a587c7b46e39@mail.gmail.com>
	<loom.20080618T174848-255@post.gmane.org>
	<4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com>
	<loom.20080618T180853-875@post.gmane.org>
	<4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com>
	<e27efe130806181436l114498cdg57154d47bb98f488@mail.gmail.com>
	<6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com>
	<08Jun19.100622pdt."58698"@synergy1.parc.xerox.com>
Message-ID: <6167796BFEB5D0438720AC212E89A6B007872765@exchange.onresolve.com>

> This is a common problem.  Binding to '127.0.0.1' will bind to *only*
> that address;

Indeed.

> binding to "" will bind to *all* addresses the machine
> is known by.

Agreed again.  I believe what we're dealing with here though is a lack of clarity regarding what role the 'address' attribute exposed by multiprocess.connection.Listener should play.  The way test_listener_client() is written, it effectively treats 'address' as an end-point that can be connected to directly (irrespective of the underlying family (i.e. AF_INET, AF_UNIX, AF_PIPE)).

I believe the problems we've run into stem from the fact that the API doesn't provide any guarantees as to what 'address' represents.  The test suite assumes it always reflects a connectable end-point, which I think is more than reasonable.  Unfortunately, nothing stops us from breaking this invariant by constructing the object as Listener(family='AF_INET', address=('0.0.0.0', 0)).

How do I connect to an AF_INET Listener (i.e. SocketListener) instance whose 'address' attribute reports '0.0.0.0' as the host?  I can't.

So, for now, I think we should enforce this invariant by raising an exception in Listener.__init__() if self._socket.getsockbyname()[0] returns '0.0.0.0'.  In effect, tightening up the API such that we can guarantee  Listener.address will always represent a connectable end-point.  We can look at how to service 'listen on all available interfaces' semantics at a later date -- that adds far less value IMO than being able to depend on the said guarantee.

Thoughts?

        Trent.

From amk at amk.ca  Fri Jun 20 15:09:49 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 20 Jun 2008 09:09:49 -0400
Subject: [Python-Dev] [Python-checkins] r64407
	-	python/trunk/Doc/library/multiprocessing.rst
In-Reply-To: <4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com>
References: <20080619194843.0A12B1E4002@bag.python.org>
	<1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com>
	<20080619203030.GA10666@amk-desktop.matrixgroup.net>
	<4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com>
Message-ID: <20080620130949.GA6641@amk-desktop.matrixgroup.net>

On Thu, Jun 19, 2008 at 05:16:38PM -0400, Jesse Noller wrote:
> Where would that chapter end up (source-wise) I think a few of us
> might have additional things to add ;)

This would be Doc/library/ipc.rst.  The chapter is 'Interprocess
Communication and Networking'.

Is anyone else finding it increasingly odd that subprocess, signal,
socket/ssl, and syncore are in the same chapter?  I'm tempted to move
socket, ssl, asyncore+asynchat into a 'networking' chapter, and then
also move SocketServer from the 'Internet Protocols' chapter into this
new chapter.

--amk

From bioinformed at gmail.com  Fri Jun 20 16:15:49 2008
From: bioinformed at gmail.com (Kevin Jacobs <jacobs@bioinformed.com>)
Date: Fri, 20 Jun 2008 10:15:49 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <485AFBAB.2060404@canterbury.ac.nz>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
Message-ID: <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>

+1 on a C API for enabling and disabling GC.

I have several instances where I create a large number of objects non-cyclic
objects where I see huge GC overhead (30+ seconds with gc enabled, 0.15
seconds when disabled).

+1000 to fixing the garbage collector to be smart enough to self-regulate
itself better.

In the mean time, I use the following context manager to deal with the
hotspots as I find them:

class gcdisabled(object):
  '''
  Conext manager to temporarily disable Python's cyclic garbage collector.
  The primary use is to avoid thrashing while allocating large numbers of
  non-cyclic objects due to an overly aggressive garbage collector behavior.

  Will disable GC if it is enabled upon entry and renable upon exit:

  >>> gc.isenabled()
  True
  >>> with gcdisabled():
  ...   print gc.isenabled()
  False
  >>> print gc.isenabled()
  True

  Will not reenable if GC was disabled upon entry:

  >>> gc.disable()
  >>> gc.isenabled()
  False
  >>> with gcdisabled():
  ...   gc.isenabled()
  False
  >>> gc.isenabled()
  False
  '''
  def __init__(self):
    self.isenabled = gc.isenabled()

  def __enter__(self):
    gc.disable()

  def __exit__(self, type, value, traceback):
    if self.isenabled:
      gc.enable()
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080620/3c7b7a48/attachment.htm>

From solipsis at pitrou.net  Fri Jun 20 16:25:01 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 20 Jun 2008 14:25:01 +0000 (UTC)
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
Message-ID: <loom.20080620T142315-395@post.gmane.org>


Hi,

Kevin Jacobs <jacobs <at> bioinformed.com> <bioinformed <at> gmail.com> writes:
> 
> +1 on a C API for enabling and disabling GC.? I have several instances where 
I create a large number of objects non-cyclic objects where I see huge GC 
overhead (30+ seconds with gc enabled, 0.15 seconds when disabled).

Could you try to post a stripped-down, self-contained example of such behaviour?

Antoine.



From bioinformed at gmail.com  Fri Jun 20 17:04:00 2008
From: bioinformed at gmail.com (Kevin Jacobs <jacobs@bioinformed.com>)
Date: Fri, 20 Jun 2008 11:04:00 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <loom.20080620T142315-395@post.gmane.org>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
Message-ID: <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>

On Fri, Jun 20, 2008 at 10:25 AM, Antoine Pitrou <solipsis at pitrou.net>
wrote:

>
> Kevin Jacobs <jacobs <at> bioinformed.com> <bioinformed <at> gmail.com>
> writes:
> >
> > +1 on a C API for enabling and disabling GC.  I have several instances
> where
> I create a large number of objects non-cyclic objects where I see huge GC
> overhead (30+ seconds with gc enabled, 0.15 seconds when disabled).
>
> Could you try to post a stripped-down, self-contained example of such
> behaviour?



$ python -m timeit 'zip(*[range(1000000)]*5)'
10 loops, best of 3: 496 msec per loop

$ python -m timeit -s 'import gc; gc.enable()' 'zip(*[range(1000000)]*5)'
10 loops, best of 3: 2.93 sec per loop

Note that timeit cheats and disables GC by default.

Attached is a less stripped down script to demonstrate the super-linear
behavior for somewhat naively coded transpose operators.  The output is
listed below:

FUNCTION        ROWS         COLUMNS     GC ENABLED    GC DISABLED
--------------  -----------  ----------  ------------  ------------
transpose_comp            5           0        0.0000        0.0000
transpose_comp            5      250000        0.5535        0.3537
transpose_comp            5      500000        1.4359        0.6868
transpose_comp            5      750000        2.7148        1.0760
transpose_comp            5     1000000        3.8070        1.3936
transpose_comp            5     1250000        5.5184        1.7617
transpose_comp            5     1500000        7.8828        2.1308
transpose_comp            5     1750000        9.3279        2.5364
transpose_comp            5     2000000       11.8248        2.7399
transpose_comp            5     2250000       14.7436        3.1585
transpose_comp            5     2500000       18.4452        3.5818
transpose_comp            5     2750000       21.4856        3.8988
transpose_comp            5     3000000       24.4110        4.3148
transpose_zip             5           0        0.0000        0.0000
transpose_zip             5      250000        0.2537        0.0658
transpose_zip             5      500000        0.8380        0.1324
transpose_zip             5      750000        1.7507        0.1989
transpose_zip             5     1000000        2.6169        0.2648
transpose_zip             5     1250000        4.0760        0.3317
transpose_zip             5     1500000        5.8852        0.4145
transpose_zip             5     1750000        7.3925        0.5161
transpose_zip             5     2000000       10.0755        0.6708
transpose_zip             5     2250000       14.2698        0.7760
transpose_zip             5     2500000       16.7291        0.9022
transpose_zip             5     2750000       20.3833        1.0179
transpose_zip             5     3000000       24.5515        1.0971

Hope this helps,
-Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080620/b6b776b9/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bench_transpose.py
Type: text/x-python-script
Size: 796 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080620/b6b776b9/attachment.bin>

From amauryfa at gmail.com  Fri Jun 20 17:44:40 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 20 Jun 2008 17:44:40 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
Message-ID: <e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>

2008/6/20 Kevin Jacobs <jacobs at bioinformed.com> <bioinformed at gmail.com>:
> On Fri, Jun 20, 2008 at 10:25 AM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
>>
>> Kevin Jacobs <jacobs <at> bioinformed.com> <bioinformed <at> gmail.com>
>> writes:
>> >
>> > +1 on a C API for enabling and disabling GC.  I have several instances
>> > where
>> I create a large number of objects non-cyclic objects where I see huge GC
>> overhead (30+ seconds with gc enabled, 0.15 seconds when disabled).
>>
>> Could you try to post a stripped-down, self-contained example of such
>> behaviour?
>
> $ python -m timeit 'zip(*[range(1000000)]*5)'
> 10 loops, best of 3: 496 msec per loop
>
> $ python -m timeit -s 'import gc; gc.enable()' 'zip(*[range(1000000)]*5)'
> 10 loops, best of 3: 2.93 sec per loop

I remember that a similar issue was discussed some months ago:
http://bugs.python.org/issue2607

In short: the gc is tuned for typical usage. If your usage of python
is specific,
use gc.set_threshold and increase its values.

-- 
Amaury Forgeot d'Arc

From janssen at parc.com  Fri Jun 20 17:55:13 2008
From: janssen at parc.com (Bill Janssen)
Date: Fri, 20 Jun 2008 08:55:13 PDT
Subject: [Python-Dev] [Python-checkins] r64407 -
	python/trunk/Doc/library/multiprocessing.rst
In-Reply-To: <20080620130949.GA6641@amk-desktop.matrixgroup.net> 
References: <20080619194843.0A12B1E4002@bag.python.org>
	<1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com>
	<20080619203030.GA10666@amk-desktop.matrixgroup.net>
	<4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com>
	<20080620130949.GA6641@amk-desktop.matrixgroup.net>
Message-ID: <08Jun20.085516pdt."58698"@synergy1.parc.xerox.com>

> Is anyone else finding it increasingly odd that subprocess, signal,
> socket/ssl, and syncore are in the same chapter?  I'm tempted to move
> socket, ssl, asyncore+asynchat into a 'networking' chapter, and then
> also move SocketServer from the 'Internet Protocols' chapter into this
> new chapter.

Sounds like you mean, 'Non-HTTP Networking'.

Bill

From status at bugs.python.org  Fri Jun 20 18:06:51 2008
From: status at bugs.python.org (Python tracker)
Date: Fri, 20 Jun 2008 18:06:51 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20080620160651.ADE90780F2@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (06/13/08 - 06/20/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1917 open (+28) / 13057 closed (+19) / 14974 total (+47)

Open issues with patches:   581

Average duration of open issues: 712 days.
Median duration of open issues: 1479 days.

Open Issues Breakdown
   open  1892 (+28)
pending    25 ( +0)

Issues Created Or Reopened (50)
_______________________________

Implement __format__ for Decimal                                 06/20/08
       http://bugs.python.org/issue2110    reopened benjamin.peterson         
       patch                                                                   

Create the urllib package                                        06/18/08
       http://bugs.python.org/issue2885    reopened brett.cannon              
                                                                               

Python 2.5.2 Windows Source Distribution missing Visual Studio 2 06/13/08
       http://bugs.python.org/issue3105    created  MarkE                     
                                                                               

speedup some comparisons                                         06/13/08
       http://bugs.python.org/issue3106    created  pitrou                    
       patch                                                                   

memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linu 06/13/08
       http://bugs.python.org/issue3107    created  hushp1pt                  
                                                                               

Implicit exception chaining (PEP 3134)                           06/13/08
CLOSED http://bugs.python.org/issue3108    created  pitrou                    
       patch                                                                   

test_multiprocessing seems fragile                               06/14/08
       http://bugs.python.org/issue3109    created  pitrou                    
                                                                               

Multiprocessing package build problem on Solaris 10              06/14/08
       http://bugs.python.org/issue3110    created  jnoller                   
                                                                               

multiprocessing ppc Debian/ ia64 Ubuntu compilation error        06/14/08
       http://bugs.python.org/issue3111    created  jnoller                   
                                                                               

implement PEP 3134 exception reporting                           06/14/08
       http://bugs.python.org/issue3112    created  benjamin.peterson         
       patch                                                                   

Document exception chaining                                      06/14/08
       http://bugs.python.org/issue3113    created  benjamin.peterson         
                                                                               

bus error on lib2to3                                             06/15/08
CLOSED http://bugs.python.org/issue3114    reopened gvanrossum                
       patch                                                                   

os.listdir randomly fails on occasions when it shouldn't         06/14/08
       http://bugs.python.org/issue3115    created  philipspencer             
       patch                                                                   

Fix quadratic behavior for marshal.dumps() when len>32Mb         06/15/08
CLOSED http://bugs.python.org/issue3116    created  rhettinger                
       patch, patch                                                            

segfault with (None,) as argument in a def/lambda                06/15/08
CLOSED http://bugs.python.org/issue3117    created  santoemma                 
                                                                               

test_math fails on 64bit                                         06/16/08
CLOSED http://bugs.python.org/issue3118    created  benjamin.peterson         
       patch, 64bit                                                            

pickle.py is limited by python's call stack                      06/16/08
       http://bugs.python.org/issue3119    created  habnabit                  
       patch                                                                   

subprocess module truncates handles on AMD64                     06/16/08
       http://bugs.python.org/issue3120    created  rupole                    
                                                                               

test_urllibnet fails                                             06/16/08
CLOSED http://bugs.python.org/issue3121    created  cartman                   
                                                                               

sys.getsizeof() gives an AttributeError for _sre objects.        06/16/08
       http://bugs.python.org/issue3122    created  schuppenies               
       patch, patch                                                            

2to3 fails                                                       06/16/08
CLOSED http://bugs.python.org/issue3123    created  djc                       
                                                                               

test_multiprocessing segfaults on Windows                        06/16/08
CLOSED http://bugs.python.org/issue3124    created  theller                   
                                                                               

test_multiprocessing causes test_ctypes to fail                  06/16/08
       http://bugs.python.org/issue3125    created  theller                   
       patch                                                                   

Lib/logging/__init__.py assumes its stream has flush and close m 06/16/08
CLOSED http://bugs.python.org/issue3126    created  bobf                      
       patch                                                                   

Segfault provoked by generators and exceptions                   06/17/08
CLOSED http://bugs.python.org/issue3127    created  amajorek                  
                                                                               

Regex causes python to hang up? / loop infinite?                 06/17/08
CLOSED http://bugs.python.org/issue3128    created  computercrustie           
                                                                               

struct allows repeat spec. without a format specifier            06/17/08
       http://bugs.python.org/issue3129    created  carrus85                  
                                                                               

In some UCS4 builds, sizeof(Py_UNICODE) could end up being more  06/17/08
       http://bugs.python.org/issue3130    created  schuppenies               
                                                                               

2to3 can't find fixes_dir                                        06/17/08
       http://bugs.python.org/issue3131    created  bhy                       
                                                                               

implement PEP 3118 struct changes                                06/17/08
       http://bugs.python.org/issue3132    created  benjamin.peterson         
                                                                               

CGIHTTPRequestHandler does not work on windows                   06/18/08
CLOSED http://bugs.python.org/issue3133    created  amaury.forgeotdarc        
       patch                                                                   

shutil references undefined WindowsError symbol                  06/18/08
       http://bugs.python.org/issue3134    created  dvitek                    
       patch                                                                   

inspect.getcallargs()                                            06/18/08
       http://bugs.python.org/issue3135    created  gsakkis                   
                                                                               

[PATCH] logging.config.fileConfig() compulsivly disable all exis 06/19/08
CLOSED http://bugs.python.org/issue3136    created  llucax                    
       patch                                                                   

Python doesn't handle SIGINT well if it arrives during interpret 06/19/08
       http://bugs.python.org/issue3137    created  gjb1002                   
                                                                               

Hang when calling get() on an empty queue in the queue module    06/19/08
CLOSED http://bugs.python.org/issue3138    created  slash2314                 
                                                                               

print is not thread safe                                         06/19/08
       http://bugs.python.org/issue3139    created  amaury.forgeotdarc        
       patch                                                                   

str.format("{0:n}") poss. bug with setlocale()                   06/19/08
       http://bugs.python.org/issue3140    created  mark                      
                                                                               

Linux build requires Mac module _gestalt                         06/19/08
CLOSED http://bugs.python.org/issue3141    created  mark                      
                                                                               

urllib docs don't match the new modules                          06/19/08
       http://bugs.python.org/issue3142    created  mark                      
                                                                               

development docs waste a lot of horizontal space on left nav bar 06/19/08
       http://bugs.python.org/issue3143    created  forest                    
                                                                               

popen / popen[234] inconsistent fd behavior                      06/19/08
CLOSED http://bugs.python.org/issue3144    created  justincappos              
                                                                               

help> modules os raises UnicodeDecodeError                       06/19/08
CLOSED http://bugs.python.org/issue3145    created  msyang                    
                                                                               

Sphinx/LaTeX fails on Python 3.0b1 documentation                 06/19/08
       http://bugs.python.org/issue3146    created  vmanis1                   
                                                                               

tests for sys.getsizeof fail on win64                            06/19/08
       http://bugs.python.org/issue3147    created  amaury.forgeotdarc        
                                                                               

multiprocessing test hangs                                       06/20/08
CLOSED http://bugs.python.org/issue3148    created  MrJean1                   
                                                                               

multiprocessing build fails on Solaris 10                        06/20/08
       http://bugs.python.org/issue3149    created  MrJean1                   
                                                                               

multiprocessing module not being installed                       06/20/08
CLOSED http://bugs.python.org/issue3150    created  hdiogenes                 
       patch, easy                                                             

elementtree serialization bug for weird namespace urls           06/20/08
       http://bugs.python.org/issue3151    created  xrotwang                  
                                                                               

SimpleHTTPServer reports wrong content-length for text files     06/13/08
       http://bugs.python.org/issue839496  reopened tjreedy                   
       patch, easy                                                             



Issues Now Closed (34)
______________________

test_socket_ssl hanhs on Windows (deadlock)                       209 days
       http://bugs.python.org/issue1489    amaury.forgeotdarc        
                                                                               

o(n*n) marshal.dumps performance for largish objects with patch   156 days
       http://bugs.python.org/issue1792    rhettinger                
       patch, easy                                                             

Add a factorial function                                           92 days
       http://bugs.python.org/issue2138    marketdickinson           
                                                                               

Make range __eq__ work                                             67 days
       http://bugs.python.org/issue2603    gvanrossum                
       patch                                                                   

range: lean and mean                                               45 days
       http://bugs.python.org/issue2735    gvanrossum                
       patch                                                                   

Add memory footprint query                                         32 days
       http://bugs.python.org/issue2898    schuppenies               
       patch                                                                   

Bug in slice.indices()                                             22 days
       http://bugs.python.org/issue3004    marketdickinson           
       patch                                                                   

3.0a5 tarballs contain the code twice                              20 days
       http://bugs.python.org/issue3009    barry                     
                                                                               

getsizeof incorrect for Unicode strings                             7 days
       http://bugs.python.org/issue3048    schuppenies               
                                                                               

tokenize.py improvements                                            6 days
       http://bugs.python.org/issue3078    rhettinger                
       patch                                                                   

All 3.0 buildbots are red                                           7 days
       http://bugs.python.org/issue3089    barry                     
                                                                               

sys.sizeof test fails with wide unicode                             5 days
       http://bugs.python.org/issue3098    schuppenies               
       patch                                                                   

Implicit exception chaining (PEP 3134)                              1 days
       http://bugs.python.org/issue3108    benjamin.peterson         
       patch                                                                   

bus error on lib2to3                                                1 days
       http://bugs.python.org/issue3114    benjamin.peterson         
       patch                                                                   

Fix quadratic behavior for marshal.dumps() when len>32Mb            1 days
       http://bugs.python.org/issue3116    rhettinger                
       patch, patch                                                            

segfault with (None,) as argument in a def/lambda                   0 days
       http://bugs.python.org/issue3117    georg.brandl              
                                                                               

test_math fails on 64bit                                            2 days
       http://bugs.python.org/issue3118    benjamin.peterson         
       patch, 64bit                                                            

test_urllibnet fails                                                3 days
       http://bugs.python.org/issue3121    amaury.forgeotdarc        
                                                                               

2to3 fails                                                          0 days
       http://bugs.python.org/issue3123    benjamin.peterson         
                                                                               

test_multiprocessing segfaults on Windows                           0 days
       http://bugs.python.org/issue3124    amaury.forgeotdarc        
                                                                               

Lib/logging/__init__.py assumes its stream has flush and close m    1 days
       http://bugs.python.org/issue3126    vsajip                    
       patch                                                                   

Segfault provoked by generators and exceptions                      0 days
       http://bugs.python.org/issue3127    amaury.forgeotdarc        
                                                                               

Regex causes python to hang up? / loop infinite?                    0 days
       http://bugs.python.org/issue3128    computercrustie           
                                                                               

CGIHTTPRequestHandler does not work on windows                      1 days
       http://bugs.python.org/issue3133    amaury.forgeotdarc        
       patch                                                                   

[PATCH] logging.config.fileConfig() compulsivly disable all exis    1 days
       http://bugs.python.org/issue3136    llucax                    
       patch                                                                   

Hang when calling get() on an empty queue in the queue module       0 days
       http://bugs.python.org/issue3138    benjamin.peterson         
                                                                               

Linux build requires Mac module _gestalt                            0 days
       http://bugs.python.org/issue3141    benjamin.peterson         
                                                                               

popen / popen[234] inconsistent fd behavior                         0 days
       http://bugs.python.org/issue3144    amaury.forgeotdarc        
                                                                               

help> modules os raises UnicodeDecodeError                          0 days
       http://bugs.python.org/issue3145    amaury.forgeotdarc        
                                                                               

multiprocessing test hangs                                          0 days
       http://bugs.python.org/issue3148    benjamin.peterson         
                                                                               

multiprocessing module not being installed                          0 days
       http://bugs.python.org/issue3150    benjamin.peterson         
       patch, easy                                                             

Allow PyArg_ParseTupleAndKeywords to unpack tuples                786 days
       http://bugs.python.org/issue1474454 loewis                    
       patch                                                                   

Python 2.5b2 fails to build on Solaris 10 (Sun Compiler)          695 days
       http://bugs.python.org/issue1528620 akuchling                 
                                                                               

Add Tkinter.Checkbutton get() and set(value)                      307 days
       http://bugs.python.org/issue1774370 gpolo                     
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 17 test_multiprocessing causes test_ctypes to fail                    4 days
open    http://bugs.python.org/issue3125   

 17 sys.sizeof test fails with wide unicode                            5 days
closed  http://bugs.python.org/issue3098   

 13 multiprocessing ppc Debian/ ia64 Ubuntu compilation error          6 days
open    http://bugs.python.org/issue3111   

 12 bus error on lib2to3                                               1 days
closed  http://bugs.python.org/issue3114   

 11 memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Lin    7 days
open    http://bugs.python.org/issue3107   

  8 test_multiprocessing hangs on OS X 10.5.3                          8 days
open    http://bugs.python.org/issue3088   

  8 Bug in slice.indices()                                            22 days
closed  http://bugs.python.org/issue3004   

  7 Let bin() show floats                                             21 days
open    http://bugs.python.org/issue3008   

  7 robotparser.py fixes                                             304 days
open    http://bugs.python.org/issue1778443

  6 Regex causes python to hang up? / loop infinite?                   0 days
closed  http://bugs.python.org/issue3128   




From rhamph at gmail.com  Fri Jun 20 19:54:15 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 20 Jun 2008 11:54:15 -0600
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
Message-ID: <aac2c7cb0806201054qddc7751p1f85b84e06dfc70d@mail.gmail.com>

On Fri, Jun 20, 2008 at 9:44 AM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> 2008/6/20 Kevin Jacobs <jacobs at bioinformed.com> <bioinformed at gmail.com>:
>> On Fri, Jun 20, 2008 at 10:25 AM, Antoine Pitrou <solipsis at pitrou.net>
>> wrote:
>>>
>>> Kevin Jacobs <jacobs <at> bioinformed.com> <bioinformed <at> gmail.com>
>>> writes:
>>> >
>>> > +1 on a C API for enabling and disabling GC.  I have several instances
>>> > where
>>> I create a large number of objects non-cyclic objects where I see huge GC
>>> overhead (30+ seconds with gc enabled, 0.15 seconds when disabled).
>>>
>>> Could you try to post a stripped-down, self-contained example of such
>>> behaviour?
>>
>> $ python -m timeit 'zip(*[range(1000000)]*5)'
>> 10 loops, best of 3: 496 msec per loop
>>
>> $ python -m timeit -s 'import gc; gc.enable()' 'zip(*[range(1000000)]*5)'
>> 10 loops, best of 3: 2.93 sec per loop
>
> I remember that a similar issue was discussed some months ago:
> http://bugs.python.org/issue2607
>
> In short: the gc is tuned for typical usage. If your usage of python
> is specific,
> use gc.set_threshold and increase its values.

For very large bursts of allocation, tuning is no different from
disabling it outright, and disabling is simpler/more reliable.


-- 
Adam Olsen, aka Rhamphoryncus

From amk at amk.ca  Fri Jun 20 19:58:18 2008
From: amk at amk.ca (A.M. Kuchling)
Date: Fri, 20 Jun 2008 13:58:18 -0400
Subject: [Python-Dev] r64407
	-	python/trunk/Doc/library/multiprocessing.rst
In-Reply-To: <08Jun20.085516pdt."58698"@synergy1.parc.xerox.com>
References: <20080619194843.0A12B1E4002@bag.python.org>
	<1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com>
	<20080619203030.GA10666@amk-desktop.matrixgroup.net>
	<4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com>
	<20080620130949.GA6641@amk-desktop.matrixgroup.net>
	<08Jun20.085516pdt."58698"@synergy1.parc.xerox.com>
Message-ID: <20080620175818.GA7872@amk-desktop.matrixgroup.net>

On Fri, Jun 20, 2008 at 08:55:13AM -0700, Bill Janssen wrote:
> > Is anyone else finding it increasingly odd that subprocess, signal,
> > socket/ssl, and syncore are in the same chapter?  I'm tempted to move
> > socket, ssl, asyncore+asynchat into a 'networking' chapter, and then
> > also move SocketServer from the 'Internet Protocols' chapter into this
> > new chapter.
> 
> Sounds like you mean, 'Non-HTTP Networking'.

I don't think so -- SMTP, FTP, NNTP, and telnet have nothing
to do with HTTP, but they're certainly Internet protocols.

--amk

From janssen at parc.com  Fri Jun 20 20:26:52 2008
From: janssen at parc.com (Bill Janssen)
Date: Fri, 20 Jun 2008 11:26:52 PDT
Subject: [Python-Dev] r64407 -
	python/trunk/Doc/library/multiprocessing.rst
In-Reply-To: <20080620175818.GA7872@amk-desktop.matrixgroup.net> 
References: <20080619194843.0A12B1E4002@bag.python.org>
	<1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com>
	<20080619203030.GA10666@amk-desktop.matrixgroup.net>
	<4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com>
	<20080620130949.GA6641@amk-desktop.matrixgroup.net>
	<08Jun20.085516pdt."58698"@synergy1.parc.xerox.com>
	<20080620175818.GA7872@amk-desktop.matrixgroup.net>
Message-ID: <08Jun20.112653pdt."58698"@synergy1.parc.xerox.com>

> On Fri, Jun 20, 2008 at 08:55:13AM -0700, Bill Janssen wrote:
> > > Is anyone else finding it increasingly odd that subprocess, signal,
> > > socket/ssl, and syncore are in the same chapter?  I'm tempted to move
> > > socket, ssl, asyncore+asynchat into a 'networking' chapter, and then
> > > also move SocketServer from the 'Internet Protocols' chapter into this
> > > new chapter.
> > 
> > Sounds like you mean, 'Non-HTTP Networking'.
> 
> I don't think so -- SMTP, FTP, NNTP, and telnet have nothing
> to do with HTTP, but they're certainly Internet protocols.

Sure, but there's typically only one protocol that's carried over any
of them.  Whereas with REST and XML-RPC and so forth, there's a whole
cottage industry of using HTTP as a carrier, in much the same way that
TCP is used at a lower level.

But you're right: there's "Networking / Generic Support", "Networking
/ Specific Protocols", and "Networking / HTTP-based", or some such.

Bill

From eric+python-dev at trueblade.com  Fri Jun 20 23:17:23 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Fri, 20 Jun 2008 17:17:23 -0400
Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:
 Include/object.h Lib/test/test_sys.py Misc/NEWS
 Objects/intobject.c	Objects/longobject.c Objects/typeobject.c
 Python/bltinmodule.c
In-Reply-To: <20080620041816.4D5E81E4002@bag.python.org>
References: <20080620041816.4D5E81E4002@bag.python.org>
Message-ID: <485C1E63.2020900@trueblade.com>

I thought there was a discussion of this earlier, and the idea was to 
leave the prior implementation, because that's how it's implemented in 
3.0.  bin() is a new feature in 2.6, so there's no particular need to 
make it work like hex() and oct().

Recall that in 3.0, __bin__, __oct__, and __hex__ don't exist.  Instead, 
you use __index__ for integer conversions.  That's how bin() worked in 
2.6 until this checkin.

But now that I look for it, I can't find the original discussion.

raymond.hettinger wrote:
> Author: raymond.hettinger
> Date: Fri Jun 20 06:18:15 2008
> New Revision: 64424
> 
> Log:
> Make bin() implementation parallel oct() and hex() so that int/long subclasses can override or so that other classes can support.

From steve at holdenweb.com  Fri Jun 20 23:46:24 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 20 Jun 2008 17:46:24 -0400
Subject: [Python-Dev] r64407 -
	python/trunk/Doc/library/multiprocessing.rst
In-Reply-To: <20080620175818.GA7872@amk-desktop.matrixgroup.net>
References: <20080619194843.0A12B1E4002@bag.python.org>	<1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com>	<20080619203030.GA10666@amk-desktop.matrixgroup.net>	<4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com>	<20080620130949.GA6641@amk-desktop.matrixgroup.net>	<08Jun20.085516pdt."58698"@synergy1.parc.xerox.com>
	<20080620175818.GA7872@amk-desktop.matrixgroup.net>
Message-ID: <g3h8fg$q4v$1@ger.gmane.org>

A.M. Kuchling wrote:
> On Fri, Jun 20, 2008 at 08:55:13AM -0700, Bill Janssen wrote:
>>> Is anyone else finding it increasingly odd that subprocess, signal,
>>> socket/ssl, and syncore are in the same chapter?  I'm tempted to move
>>> socket, ssl, asyncore+asynchat into a 'networking' chapter, and then
>>> also move SocketServer from the 'Internet Protocols' chapter into this
>>> new chapter.
>> Sounds like you mean, 'Non-HTTP Networking'.
> 
> I don't think so -- SMTP, FTP, NNTP, and telnet have nothing
> to do with HTTP, but they're certainly Internet protocols.
> 
Perhaps we need a split between "networking technologies" and 
"network-based applications".

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From fdrake at acm.org  Fri Jun 20 23:55:57 2008
From: fdrake at acm.org (Fred Drake)
Date: Fri, 20 Jun 2008 17:55:57 -0400
Subject: [Python-Dev] r64407 -
	python/trunk/Doc/library/multiprocessing.rst
In-Reply-To: <g3h8fg$q4v$1@ger.gmane.org>
References: <20080619194843.0A12B1E4002@bag.python.org>	<1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com>	<20080619203030.GA10666@amk-desktop.matrixgroup.net>	<4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com>	<20080620130949.GA6641@amk-desktop.matrixgroup.net>	<08Jun20.085516pdt."58698"@synergy1.parc.xerox.com>
	<20080620175818.GA7872@amk-desktop.matrixgroup.net>
	<g3h8fg$q4v$1@ger.gmane.org>
Message-ID: <9438CBE3-FA32-451A-B756-DDE78C5CE060@acm.org>

On Jun 20, 2008, at 5:46 PM, Steve Holden wrote:
> Perhaps we need a split between "networking technologies" and  
> "network-based applications".


Perhaps that would help.

I certainly see HTTP as being on the same layer as SMTP and the like,  
but application protocols that ride on top of HTTP are a different  
beast.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From dbpokorny at gmail.com  Sat Jun 21 03:51:02 2008
From: dbpokorny at gmail.com (dbpokorny at gmail.com)
Date: Fri, 20 Jun 2008 18:51:02 -0700 (PDT)
Subject: [Python-Dev] Status of Issue 2331 - Backport parameter
	annotations
In-Reply-To: <A6B48427-3D8C-4CBC-B47C-DAF7511F348C@PageDNA.com>
References: <cad407ed0806191559s2ec37894mbae60bc6c775e7c0@mail.gmail.com> 
	<ca471dc20806191616v2baa73ccn9b719bddbe5846cc@mail.gmail.com> 
	<f1161fd1-cb61-4a0b-bcdd-6610563c8704@y22g2000prd.googlegroups.com> 
	<A6B48427-3D8C-4CBC-B47C-DAF7511F348C@PageDNA.com>
Message-ID: <2be1c9ce-41df-41e5-a50a-47de509cb3e9@z24g2000prf.googlegroups.com>

> I can help. I don't have a patch against the trunk but my first
> revisions of the patch
> for annotations did handle things like tuple parameters which are
> relevant to 2.6.

Ah yes, I forgot about nested parameters. I see that 53170 still has
nested parameters, but they were removed at some later point. Do you
think this revision (and the patch it checked in) is the best starting
point, or are my chances better with the patches at http://bugs.python.org/issue1607548
?

I just learned that default argument values are not allowed for nested
parameters (in any version of Python AFAIK). Is there any reason for
this other than lack of tuits?

Cheers,
David

From solipsis at pitrou.net  Sat Jun 21 03:39:46 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 21 Jun 2008 03:39:46 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
Message-ID: <1214012386.6402.4.camel@fsol>

Le vendredi 20 juin 2008 ? 17:44 +0200, Amaury Forgeot d'Arc a ?crit :
> In short: the gc is tuned for typical usage. If your usage of python
> is specific,
> use gc.set_threshold and increase its values.

It's fine for people "in the know" who take the time to test their code
using various gc parameters. But most people don't (I know I never did,
and until recently I didn't even know the gc could have such a
disastrous effect on performance).

I don't think expecting people to tweak gc parameters when they witness
performance problems is reasonable.



From martin at v.loewis.de  Sat Jun 21 10:33:30 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 21 Jun 2008 10:33:30 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <1214012386.6402.4.camel@fsol>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>	<485AFBAB.2060404@canterbury.ac.nz>	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>	<loom.20080620T142315-395@post.gmane.org>	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
	<1214012386.6402.4.camel@fsol>
Message-ID: <485CBCDA.4020601@v.loewis.de>

> I don't think expecting people to tweak gc parameters when they witness
> performance problems is reasonable.

What follows from that? To me, the natural conclusion is "people who
witness performance problems just need to despair, or accept them, as
they can't do anything about it", however, I don't think this is the
conclusion that you had in mind.

Regards,
Martin

From dbpokorny at gmail.com  Sat Jun 21 10:41:04 2008
From: dbpokorny at gmail.com (dbpokorny at gmail.com)
Date: Sat, 21 Jun 2008 01:41:04 -0700 (PDT)
Subject: [Python-Dev] Status of Issue 2331 - Backport parameter
	annotations
In-Reply-To: <g3f3kv$25r$1@ger.gmane.org>
References: <cad407ed0806191559s2ec37894mbae60bc6c775e7c0@mail.gmail.com> 
	<g3f3kv$25r$1@ger.gmane.org>
Message-ID: <f5eb4430-a41a-485e-a51d-c79ec3281f11@u6g2000prc.googlegroups.com>

> Does it bother you that you need ()s to make instances elsewhere?
> That you have to type int('123') instead of int, '123'?

Not at all...Python never supported this.

Cheers,
David

From drnlmuller+python at gmail.com  Sat Jun 21 12:16:24 2008
From: drnlmuller+python at gmail.com (Neil Muller)
Date: Sat, 21 Jun 2008 12:16:24 +0200
Subject: [Python-Dev] Issue 2722
Message-ID: <a4cf72520806210316w62c9f45r5a7f04e80bdc92f5@mail.gmail.com>

Could some one have a look at the suggested fix for issue 2722? While
not a particularly common problem, it can be an annoyance, and the
patch looks reasonable.

-- 
Neil Muller

From skip at pobox.com  Sat Jun 21 13:06:24 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 21 Jun 2008 06:06:24 -0500
Subject: [Python-Dev] csv module and Unicode in 2.6 and 3.0
Message-ID: <18524.57520.222364.658608@montanaro-dyndns-org.local>


If I understand things correctly, the csv module in 3.0 supports Unicode,
kind of for free because all strings are Unicode.  I suspect the Unicode
examples at the bottom of the 3.0 version of the module doc need to be
removed.

Does the 2.6 version support Unicode out of the box as well or are the
Unicode examples still required there?

Skip

From solipsis at pitrou.net  Sat Jun 21 15:18:31 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 21 Jun 2008 15:18:31 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <485CBCDA.4020601@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
	<1214012386.6402.4.camel@fsol>  <485CBCDA.4020601@v.loewis.de>
Message-ID: <1214054311.5874.12.camel@fsol>

Le samedi 21 juin 2008 ? 10:33 +0200, "Martin v. L?wis" a ?crit :
> > I don't think expecting people to tweak gc parameters when they witness
> > performance problems is reasonable.
> 
> What follows from that? To me, the natural conclusion is "people who
> witness performance problems just need to despair, or accept them, as
> they can't do anything about it",

To me, Amaury's answer implies that most people can't do anything about
it, indeed.

Well, they could hang themselves or switch to another language (which
some people might view as equivalent :-)), but perhaps optimistically
the various propositions that were sketched out in this thread (by Adam
Olsen and Greg Ewing) could bring an improvement. I don't know how
realistic they are, perhaps an expert would have an answer.

Regards

Antoine.



From bioinformed at gmail.com  Sat Jun 21 16:27:43 2008
From: bioinformed at gmail.com (Kevin Jacobs <jacobs@bioinformed.com>)
Date: Sat, 21 Jun 2008 10:27:43 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <485CBCDA.4020601@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
	<1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de>
Message-ID: <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com>

On Sat, Jun 21, 2008 at 4:33 AM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> > I don't think expecting people to tweak gc parameters when they witness
> > performance problems is reasonable.
>
> What follows from that? To me, the natural conclusion is "people who
> witness performance problems just need to despair, or accept them, as
> they can't do anything about it", however, I don't think this is the
> conclusion that you had in mind.
>

I can say with complete certainty that of the 20+ programmers I've had
working for me, many who have used Python for 3+ years, not a single one
would think to question the garbage collector if they observed the kind of
quadratic time complexity I've demonstrated.  This is not because they are
stupid, but because they have only a vague idea that Python even has a
garbage collector, never mind that it could be behaving badly for such
innocuous looking code.

Maybe we should consider more carefully before declaring the status quo
sufficient.  Average developers do allocate millions of objects in bursts
and super-linear time complexity for such operations is not acceptable.
Thankfully I am around to help my programmers work around such issues or
else they'd be pushing to switch to Java, Ruby, C#, or whatever since Python
was inexplicably "too slow" for "real work".  This being open source, I'm
certainly willing to help in the effort to do so, but not if potential
solutions will be ruled out as being unnecessary.

-Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080621/a3c4ac50/attachment.htm>

From martin at v.loewis.de  Sat Jun 21 17:11:16 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Jun 2008 17:11:16 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>	
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>	
	<485AFBAB.2060404@canterbury.ac.nz>	
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>	
	<loom.20080620T142315-395@post.gmane.org>	
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>	
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	
	<1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de>
	<2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com>
Message-ID: <485D1A14.60005@v.loewis.de>

> I can say with complete certainty that of the 20+ programmers I've had
> working for me, many who have used Python for 3+ years, not a single one
> would think to question the garbage collector if they observed the kind
> of quadratic time complexity I've demonstrated.  This is not because
> they are stupid, but because they have only a vague idea that Python
> even has a garbage collector, never mind that it could be behaving badly
> for such innocuous looking code.
> 
> Maybe we should consider more carefully before declaring the status quo
> sufficient.

This was precisely my question: What follows from the above observation?

I personally didn't declare the status quo sufficient - I merely
declared it as being the status quo.

> Average developers do allocate millions of objects in
> bursts and super-linear time complexity for such operations is not
> acceptable.  Thankfully I am around to help my programmers work around
> such issues or else they'd be pushing to switch to Java, Ruby, C#, or
> whatever since Python was inexplicably "too slow" for "real work".  This
> being open source, I'm certainly willing to help in the effort to do so,
> but not if potential solutions will be ruled out as being unnecessary.

I wouldn't rule out solutions as being unnecessary. I might rule out
solutions that negatively impact existing software, for the sake of
improving other existing software.

Unfortunately, the only way to find out whether a solution will be ruled
out is to propose it first. Only then you'll see what response you get.

Regards,
Martin

From martin at v.loewis.de  Sat Jun 21 17:20:07 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 21 Jun 2008 17:20:07 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <1214054311.5874.12.camel@fsol>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>	<485AFBAB.2060404@canterbury.ac.nz>	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>	<loom.20080620T142315-395@post.gmane.org>	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	<1214012386.6402.4.camel@fsol>
	<485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol>
Message-ID: <485D1C27.60408@v.loewis.de>

> Well, they could hang themselves or switch to another language (which
> some people might view as equivalent :-)), but perhaps optimistically
> the various propositions that were sketched out in this thread (by Adam
> Olsen and Greg Ewing) could bring an improvement. I don't know how
> realistic they are, perhaps an expert would have an answer.

In general, any solution of the "do GC less often" needs to deal with
cases where lots of garbage gets produced in a short amount of time
(e.g. in a tight loop), and which run out of memory when GC is done less
often.

Given the choice of "run slower" and "run out of memory", Python should
always prefer the former.

One approach could be to measure how successful a GC run was: if GC
finds that more-and-more objects get allocated and very few (or none)
are garbage, it might conclude that this is an allocation spike, and
back off. The tricky question is how to find out that the spike is
over.

Regards,
Martin

From bioinformed at gmail.com  Sat Jun 21 17:33:14 2008
From: bioinformed at gmail.com (Kevin Jacobs <jacobs@bioinformed.com>)
Date: Sat, 21 Jun 2008 11:33:14 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <485D1C27.60408@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
	<1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de>
	<1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de>
Message-ID: <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>

On Sat, Jun 21, 2008 at 11:20 AM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> In general, any solution of the "do GC less often" needs to deal with
> cases where lots of garbage gets produced in a short amount of time
> (e.g. in a tight loop), and which run out of memory when GC is done less
> often.
>


Idea 1: Allow GC to run automatically no more often than n CPU seconds, n
being perhaps 5 or 10.
Idea 2: Allow GC to run no more often than f(n) CPU seconds, where n is the
time taken by the last GC round.

These limits could be reset or scaled by the GC collecting more than n% of
the generation 0 objects or maybe the number of PyMalloc arenas increasing
by a certain amount?

-Kevin


> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jacobs%40bioinformed.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080621/f0126b1a/attachment.htm>

From martin at v.loewis.de  Sat Jun 21 17:49:49 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Jun 2008 17:49:49 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	
	<485AFBAB.2060404@canterbury.ac.nz>	
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>	
	<loom.20080620T142315-395@post.gmane.org>	
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>	
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	
	<1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de>	
	<1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de>
	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>
Message-ID: <485D231D.9030806@v.loewis.de>

> Idea 1: Allow GC to run automatically no more often than n CPU seconds,
> n being perhaps 5 or 10.

I think it's very easy to exhaust the memory with such a policy, even
though much memory would still be available. Worse, in a program
producing a lot of garbage, performance will go significantly down as
the python starts thrashing the swap space.

> Idea 2: Allow GC to run no more often than f(n) CPU seconds, where n is
> the time taken by the last GC round.

How would that take the incremental GC into account? (i.e. what is
"the time taken by the last GC round"?)

Furthermore, the GC run time might well be under the resolution of the
CPU seconds clock.

> These limits could be reset or scaled by the GC collecting more than n%
> of the generation 0 objects or maybe the number of PyMalloc arenas
> increasing by a certain amount?

I don't think any strategies based on timing will be successful.
Instead, one should count and analyze objects (although I'm unsure
how exactly that could work).

Regards,
Martin

From aahz at pythoncraft.com  Sat Jun 21 18:17:42 2008
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 21 Jun 2008 09:17:42 -0700
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <485D1C27.60408@v.loewis.de>
References: <acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
	<1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de>
	<1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de>
Message-ID: <20080621161742.GA15793@panix.com>

On Sat, Jun 21, 2008, "Martin v. L??wis" wrote:
> 
> In general, any solution of the "do GC less often" needs to deal with
> cases where lots of garbage gets produced in a short amount of time
> (e.g. in a tight loop), and which run out of memory when GC is done less
> often.
> 
> Given the choice of "run slower" and "run out of memory", Python should
> always prefer the former.

I'm not sure I agree with this.  GC IIRC was introduced primarily to
alleviate *long-term* memory starvation.  You are now IMO adding a new
goal for GC that has not been previously articulated.  I believe this
requires consensus rather than a simple declaration of principle.

Guido's opinion if he has one obviously overrules.  ;-)  Guido?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha

From martin at v.loewis.de  Sat Jun 21 18:36:10 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Jun 2008 18:36:10 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <20080621161742.GA15793@panix.com>
References: <acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>	<485AFBAB.2060404@canterbury.ac.nz>	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>	<loom.20080620T142315-395@post.gmane.org>	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	<1214012386.6402.4.camel@fsol>
	<485CBCDA.4020601@v.loewis.de>	<1214054311.5874.12.camel@fsol>
	<485D1C27.60408@v.loewis.de> <20080621161742.GA15793@panix.com>
Message-ID: <485D2DFA.6070306@v.loewis.de>

> I'm not sure I agree with this.  GC IIRC was introduced primarily to
> alleviate *long-term* memory starvation.

I don't think that's historically the case. GC would not need to be
generational if releasing short-lived objects shortly after they become
garbage was irrelevant. Of course, it was always expected that much
memory is released through mere reference counting, and that GC only
kicks in "after some time". However "after some time" was changed from
"after 5000 allocations" to "after 700 allocations" in

------------------------------------------------------------------------
r17274 | jhylton | 2000-09-05 17:44:50 +0200 (Di, 05 Sep 2000) | 2 lines
Ge?nderte Pfade:
   M /python/trunk/Modules/gcmodule.c

compromise value for threshold0: not too high, not too low

------------------------------------------------------------------------

Regards,
Martin

From janssen at parc.com  Sat Jun 21 19:10:06 2008
From: janssen at parc.com (Bill Janssen)
Date: Sat, 21 Jun 2008 10:10:06 PDT
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com> 
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
	<1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de>
	<2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com>
Message-ID: <08Jun21.101016pdt."58698"@synergy1.parc.xerox.com>

> > What follows from that? To me, the natural conclusion is "people who
> > witness performance problems just need to despair, or accept them, as
> > they can't do anything about it", however, I don't think this is the
> > conclusion that you had in mind.
> >
> 
> I can say with complete certainty that of the 20+ programmers I've had
> working for me, many who have used Python for 3+ years, not a single one
> would think to question the garbage collector if they observed the kind of
> quadratic time complexity I've demonstrated.  This is not because they are
> stupid, but because they have only a vague idea that Python even has a
> garbage collector, never mind that it could be behaving badly for such
> innocuous looking code.

Perhaps this is something documentation could help.  I'm thinking of a
one-page checklist listing places they might look for performance
problems, that your programmers could work through.

Bill

From tjreedy at udel.edu  Sat Jun 21 20:40:16 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 21 Jun 2008 14:40:16 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>	<485AFBAB.2060404@canterbury.ac.nz>	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>	<loom.20080620T142315-395@post.gmane.org>	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	<1214012386.6402.4.camel@fsol>
	<485CBCDA.4020601@v.loewis.de>
	<2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com>
Message-ID: <g3jhub$cao$1@ger.gmane.org>



Kevin Jacobs <jacobs at bioinformed.com> wrote:

> I can say with complete certainty that of the 20+ programmers I've had 
> working for me, many who have used Python for 3+ years, not a single one 
> would think to question the garbage collector if they observed the kind 
> of quadratic time complexity I've demonstrated.  This is not because 
> they are stupid, but because they have only a vague idea that Python 
> even has a garbage collector, never mind that it could be behaving badly 
> for such innocuous looking code.

As I understand it, gc is needed now more that ever because new style 
classes make reference cycles more common.  On the other hand, greatly 
increased RAM size (from some years ago) makes megaobject bursts 
possible.  Such large bursts move the hidden quadratic do-nothing drag 
out of the relatively flat part of the curve (total time just double or 
triple what it should be) to where it can really bite.  Leaving aside 
what you do for your local group, can we better warn Python programmers 
now, for the upcoming 2.5, 2.6, and 3.0 releases?

Paragraph 3 of the Reference Manual chapter on Data Model(3.0 version) says:
"Objects are never explicitly destroyed; however, when they become 
unreachable they may be garbage-collected. An implementation is allowed 
to postpone garbage collection or omit it altogether ? it is a matter of 
implementation quality how garbage collection is implemented, as long as 
no objects are collected that are still reachable. (Implementation note: 
the current implementation uses a reference-counting scheme with 
(optional) delayed detection of cyclically linked garbage, which 
collects most objects as soon as they become unreachable, but is not 
guaranteed to collect garbage containing circular references. See the 
documentation of the gc module for information on controlling the 
collection of cyclic garbage.)"
I am not sure what to add here, (especially for those who do not read it;-).

The Library Manual gc section says "Since the collector supplements the 
reference counting already used in Python, you can disable the collector 
if you are sure your program does not create reference cycles."  Perhaps 
  it should also say "You should disable when creating millions of 
objects without cycles".

The installed documentation set (on Windows, at least) include some 
Python HOWTOs.  If one were added on Space Management (implementations, 
problems, and solutions), would your developers read it?

> Maybe we should consider more carefully before declaring the status quo 
> sufficient.  Average developers do allocate millions of objects in 
> bursts and super-linear time complexity for such operations is not 
> acceptable.  Thankfully I am around to help my programmers work around 
> such issues or else they'd be pushing to switch to Java, Ruby, C#, or 
> whatever since Python was inexplicably "too slow" for "real work".  This 
> being open source, I'm certainly willing to help in the effort to do so, 
> but not if potential solutions will be ruled out as being unnecessary.

To me, 'sufficient' (time-dependent) and 'necessary' are either too 
vague or  too strict to being about what you want -- change.  This is 
the third thread I have read (here + c.l.p) on default-mode gc  problems 
(but all in the last couple of years or so).  So, especially with the 
nice table someone posted recently, on time with and without gc, and 
considering that installed RAM continues to grow, I am persuaded that 
default behavior improvement that does not negatively impact the vast 
majority would be desirable.

Terry Jan Reedy


From solipsis at pitrou.net  Sat Jun 21 21:10:10 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 21 Jun 2008 21:10:10 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <485D231D.9030806@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
	<1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de>
	<1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de>
	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>
	<485D231D.9030806@v.loewis.de>
Message-ID: <1214075410.5874.26.camel@fsol>


Le samedi 21 juin 2008 ? 17:49 +0200, "Martin v. L?wis" a ?crit : 
> I don't think any strategies based on timing will be successful.
> Instead, one should count and analyze objects (although I'm unsure
> how exactly that could work).

Would it be helpful if the GC was informed of memory growth by the
Python memory allocator (that is, each time it either asks or gives back
a block of memory to the system allocator) ?



From martin at v.loewis.de  Sat Jun 21 21:56:44 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 21 Jun 2008 21:56:44 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <1214075410.5874.26.camel@fsol>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<485AFBAB.2060404@canterbury.ac.nz>	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>	<loom.20080620T142315-395@post.gmane.org>	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	<1214012386.6402.4.camel@fsol>
	<485CBCDA.4020601@v.loewis.de>	<1214054311.5874.12.camel@fsol>
	<485D1C27.60408@v.loewis.de>	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>	<485D231D.9030806@v.loewis.de>
	<1214075410.5874.26.camel@fsol>
Message-ID: <485D5CFC.5020705@v.loewis.de>

Antoine Pitrou wrote:
> Le samedi 21 juin 2008 ? 17:49 +0200, "Martin v. L?wis" a ?crit : 
>> I don't think any strategies based on timing will be successful.
>> Instead, one should count and analyze objects (although I'm unsure
>> how exactly that could work).
> 
> Would it be helpful if the GC was informed of memory growth by the
> Python memory allocator (that is, each time it either asks or gives back
> a block of memory to the system allocator) ?

I don't see how. The garbage collector is already informed about memory
growth; it learns exactly when a container object is allocated or
deallocated. That the allocator then requests memory from the system
only confirms what the garbage collector already knew: that there are
lots of allocated objects. From that, one could infer that it might
be time to perform garbage collection - or one could infer that all
the objects are really useful, and no garbage can be collected.

Regards,
Martin

From martin at v.loewis.de  Sat Jun 21 22:23:16 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Jun 2008 22:23:16 +0200
Subject: [Python-Dev] Proposal: Run GC less often
Message-ID: <485D6334.6080102@v.loewis.de>

Here is my proposal for making the GC run less often.
The objective is to avoid the quadratic-time behavior
if many objects are created and none of them is garbage.

The youngest generation remains collected after 700
allocations, the middle generation after 10 young collections.
Therefore, full GC remains considered every 7000 allocations.
The two youngest generations don't cause quadratic behavior,
as the number of objects in each generation is bounded.

Currently, full GC is invoked every 10 middle collections.
Under my proposal, 10 middle collections must have passed,
PLUS the number of survivor objects from the middle generation
must exceed 10% of the number of objects in the oldest
generation.

As a consequence, garbage collection becomes less frequent
as the number of objects on the heap grows, resulting in
an amortized O(N) overhead for garbage collection (under
the access pattern stated above).

To determine the number of survivor objects, counts are
taken during the collection. Objects deallocated through
reference counting still remain accounted for as survivors
(likewise for determining the size of the oldest generation).

I made a simulation assuming an initially-empty heap,
and invocations of the collector every M=7000 objects.
The table below is in units of M and shows the number
of objects allocated, and the number of objects inspected
since the start of the simulation, for the old and the
new scheme (the asterisk indicates that a collection
took place; lines with no collection are dropped):

 total old_inspected new_inspected
   10         10*         10*
   20         30*         30*
   30         60*         60*
   40        100*        100*
   50        150*        150*
   60        210*        210*
   70        280*        280*
   80        360*        360*
   90        450*        450*
  100        550*        450
  102        550         552*
  110        660*        552
  115        660         667*
  120        780*        667
  129        780         796*
  130        910*        796
  140       1050*        796
...
  940      44650*       7724
  942      44650        8666*
  950      45600*       8666
  960      46560*       8666
  970      47530*       8666
  980      48510*       8666
  990      49500*       8666
 1000      50500*       8666
...
 9990    4995000*      95887

As you can see, the old and the new scheme would start
of equally until 100M objects have been allocated. The
table shows how the old scheme grows quadratically, and
the new scheme eventually approaches roughly a factor
of ten between the number of objects and the number of
times that GC considers an object.

Applications with a small number of objects will see no
change in behavior, also, applications that create
lots of small cycles will continue to see them collected
in the youngest or middle generations.

Please let me know what you think.

Regards,
Martin

P.S. I don't plan to implement this myself, so if you like
the idea, go ahead.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: comp.py
Type: text/x-python
Size: 719 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080621/6bf90fed/attachment.py>

From brett at python.org  Sat Jun 21 22:54:51 2008
From: brett at python.org (Brett Cannon)
Date: Sat, 21 Jun 2008 13:54:51 -0700
Subject: [Python-Dev] Proposal: Run GC less often
In-Reply-To: <485D6334.6080102@v.loewis.de>
References: <485D6334.6080102@v.loewis.de>
Message-ID: <bbaeab100806211354h48a31dbfu7d887a33edf9bd31@mail.gmail.com>

On Sat, Jun 21, 2008 at 1:23 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Here is my proposal for making the GC run less often.
> The objective is to avoid the quadratic-time behavior
> if many objects are created and none of them is garbage.
>
[SNIP]
> Applications with a small number of objects will see no
> change in behavior, also, applications that create
> lots of small cycles will continue to see them collected
> in the youngest or middle generations.
>
> Please let me know what you think.
>

Interesting. Seems reasonable to me if the impact really is as minimal
as you say, Martin.

-Brett

From stephen at xemacs.org  Sat Jun 21 23:36:00 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 22 Jun 2008 06:36:00 +0900
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <485D1C27.60408@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
	<1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de>
	<1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de>
Message-ID: <87bq1umpxb.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:

 > Given the choice of "run slower" and "run out of memory", Python should
 > always prefer the former.
 > 
 > One approach could be to measure how successful a GC run was: if GC
 > finds that more-and-more objects get allocated and very few (or none)
 > are garbage, it might conclude that this is an allocation spike, and
 > back off. The tricky question is how to find out that the spike is
 > over.

XEmacs implements this strategy in a way which is claimed to give
constant amortized time (ie, averaged over memory allocated).  I
forget the exact parameters, but ISTR it's just period ("time"
measured by bytes allocated) is proportional to currently allocated
memory.  Some people claim this is much more comfortable than the
traditional "GC after N bytes are allocated" algorithm but I don't
notice much difference.  I don't know how well this intuition carries
over to noninteractive applications.

In XEmacs experimenting with such strategies is pretty easy, since the
function that determines period is only a few lines long.  I assume
that would be true of Python, too.

However, isn't the real question whether there is memory pressure or
not?  If you've got an unloaded machine with 2GB of memory, even a 1GB
spike might have no observable consequences.  How about a policy of
GC-ing with decreasing period ("time" measured by bytes allocated or
number of allocations) as the fraction of memory used increases,
starting from a pretty large fraction (say 50% by default)?  The total
amount of memory could be a soft limit, defaulting to the amount of
fast memory actually available.

For interactive and maybe some batch applications, it might be
appropriate to generate a runtime warning as memory use approches some
limits, too.

Nevertheless, I think the real solution has to be for Python
programmers to be aware that there is GC, and that they can tune it.

From ralfs72 at googlemail.com  Sat Jun 21 23:28:22 2008
From: ralfs72 at googlemail.com (none)
Date: Sat, 21 Jun 2008 23:28:22 +0200
Subject: [Python-Dev] Another Proposal: Run GC less often
Message-ID: <485d7277.0b38560a.065a.ffffcada@mx.google.com>

Instead of collecting objects after a fixed number of allocations (700) 
You could make it dynamic like this:

# initial values
min_allocated_memory = 0
max_allocated_memory = 0
next_gc_run = 1024 * 1024

def manage_memory():
     allocated_memory = get_allocated_memory()
     min_allocated_memory = min(min_allocated_memory, allocated_memory)
     max_allocated_memory = max(max_allocated_memory, allocated_memory)

     if max_allocated_memory - min_allocated_memory > next_gc_run:
         # run the gc
         memory_freed, allocated_memory = run_gc()
         next_gc_run = max(
         	allocated_memory * 1.5 - memory_freed, 1024 * 1024
     	)
         min_allocated_memory = allocated_memory
         max_allocated_memory = allocated_memory


manage_memory() should be called after every allocation and after a ref 
count of an object reaches 0 (memory is freed)


Expected behaviours:

=> As less objects contain cyclic references as less often the GC will 
run (memory_freed is small)

=> As more objects contain cyclic references as more often the GC will 
run (memory_freed is large)

=> If memory utiliaziation grows fast (burst allocations) GC will run 
less often: next_gc_run = allocated_memory * 1.5 - memory_freed

... Of course the constants: 1.5 and 1024 * 1024 are only suggestions...

- Ralf

From adlaiff6 at gmail.com  Sat Jun 21 23:33:15 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Sat, 21 Jun 2008 14:33:15 -0700
Subject: [Python-Dev] Proposal: Run GC less often
In-Reply-To: <485D6334.6080102@v.loewis.de>
References: <485D6334.6080102@v.loewis.de>
Message-ID: <cc7430500806211433g11169c73xb6eef3f06c26c37c@mail.gmail.com>

If you can get me a version of the interpreter with this change made
(I wouldn't know what to change), I can run a very
allocation/deallocation-heavy application I have lying around, and get
you some good benchmarks.

On Sat, Jun 21, 2008 at 1:23 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
- Show quoted text -
> Here is my proposal for making the GC run less often.
> The objective is to avoid the quadratic-time behavior
> if many objects are created and none of them is garbage.
>
> The youngest generation remains collected after 700
> allocations, the middle generation after 10 young collections.
> Therefore, full GC remains considered every 7000 allocations.
> The two youngest generations don't cause quadratic behavior,
> as the number of objects in each generation is bounded.
>
> Currently, full GC is invoked every 10 middle collections.
> Under my proposal, 10 middle collections must have passed,
> PLUS the number of survivor objects from the middle generation
> must exceed 10% of the number of objects in the oldest
> generation.
>
> As a consequence, garbage collection becomes less frequent
> as the number of objects on the heap grows, resulting in
> an amortized O(N) overhead for garbage collection (under
> the access pattern stated above).
>
> To determine the number of survivor objects, counts are
> taken during the collection. Objects deallocated through
> reference counting still remain accounted for as survivors
> (likewise for determining the size of the oldest generation).
>
> I made a simulation assuming an initially-empty heap,
> and invocations of the collector every M=7000 objects.
> The table below is in units of M and shows the number
> of objects allocated, and the number of objects inspected
> since the start of the simulation, for the old and the
> new scheme (the asterisk indicates that a collection
> took place; lines with no collection are dropped):
>
>  total old_inspected new_inspected
>   10         10*         10*
>   20         30*         30*
>   30         60*         60*
>   40        100*        100*
>   50        150*        150*
>   60        210*        210*
>   70        280*        280*
>   80        360*        360*
>   90        450*        450*
>  100        550*        450
>  102        550         552*
>  110        660*        552
>  115        660         667*
>  120        780*        667
>  129        780         796*
>  130        910*        796
>  140       1050*        796
> ...
>  940      44650*       7724
>  942      44650        8666*
>  950      45600*       8666
>  960      46560*       8666
>  970      47530*       8666
>  980      48510*       8666
>  990      49500*       8666
>  1000      50500*       8666
> ...
>  9990    4995000*      95887
>
> As you can see, the old and the new scheme would start
> of equally until 100M objects have been allocated. The
> table shows how the old scheme grows quadratically, and
> the new scheme eventually approaches roughly a factor
> of ten between the number of objects and the number of
> times that GC considers an object.
>
> Applications with a small number of objects will see no
> change in behavior, also, applications that create
> lots of small cycles will continue to see them collected
> in the youngest or middle generations.
>
> Please let me know what you think.
>
> Regards,
> Martin
>
> P.S. I don't plan to implement this myself, so if you like
> the idea, go ahead.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/adlaiff6%40gmail.com
>
>



--
Cheers,
Leif

From leif.walsh at gmail.com  Sat Jun 21 23:45:55 2008
From: leif.walsh at gmail.com (Leif Walsh)
Date: Sat, 21 Jun 2008 14:45:55 -0700
Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6b1 and 3.0b1
In-Reply-To: <1972109D-735D-4485-82F4-9BC8F2984967@python.org>
References: <1972109D-735D-4485-82F4-9BC8F2984967@python.org>
Message-ID: <cc7430500806211445n52252bceq615e0220d0f1fc15@mail.gmail.com>

There is a typo on the download page (http://python.org/download/releases/3.0/):

"""The release plan is to have a series of alpha releases in 2007,
beta releases in 2008, and a final release in September 2008. The
alpha releases are primarily aimed at developers who want a sneak peek
at the new langauge, especially those folks who plan to port their
code to Python 3000. The hope is that by the time of the final
release, many 3rd party packages will already be available in a
3.0-compatible form.
"""

Should be

"""The release plan is to have a series of alpha releases in 2007,
beta releases in 2008, and a final release in September 2008. The
alpha releases are primarily aimed at developers who want a sneak peek
at the new *language*, especially those folks who plan to port their
code to Python 3000. The hope is that by the time of the final
release, many 3rd party packages will already be available in a
3.0-compatible form.
"""

-- 
Cheers,
Leif

From adlaiff6 at gmail.com  Sat Jun 21 23:48:05 2008
From: adlaiff6 at gmail.com (Leif Walsh)
Date: Sat, 21 Jun 2008 14:48:05 -0700
Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6b1 and 3.0b1
In-Reply-To: <1972109D-735D-4485-82F4-9BC8F2984967@python.org>
References: <1972109D-735D-4485-82F4-9BC8F2984967@python.org>
Message-ID: <cc7430500806211448k5f25bdf3i458c7cb86bcba6b9@mail.gmail.com>

There is a typo on the download page (http://python.org/download/releases/3.0/):

"""The release plan is to have a series of alpha releases in 2007,
beta releases in 2008, and a final release in September 2008. The
alpha releases are primarily aimed at developers who want a sneak peek
at the new langauge, especially those folks who plan to port their
code to Python 3000. The hope is that by the time of the final
release, many 3rd party packages will already be available in a
3.0-compatible form.
"""

Should be

"""The release plan is to have a series of alpha releases in 2007,
beta releases in 2008, and a final release in September 2008. The
alpha releases are primarily aimed at developers who want a sneak peek
at the new *language*, especially those folks who plan to port their
code to Python 3000. The hope is that by the time of the final
release, many 3rd party packages will already be available in a
3.0-compatible form.
"""

(I really need to add my other e-mail address to these lists...sorry
if my first message got through to one of them and this is a dupe.)

--
Cheers,
Leif

From barry at python.org  Sun Jun 22 00:55:33 2008
From: barry at python.org (Barry Warsaw)
Date: Sat, 21 Jun 2008 18:55:33 -0400
Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6b1 and 3.0b1
In-Reply-To: <cc7430500806211445n52252bceq615e0220d0f1fc15@mail.gmail.com>
References: <1972109D-735D-4485-82F4-9BC8F2984967@python.org>
	<cc7430500806211445n52252bceq615e0220d0f1fc15@mail.gmail.com>
Message-ID: <ACE6F895-8D64-4703-B3ED-7D5BC850A639@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 21, 2008, at 5:45 PM, Leif Walsh wrote:

> There is a typo on the download page (http://python.org/download/releases/3.0/ 
> ):

Fixed, thanks!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSF2G5nEjvBPtnXfVAQJbqwQAkEMtWMYdsMFWnGmQCrpEV3BXZGipmZQs
7dnWoJIVsjN7pBY/5aVMiQEl5JM3n8IdVpwZMWM9Q+VHgkB04u8f7/wS2xmR7XsA
dySDMrxUJXqXX1ze/vWKosp39Yvey6HeTsiOUzvnXjS6IKM757IhyB94o4KQF69K
mOS8tjydwE8=
=cY+d
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Sun Jun 22 01:00:41 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 22 Jun 2008 01:00:41 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <87bq1umpxb.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>	<485AFBAB.2060404@canterbury.ac.nz>	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>	<loom.20080620T142315-395@post.gmane.org>	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	<1214012386.6402.4.camel@fsol>	<485CBCDA.4020601@v.loewis.de>	<1214054311.5874.12.camel@fsol>	<485D1C27.60408@v.loewis.de>
	<87bq1umpxb.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <485D8819.1040708@v.loewis.de>

> XEmacs implements this strategy in a way which is claimed to give
> constant amortized time (ie, averaged over memory allocated).

See my recent proposal. The old trick is to do reorganizations
in a fixed fraction of the total size, resulting in a per-increase
amortized-constant overhead (assuming each reorganization takes time
linear with total size).

> However, isn't the real question whether there is memory pressure or
> not?  If you've got an unloaded machine with 2GB of memory, even a 1GB
> spike might have no observable consequences.  How about a policy of
> GC-ing with decreasing period ("time" measured by bytes allocated or
> number of allocations) as the fraction of memory used increases,
> starting from a pretty large fraction (say 50% by default)?

The problem with such an approach is that it is very difficult to
measure. What to do about virtual memory? What to do about other
applications that also consume memory?

On some systems (Windows in particular), the operating system indicates
memory pressure through some IPC mechanism; on such systems, it might
be reasonable to perform garbage collection (only?) when the system asks
for it. However, the system might not ask for GC while the swap space
is still not exhausted, meaning that the deferred GC would take a long
time to complete (having to page in every object).

> Nevertheless, I think the real solution has to be for Python
> programmers to be aware that there is GC, and that they can tune it.

I don't think there is a "real solution". I think programmers should
abstain from complaining if they can do something about the problem
in their own application (unless the complaint is formulated as a
patch) - wait - I think programmers should abstain from complaining,
period.

Regards,
Martin


From greg.ewing at canterbury.ac.nz  Sun Jun 22 03:16:59 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 22 Jun 2008 13:16:59 +1200
Subject: [Python-Dev] Proposal: Run GC less often
In-Reply-To: <485D6334.6080102@v.loewis.de>
References: <485D6334.6080102@v.loewis.de>
Message-ID: <485DA80B.2080404@canterbury.ac.nz>

Martin v. L?wis wrote:

> Under my proposal, 10 middle collections must have passed,
> PLUS the number of survivor objects from the middle generation
> must exceed 10% of the number of objects in the oldest
> generation.

What happens if the program enters a phase where it's not
producing any new cyclic garbage, but is breaking references
among the old objects in such a way that cycles of them
are being left behind? Under this rule, the oldest
generation would never be scanned, so those cycles would
never be collected.

> As a consequence, garbage collection becomes less frequent
> as the number of objects on the heap grows

Wouldn't it be simpler just to base the collection frequency
directly on the total number of objects in the heap?
 From what another poster said, this seems to be what
emacs does.

-- 
Greg

From ncoghlan at gmail.com  Sun Jun 22 03:56:39 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 22 Jun 2008 11:56:39 +1000
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <485D5CFC.5020705@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<485AFBAB.2060404@canterbury.ac.nz>	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>	<loom.20080620T142315-395@post.gmane.org>	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	<1214012386.6402.4.camel@fsol>	<485CBCDA.4020601@v.loewis.de>	<1214054311.5874.12.camel@fsol>	<485D1C27.60408@v.loewis.de>	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>	<485D231D.9030806@v.loewis.de>	<1214075410.5874.26.camel@fsol>
	<485D5CFC.5020705@v.loewis.de>
Message-ID: <485DB157.1080907@gmail.com>

Martin v. L?wis wrote:
> Antoine Pitrou wrote:
>> Le samedi 21 juin 2008 ? 17:49 +0200, "Martin v. L?wis" a ?crit : 
>>> I don't think any strategies based on timing will be successful.
>>> Instead, one should count and analyze objects (although I'm unsure
>>> how exactly that could work).
>> Would it be helpful if the GC was informed of memory growth by the
>> Python memory allocator (that is, each time it either asks or gives back
>> a block of memory to the system allocator) ?
> 
> I don't see how. The garbage collector is already informed about memory
> growth; it learns exactly when a container object is allocated or
> deallocated. That the allocator then requests memory from the system
> only confirms what the garbage collector already knew: that there are
> lots of allocated objects. From that, one could infer that it might
> be time to perform garbage collection - or one could infer that all
> the objects are really useful, and no garbage can be collected.

I was wondering whether it might be useful to detect the end of an 
allocation spike: if PyMalloc incremented an internal counter each time 
it requested more memory from the system, then the GC could check 
whether that number had changed on each collection cycle. If the GC goes 
a certain number of collection cycles without more memory being 
requested from the system, then it can assume the allocation spike is 
over and tighten up the default tuning again.

Such a pymalloc counter would provide a slightly more holistic view of 
overall memory usage changes than the container-focused view of the 
cyclic GC.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From tonynelson at georgeanelson.com  Sun Jun 22 04:40:48 2008
From: tonynelson at georgeanelson.com (Tony Nelson)
Date: Sat, 21 Jun 2008 22:40:48 -0400
Subject: [Python-Dev] Another Proposal: Run GC less often
In-Reply-To: <485d7277.0b38560a.065a.ffffcada@mx.google.com>
References: <485d7277.0b38560a.065a.ffffcada@mx.google.com>
Message-ID: <p04330100c4836a290835@[192.168.123.162]>

At 11:28 PM +0200 6/21/08, none wrote:
>Instead of collecting objects after a fixed number of allocations (700)
 ...

I've seen this asserted several times in this thread:  that GC is done
every fixed number of allocations.  This is not correct.  GC is done when
the surplus of allocations less deallocations exceeds a threashold.  See
Modules/gcmodule.c and look for ".count++" and ".count--".  In normal
operation, allocations and deallocations stay somewhat balanced, but when
creating a large data structure, it's allocations all the way and GC runs
often.
-- 
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson at georgeanelson.com>
      '                              <http://www.georgeanelson.com/>

From stephen at xemacs.org  Sun Jun 22 05:43:26 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 22 Jun 2008 12:43:26 +0900
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <485D8819.1040708@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<aac2c7cb0805312128r7710a4ck4c2e3f29852503f8@mail.gmail.com>
	<acd65fa20806191423w6cd40533m67244967d38a6d1b@mail.gmail.com>
	<485AFBAB.2060404@canterbury.ac.nz>
	<2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com>
	<loom.20080620T142315-395@post.gmane.org>
	<2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
	<1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de>
	<1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de>
	<87bq1umpxb.fsf@uwakimon.sk.tsukuba.ac.jp>
	<485D8819.1040708@v.loewis.de>
Message-ID: <877icim8wx.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:
 > > XEmacs implements this strategy in a way which is claimed to give
 > > constant amortized time (ie, averaged over memory allocated).
 > 
 > See my recent proposal.

I did, crossed in the mail.  To the extent that I understand both
systems, your proposal looks like an improvement over what we've got.

 > > However, isn't the real question whether there is memory pressure or
 > > not?  If you've got an unloaded machine with 2GB of memory, even a 1GB
 > > spike might have no observable consequences.  How about a policy of
 > > GC-ing with decreasing period ("time" measured by bytes allocated or
 > > number of allocations) as the fraction of memory used increases,
 > > starting from a pretty large fraction (say 50% by default)?
 > 
 > The problem with such an approach is that it is very difficult to
 > measure.

On some systems it should be possible to get information about how
much paging is taking place, which would be an indicator of pressure.

 > What to do about virtual memory?

If you're in virtual memory, you're already in trouble.  Once you're
paging, you need to increase time between GCs, so the rule would not
be monotonic.

From martin at v.loewis.de  Sun Jun 22 07:13:15 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 22 Jun 2008 07:13:15 +0200
Subject: [Python-Dev] Proposal: Run GC less often
In-Reply-To: <485DA80B.2080404@canterbury.ac.nz>
References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz>
Message-ID: <485DDF6B.9050401@v.loewis.de>

> What happens if the program enters a phase where it's not
> producing any new cyclic garbage, but is breaking references
> among the old objects in such a way that cycles of them
> are being left behind? Under this rule, the oldest
> generation would never be scanned, so those cycles would
> never be collected.

Correct. However, this is the case already today.

> Wouldn't it be simpler just to base the collection frequency
> directly on the total number of objects in the heap?

Using what precise formula?

Regards,
Martin

From facundobatista at gmail.com  Sun Jun 22 15:05:47 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Sun, 22 Jun 2008 10:05:47 -0300
Subject: [Python-Dev] Issue 2722
In-Reply-To: <a4cf72520806210316w62c9f45r5a7f04e80bdc92f5@mail.gmail.com>
References: <a4cf72520806210316w62c9f45r5a7f04e80bdc92f5@mail.gmail.com>
Message-ID: <e04bdf310806220605g1e61931fj4c95e55576ca535e@mail.gmail.com>

2008/6/21 Neil Muller <drnlmuller+python at gmail.com>:

> Could some one have a look at the suggested fix for issue 2722? While
> not a particularly common problem, it can be an annoyance, and the
> patch looks reasonable.

I'm on it... Python Bug Day!

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From solipsis at pitrou.net  Sun Jun 22 16:32:09 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 22 Jun 2008 14:32:09 +0000 (UTC)
Subject: [Python-Dev] Proposal: Run GC less often
References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz>
Message-ID: <loom.20080622T142129-876@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> What happens if the program enters a phase where it's not
> producing any new cyclic garbage, but is breaking references
> among the old objects in such a way that cycles of them
> are being left behind? Under this rule, the oldest
> generation would never be scanned, so those cycles would
> never be collected.

We could introduce a kind of "timing rule" such that there is at least
one full collection, say, every minute. While timing is not relevant
to memory management, it is relevant to the user behind the keyboard.

In any case, I think MvL's suggestion is worth trying.

Regards

Antoine.



From nas at arctrix.com  Sun Jun 22 21:27:27 2008
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 22 Jun 2008 19:27:27 +0000 (UTC)
Subject: [Python-Dev] Proposal: Run GC less often
References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz>
Message-ID: <g3m92u$qsc$1@ger.gmane.org>

Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Martin v. L?wis wrote:
>
>> Under my proposal, 10 middle collections must have passed,
>> PLUS the number of survivor objects from the middle generation
>> must exceed 10% of the number of objects in the oldest
>> generation.
>
> What happens if the program enters a phase where it's not
> producing any new cyclic garbage, but is breaking references
> among the old objects in such a way that cycles of them
> are being left behind? Under this rule, the oldest
> generation would never be scanned, so those cycles would
> never be collected.

Another problem is that the program could be slowly leaking and a
full collection will never happen.

>> As a consequence, garbage collection becomes less frequent
>> as the number of objects on the heap grows
>
> Wouldn't it be simpler just to base the collection frequency
> directly on the total number of objects in the heap?
>  From what another poster said, this seems to be what
> emacs does.

I like simple.  The whole generational collection scheme was dreamed
up by me early in the GC's life.  There was not a lot of thought or
benchmarking put into it since at that time I was more focused on
getting the basic GC working.  At some later point some tuning was
done on the collection frequencies but that 10 middle collections
scheme was never deeply investigated, AFAIK.

BTW, I suspect that documentation needs updating since I understand
that the GC is no longer optional (the stdlib and/or the Python
internals create reference cycles themselves).

  Neil


From tomerfiliba at gmail.com  Sun Jun 22 21:28:13 2008
From: tomerfiliba at gmail.com (tomer filiba)
Date: Sun, 22 Jun 2008 12:28:13 -0700 (PDT)
Subject: [Python-Dev] forceful exit
Message-ID: <07e859dc-c9cf-476f-8a9a-85f4f990de79@2g2000hsn.googlegroups.com>

hi

i'm having trouble when forking child processes to serve sockets. the
skeleton is something like the following:

def work():
    try:
        while True:
            s = listener.accept()[0]
            log("hello %s", s)
            if os.fork() == 0:
                try:
                    serve_client(sock)
                finally:
                    sys.exit()   #### <<<< (1)
            else:
                log("forked child")
                sock.close()
    except KeyboardInterrupt:
        log("got ctrl+c")
    finally:
        log("server terminated")
        listener.close()

the problem is that sys.exit() raises an exception, which propagates
all the way up and is handled by the finallies... the real code does
more than just printing logs, so i can't allow that.

i'm forced to resort to os._exit, which does the trick but doesn't
perform cleanup, which made me realize there's no clean way in python
to *force* exit.

i think there ought to be something like sys.forcedexit(), which
collects all objects nicely and then exits immediately without letting
anything propagate up. this will also solve another problem i came
across, with threads. turns out only the main thread can kill the
process -- if another thread issues sys.exit, it only kills itself.
there's no clean way for a thread to terminate the process... i think
there must be.

i can contribute a patch in a matter of days... i think it's pretty
straight forward (calling Py_Finalize and then exit). aye or nay?


-tomer

From martin at v.loewis.de  Sun Jun 22 22:36:19 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 22 Jun 2008 22:36:19 +0200
Subject: [Python-Dev] Proposal: Run GC less often
In-Reply-To: <g3m92u$qsc$1@ger.gmane.org>
References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz>
	<g3m92u$qsc$1@ger.gmane.org>
Message-ID: <485EB7C3.9060602@v.loewis.de>

> Another problem is that the program could be slowly leaking and a
> full collection will never happen.

I don't think that will be possible. If the program slowly leaks,
survivor objects leave the middle generation, and account towards
the 10%. As the count of objects in the oldest generation doesn't
change, collection will eventually occur.

However, it may occur much later than it currently does, if you have
many objects on the heap, and each middle collection only has few
survivors. One may argue that if the machine had the space to keep
N objects in memory, it probably can also keep 1.1N objects in memory.

Regards,
Martin

From tim.peters at gmail.com  Sun Jun 22 23:10:52 2008
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 22 Jun 2008 17:10:52 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <485D5CFC.5020705@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>
	<1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de>
	<1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de>
	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>
	<485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol>
	<485D5CFC.5020705@v.loewis.de>
Message-ID: <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>

[Antoine Pitrou]
>> Would it be helpful if the GC was informed of memory growth by the
>> Python memory allocator (that is, each time it either asks or gives back
>> a block of memory to the system allocator) ?

[Martin v. L?wis]
> I don't see how. The garbage collector is already informed about memory
> growth; it learns exactly when a container object is allocated or
> deallocated. That the allocator then requests memory from the system
> only confirms what the garbage collector already knew: that there are
> lots of allocated objects. From that, one could infer that it might
> be time to perform garbage collection - or one could infer that all
> the objects are really useful, and no garbage can be collected.

Really the same conundrum we currently face:  cyclic gc is currently
triggered by reaching a certain /excess/ of allocations over
deallocations.  From that we /do/ infer it's time to perform garbage
collection -- but, as some examples here showed, it's sometimes really
the case that the true meaning of the excess is that "all the objects
are really useful, and no garbage can be collected -- and I'm creating
a lot of them".

pymalloc needing to allocate a new arena would be a different way to
track an excess of allocations over deallocations, and in some ways
more sensible (since it would reflect an excess of /bytes/ allocated
over bytes freed, rather than an excess in the counts of objects
allocated-over-freed regardless of their sizes -- an implication is,
e.g., that cyclic gc would be triggered much less frequently by mass
creation of small tuples than of small dicts, since a small tuple
consumes much less memory than a small dict).

Etc. ;-)

From greg.ewing at canterbury.ac.nz  Mon Jun 23 03:10:51 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 23 Jun 2008 13:10:51 +1200
Subject: [Python-Dev] Proposal: Run GC less often
In-Reply-To: <485DDF6B.9050401@v.loewis.de>
References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz>
	<485DDF6B.9050401@v.loewis.de>
Message-ID: <485EF81B.9080901@canterbury.ac.nz>

Martin v. L?wis wrote:

>>Wouldn't it be simpler just to base the collection frequency
>>directly on the total number of objects in the heap?
> 
> Using what precise formula?

The simplest thing to try would be

  middle_collections >= num_objects_in_heap * some_constant

-- 
Greg

From martin at v.loewis.de  Mon Jun 23 05:32:01 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 23 Jun 2008 05:32:01 +0200
Subject: [Python-Dev] Proposal: Run GC less often
In-Reply-To: <485EF81B.9080901@canterbury.ac.nz>
References: <485D6334.6080102@v.loewis.de>
	<485DA80B.2080404@canterbury.ac.nz>	<485DDF6B.9050401@v.loewis.de>
	<485EF81B.9080901@canterbury.ac.nz>
Message-ID: <485F1931.7000804@v.loewis.de>

>>> Wouldn't it be simpler just to base the collection frequency
>>> directly on the total number of objects in the heap?
>>
>> Using what precise formula?
> 
> The simplest thing to try would be
> 
>  middle_collections >= num_objects_in_heap * some_constant

So what value is some_constant?

Regards,
Martin


From martin at v.loewis.de  Mon Jun 23 05:39:06 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Jun 2008 05:39:06 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	<1214012386.6402.4.camel@fsol>
	<485CBCDA.4020601@v.loewis.de>	<1214054311.5874.12.camel@fsol>
	<485D1C27.60408@v.loewis.de>	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>	<485D231D.9030806@v.loewis.de>
	<1214075410.5874.26.camel@fsol>	<485D5CFC.5020705@v.loewis.de>
	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>
Message-ID: <485F1ADA.6090107@v.loewis.de>

> pymalloc needing to allocate a new arena would be a different way to
> track an excess of allocations over deallocations, and in some ways
> more sensible (since it would reflect an excess of /bytes/ allocated
> over bytes freed, rather than an excess in the counts of objects
> allocated-over-freed regardless of their sizes -- an implication is,
> e.g., that cyclic gc would be triggered much less frequently by mass
> creation of small tuples than of small dicts, since a small tuple
> consumes much less memory than a small dict).
> 
> Etc. ;-)

:-) So my question still is: how exactly?

Currently, only youngest collections are triggered by allocation
rate; middle and old are triggered by frequency of youngest collection.
So would you now specify that the youngest collection should occur
if-and-only-if a new arena is allocated? Or discount arenas returned
from arenas allocated? Or apply this to triggering other generation
collections but youngest? How would that help the quadratic behavior
(which really needs to apply a factor somewhere)?

Regards,
Martin

From tjreedy at udel.edu  Mon Jun 23 08:13:55 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 23 Jun 2008 02:13:55 -0400
Subject: [Python-Dev] Proposal: Run GC less often
In-Reply-To: <g3m92u$qsc$1@ger.gmane.org>
References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz>
	<g3m92u$qsc$1@ger.gmane.org>
Message-ID: <g3neut$klc$1@ger.gmane.org>



Neil Schemenauer wrote:
>
> BTW, I suspect that documentation needs updating since I understand
> that the GC is no longer optional (the stdlib and/or the Python
> internals create reference cycles themselves).

Is it possible and might it be useful for those internal cycle-creating 
operations to increment a counter that was part of the gc trigger? 
Doing millions of 'safe' operations would then leave the counter alone 
and could have less effect in triggering gc.

tjr


From michael at araneidae.co.uk  Mon Jun 23 09:02:52 2008
From: michael at araneidae.co.uk (Michael Abbott)
Date: Mon, 23 Jun 2008 07:02:52 +0000 (GMT)
Subject: [Python-Dev] Suggested API change to PyOS_InputHook
Message-ID: <20080623064601.M29284@saturn.araneidae.co.uk>

I have a problem with the PyOS_InputHook() API as implemented by 
Modules/readline.c: there is no way to communicate any interruption seen 
while waiting at the keyboard back to the process actually reading the 
line.

I have solved this in my own application by creating a new module which 
reimplements the call_readline function and which has the following patch 
to readline_until_enter_or_signal():


                        FD_SET(fileno(rl_instream), &selectset);
                        /* select resets selectset if no input was available */
                        has_input = select(fileno(rl_instream) + 1, &selectset,
                                           NULL, NULL, &timeout);
-                       if(PyOS_InputHook) PyOS_InputHook();
+            if (call_readline_InputHook  &&
+                call_readline_InputHook(fileno(rl_instream)))
+            {
+                has_input = -1;
+                PyErr_SetInterrupt();
+                break;
+            }
                }

                if(has_input > 0) {
                        rl_callback_read_char();
                }
-               else if (errno == EINTR) {
+        else if (errno == EINTR || has_input < 0) {
                        int s;
 #ifdef WITH_THREAD
                        PyEval_RestoreThread(_PyOS_ReadlineTState);


(I'm sorry, this isn't a proper patch, but hopefully if you've read this 
far you can figure out where it applies.)  

Here I've replaced PyOS_InputHook with effective type signature None->None 
with my own call_readline_InputHook which takes a file descriptor and 
returns a boolean, returning True iff an interrupt occurred.  The file 
descriptor argument is a triviality, but the boolean return code is 
clearly crucial.

Now, I don't know how this change might fit into mainstream Python, but I 
think there is a case for making this change.  If the incompatibility was 
acceptable it would be better to change the signature of PyOS_InputHook (I 
declared my own variable call_readline_InputHook just to avoid accidents).  
I've checked the current svn trunk, and this change remains applicable.

Please CC me in any replies: I am quite literally unable to cope with the 
volume of traffic that a subscription to python-dev will give me.

From solipsis at pitrou.net  Mon Jun 23 12:40:26 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 23 Jun 2008 10:40:26 +0000 (UTC)
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	<1214012386.6402.4.camel@fsol>
	<485CBCDA.4020601@v.loewis.de>	<1214054311.5874.12.camel@fsol>
	<485D1C27.60408@v.loewis.de>	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>	<485D231D.9030806@v.loewis.de>
	<1214075410.5874.26.camel@fsol>	<485D5CFC.5020705@v.loewis.de>
	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>
	<485F1ADA.6090107@v.loewis.de>
Message-ID: <loom.20080623T102812-837@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> Currently, only youngest collections are triggered by allocation
> rate; middle and old are triggered by frequency of youngest collection.
> So would you now specify that the youngest collection should occur
> if-and-only-if a new arena is allocated? Or discount arenas returned
> from arenas allocated?

The latter sounds reasonable. IIRC an arena is 256KB, which is less than an 
entry level L2 cache. Therefore waiting for an arena to be filled shouldn't 
deteriorate cache locality a lot.

To avoid situations where the GC is never called we could combine that with an 
allocation counter, but with a much higher threshold than currently.

> Or apply this to triggering other generation
> collections but youngest? How would that help the quadratic behavior
> (which really needs to apply a factor somewhere)?

It would not help the quadratic behaviour - and is orthogonal to your proposal -
, but at least avoid calling the GC too often when lots of small objects are 
allocated (as opposed to lots of large objects).




From aahz at pythoncraft.com  Mon Jun 23 16:54:52 2008
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 23 Jun 2008 07:54:52 -0700
Subject: [Python-Dev] Suggested API change to PyOS_InputHook
In-Reply-To: <20080623064601.M29284@saturn.araneidae.co.uk>
References: <20080623064601.M29284@saturn.araneidae.co.uk>
Message-ID: <20080623145452.GB13722@panix.com>

On Mon, Jun 23, 2008, Michael Abbott wrote:
>
> Please CC me in any replies: I am quite literally unable to cope with the 
> volume of traffic that a subscription to python-dev will give me.

Then don't bother posting code to the list.  Please post a patch; up to
you whether you send a notice to python-dev.  And expect that you'll
probably miss at least some replies.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha

From michael at araneidae.co.uk  Mon Jun 23 18:34:02 2008
From: michael at araneidae.co.uk (Michael Abbott)
Date: Mon, 23 Jun 2008 16:34:02 +0000 (GMT)
Subject: [Python-Dev] Suggested API change to PyOS_InputHook
In-Reply-To: <20080623145452.GB13722@panix.com>
References: <20080623064601.M29284@saturn.araneidae.co.uk>
	<20080623145452.GB13722@panix.com>
Message-ID: <20080623163312.S32297@saturn.araneidae.co.uk>

On Mon, 23 Jun 2008, Aahz wrote:
> Then don't bother posting code to the list.  Please post a patch; up to
> you whether you send a notice to python-dev.  

Very well.  Issue 3180 created.  I hope I'm doing it right...

From tjreedy at udel.edu  Mon Jun 23 20:52:44 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 23 Jun 2008 14:52:44 -0400
Subject: [Python-Dev] Suggested API change to PyOS_InputHook
In-Reply-To: <20080623064601.M29284@saturn.araneidae.co.uk>
References: <20080623064601.M29284@saturn.araneidae.co.uk>
Message-ID: <g3ords$c6q$1@ger.gmane.org>



Michael Abbott wrote:

> Please CC me in any replies: I am quite literally unable to cope with the 
> volume of traffic that a subscription to python-dev will give me.

You can read python-dev (and 1000s of other mail lists, with a 
newsreader, like I do, for the same reason, via the free list to news 
gateway news.gmane.com

tjr


From martin at v.loewis.de  Mon Jun 23 23:08:29 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 23 Jun 2008 23:08:29 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <loom.20080623T102812-837@post.gmane.org>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	<1214012386.6402.4.camel@fsol>	<485CBCDA.4020601@v.loewis.de>	<1214054311.5874.12.camel@fsol>	<485D1C27.60408@v.loewis.de>	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>	<485D231D.9030806@v.loewis.de>	<1214075410.5874.26.camel@fsol>	<485D5CFC.5020705@v.loewis.de>	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>	<485F1ADA.6090107@v.loewis.de>
	<loom.20080623T102812-837@post.gmane.org>
Message-ID: <486010CD.2000904@v.loewis.de>

> It would not help the quadratic behaviour - and is orthogonal to your proposal -
> , but at least avoid calling the GC too often when lots of small objects are 
> allocated (as opposed to lots of large objects).

I'd like to see in an experiment whether this is really true. Current GC
only takes into account container objects, which, most significantly,
ignores string objects (of which most applications create plenty). If
you make actual memory consumption a trigger, the string objects would
count towards triggering GC. That may or may not be the better approach,
anyway - but it's not certain (for me) that such a scheme would cause GC
to be triggered less often, in a typical application.

Regards,
Martin

From solipsis at pitrou.net  Tue Jun 24 11:36:13 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 24 Jun 2008 09:36:13 +0000 (UTC)
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<e27efe130806200844i3156a59cpf05931f66d967d44@mail.gmail.com>	<1214012386.6402.4.camel@fsol>	<485CBCDA.4020601@v.loewis.de>	<1214054311.5874.12.camel@fsol>	<485D1C27.60408@v.loewis.de>	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>	<485D231D.9030806@v.loewis.de>	<1214075410.5874.26.camel@fsol>	<485D5CFC.5020705@v.loewis.de>	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>	<485F1ADA.6090107@v.loewis.de>
	<loom.20080623T102812-837@post.gmane.org>
	<486010CD.2000904@v.loewis.de>
Message-ID: <loom.20080624T093235-422@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> I'd like to see in an experiment whether this is really true.

Right, all those ideas should be implemented and tried out. I don't really have 
time to spend on it right now.

Also, what's missing is a suite of performance/efficiency tests for the garbage
collector.




From hsoft at hardcoded.net  Tue Jun 24 12:18:37 2008
From: hsoft at hardcoded.net (Virgil Dupras)
Date: Tue, 24 Jun 2008 12:18:37 +0200
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
Message-ID: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>

Hi Devs,

Today, I saw the Py3k beta 1 announcement, so I thought "Well, I gotta
do my duty and try to migrate my own code base to see if I can uncover
any problems". I'm just at the step where I run my test suite with the
"-3" flag, and I noticed something peculiar: I get DeprecationWarning
from stdlib code (namely, difflib, logging and unittest). Is it
something to be expected or should I create tickets for those? I'm
sure that the stdlib on the 3k side is fixed, but shouldn't the 2.6
side be fixed as well?

Sorry if this question is already answered somewhere else, I searched
for an answer but I couldn't find it.

Additionally, one thing that strikes me as strange is how hard it is
to find a "Migration guide" on the website. I had to dig in the PEP
3000 to find the brief bullet list telling me to run py26 with the -3
flag.

Virgil Dupras

From ncoghlan at gmail.com  Tue Jun 24 12:34:23 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Jun 2008 20:34:23 +1000
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>
References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>
Message-ID: <4860CDAF.2020806@gmail.com>

Virgil Dupras wrote:
> Hi Devs,
> 
> Today, I saw the Py3k beta 1 announcement, so I thought "Well, I gotta
> do my duty and try to migrate my own code base to see if I can uncover
> any problems". I'm just at the step where I run my test suite with the
> "-3" flag, and I noticed something peculiar: I get DeprecationWarning
> from stdlib code (namely, difflib, logging and unittest). Is it
> something to be expected or should I create tickets for those? I'm
> sure that the stdlib on the 3k side is fixed, but shouldn't the 2.6
> side be fixed as well?

I'd say so, yes. If tracker issues don't exist yet, please create them. 
The focus for the first beta was really on getting all the syntax and 
API changes stable - the remaining time between the first beta release 
and the first release candidate will focus on cleaning up issues like this.

> Sorry if this question is already answered somewhere else, I searched
> for an answer but I couldn't find it.
> 
> Additionally, one thing that strikes me as strange is how hard it is
> to find a "Migration guide" on the website. I had to dig in the PEP
> 3000 to find the brief bullet list telling me to run py26 with the -3
> flag.

Probably because nobody has stepped up to write a migration HOWTO yet...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From hsoft at hardcoded.net  Tue Jun 24 12:43:36 2008
From: hsoft at hardcoded.net (Virgil Dupras)
Date: Tue, 24 Jun 2008 12:43:36 +0200
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <4860CDAF.2020806@gmail.com>
References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>
	<4860CDAF.2020806@gmail.com>
Message-ID: <2a70578d0806240343i5aa9ef53ye831e27d4e81baf2@mail.gmail.com>

On Tue, Jun 24, 2008 at 12:34 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Probably because nobody has stepped up to write a migration HOWTO yet...

I'm not much of a writer, but once I'm finished with the migration,
I'll do it (if it hasn't been done then). It's about time I try to
learn reST...

Virgil Dupras

From fuzzyman at voidspace.org.uk  Tue Jun 24 12:50:16 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 24 Jun 2008 11:50:16 +0100
Subject: [Python-Dev] Python Windows Installers
Message-ID: <4860D168.9050305@voidspace.org.uk>

Hello all,

I'm just doing some housekeeping on a Windows install, and notice that 
the 'Publisher' of my Python 2.4 and 2.5 installs is shown as "Martin v. 
Lowis". Whilst I *personally* find this very reassuring I wonder if this 
is intended / ideal?

All the best,

Michael Foord

From fuzzyman at voidspace.org.uk  Tue Jun 24 12:52:19 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 24 Jun 2008 11:52:19 +0100
Subject: [Python-Dev] Python Windows Installers
In-Reply-To: <4860D168.9050305@voidspace.org.uk>
References: <4860D168.9050305@voidspace.org.uk>
Message-ID: <4860D1E3.3090906@voidspace.org.uk>

Michael Foord wrote:
> Hello all,
>
> I'm just doing some housekeeping on a Windows install, and notice that 
> the 'Publisher' of my Python 2.4 and 2.5 installs is shown as "Martin 
> v. Lowis". Whilst I *personally* find this very reassuring I wonder if 
> this is intended / ideal?
>
Just to clarify, this is on Windows Vista and shown in the Control Panel 
"Programs and Features" (what used to be "Add/Remove Programs"). 
Installers were from Python.org.

Michael Foord


> All the best,
>
> Michael Foord
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


From musiccomposition at gmail.com  Tue Jun 24 14:58:05 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 24 Jun 2008 07:58:05 -0500
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <4860CDAF.2020806@gmail.com>
References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>
	<4860CDAF.2020806@gmail.com>
Message-ID: <1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com>

On Tue, Jun 24, 2008 at 5:34 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I'd say so, yes. If tracker issues don't exist yet, please create them. The
> focus for the first beta was really on getting all the syntax and API
> changes stable - the remaining time between the first beta release and the
> first release candidate will focus on cleaning up issues like this.

I wonder if we could just 2to3 (with fixers that create
incompatibilities disabled) over the whole stdlib and see what
happens.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From ncoghlan at gmail.com  Tue Jun 24 15:08:47 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Jun 2008 23:08:47 +1000
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com>
References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>	
	<4860CDAF.2020806@gmail.com>
	<1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com>
Message-ID: <4860F1DF.5080204@gmail.com>

Benjamin Peterson wrote:
> On Tue, Jun 24, 2008 at 5:34 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I'd say so, yes. If tracker issues don't exist yet, please create them. The
>> focus for the first beta was really on getting all the syntax and API
>> changes stable - the remaining time between the first beta release and the
>> first release candidate will focus on cleaning up issues like this.
> 
> I wonder if we could just 2to3 (with fixers that create
> incompatibilities disabled) over the whole stdlib and see what
> happens.

Just running the unit test suite with -3 would probably be a better 
starting point. 2to3 makes a lot of extra changes that we don't 
particularly care about.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From musiccomposition at gmail.com  Tue Jun 24 15:11:00 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 24 Jun 2008 08:11:00 -0500
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <4860F1DF.5080204@gmail.com>
References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>
	<4860CDAF.2020806@gmail.com>
	<1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com>
	<4860F1DF.5080204@gmail.com>
Message-ID: <1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com>

On Tue, Jun 24, 2008 at 8:08 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Benjamin Peterson wrote:
>>
>> On Tue, Jun 24, 2008 at 5:34 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>
>>> I'd say so, yes. If tracker issues don't exist yet, please create them.
>>> The
>>> focus for the first beta was really on getting all the syntax and API
>>> changes stable - the remaining time between the first beta release and
>>> the
>>> first release candidate will focus on cleaning up issues like this.
>>
>> I wonder if we could just 2to3 (with fixers that create
>> incompatibilities disabled) over the whole stdlib and see what
>> happens.
>
> Just running the unit test suite with -3 would probably be a better starting
> point. 2to3 makes a lot of extra changes that we don't particularly care
> about.

Right, which is why I suggest running a few fixers (ie fix_has_key)
and then tearing the test suite apart.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From ncoghlan at gmail.com  Tue Jun 24 15:17:50 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Jun 2008 23:17:50 +1000
Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:
 Include/object.h Lib/test/test_sys.py Misc/NEWS
 Objects/intobject.c	Objects/longobject.c Objects/typeobject.c
 Python/bltinmodule.c
In-Reply-To: <4860ED9F.8030305@trueblade.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4860ED9F.8030305@trueblade.com>
Message-ID: <4860F3FE.8090307@gmail.com>

Eric Smith wrote:
> Note that in 
> http://mail.python.org/pipermail/python-dev/2008-February/077062.html, 
> Guido advocates not adding the __bin__ machinery, which is what lead to 
> the simple implementation of bin() just calling PyNumber_ToBase and 
> relying on __index__.
> 
> I don't think __bin__ should be added to 2.6.  I don't see the point in 
> adding a feature in 2.6 that's implemented so differently in 3.0.  It's 
> just asking for porting hassles.
> 
> Instead, I think the approach used in 3.0 (r64451) should be used 
> instead.  That is, if this feature exist at all.  I'm -0 on adding 
> bin(), etc. to floats.

The 3.0 approach means that non-float floating point types still can't 
be displayed properly by bin()/oct()/hex(). The current 2.6 approach 
means every such class has to implement its own equivalent of 
PyNumber_ToBase. Both are lousy solutions to a relative non-problem 
(IMO) that can easily be implemented using a separate display function 
for those applications which happen to need it.

I'd prefer to see a proper PEP for this proposing a new slot that lets 
any class return an (integer_part, fraction_part) tuple of integers, and 
  have PyNumber_ToBase take care of the actual string formatting.

Introducing such a gratuitous incompatibility between 2.6 and 3.0 at 
this late stage of the process certainly seems highly undesirable to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From python at rcn.com  Tue Jun 24 15:56:37 2008
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 24 Jun 2008 06:56:37 -0700
Subject: [Python-Dev] Implmentation details for bin/oct/hex
References: <20080620041816.4D5E81E4002@bag.python.org><4860ED9F.8030305@trueblade.com>
	<4860F3FE.8090307@gmail.com>
Message-ID: <8A1B7D10023C4F259E63D96451590855@RaymondLaptop1>

From: "Nick Coghlan" <ncoghlan at gmail.com>
> The 3.0 approach means that non-float floating point types still can't be displayed properly by bin()/oct()/hex(). The current 2.6 
> approach means every such class has to implement its own equivalent of PyNumber_ToBase.

Feel free to change the implementation details however you like as long as the basic functionality stays the same.  To support 
non-float floating point types, replace the float check with isinstance(v, real).  The rest of PyNumber_ToBase looks fine to me 
since it supports anything with an __index__ method.

Raymond








From leif.walsh at gmail.com  Tue Jun 24 17:59:43 2008
From: leif.walsh at gmail.com (Leif Walsh)
Date: Tue, 24 Jun 2008 08:59:43 -0700 (PDT)
Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:
 Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c
 Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c
In-Reply-To: <4860F3FE.8090307@gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com>
Message-ID: <Pine.LNX.4.64.0806240854300.19276@lappy>

On Tue, 24 Jun 2008, Nick Coghlan wrote:
> I'd prefer to see a proper PEP for this proposing a new slot that lets any
> class return an (integer_part, fraction_part) tuple of integers, and
>  have PyNumber_ToBase take care of the actual string formatting.

I take issue only with your notion of returning a 'fraction_part
integer'.  How do you propose we do this?  I suggest we add a third
value, exponent, to the suggested return value.

-- 
Cheers,
Leif

From guido at python.org  Tue Jun 24 18:42:29 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Jun 2008 09:42:29 -0700
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com>
References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>
	<4860CDAF.2020806@gmail.com>
	<1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com>
	<4860F1DF.5080204@gmail.com>
	<1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com>
Message-ID: <ca471dc20806240942l571c647bx4b56e565eb5405ab@mail.gmail.com>

The same generic warning against sweeping changes applies here though.
You have to manually review each change. The stdlib and especially the
test suite is likely to break if you just let 2to3 run over it, even
just a single fixer like fix_has_key.

On Tue, Jun 24, 2008 at 6:11 AM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Tue, Jun 24, 2008 at 8:08 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Benjamin Peterson wrote:
>>>
>>> On Tue, Jun 24, 2008 at 5:34 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>>
>>>> I'd say so, yes. If tracker issues don't exist yet, please create them.
>>>> The
>>>> focus for the first beta was really on getting all the syntax and API
>>>> changes stable - the remaining time between the first beta release and
>>>> the
>>>> first release candidate will focus on cleaning up issues like this.
>>>
>>> I wonder if we could just 2to3 (with fixers that create
>>> incompatibilities disabled) over the whole stdlib and see what
>>> happens.
>>
>> Just running the unit test suite with -3 would probably be a better starting
>> point. 2to3 makes a lot of extra changes that we don't particularly care
>> about.
>
> Right, which is why I suggest running a few fixers (ie fix_has_key)
> and then tearing the test suite apart.
>
>
>
> --
> Cheers,
> Benjamin Peterson
> "There's no place like 127.0.0.1."
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Jun 24 18:56:41 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Jun 2008 09:56:41 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:
	Include/object.h Lib/test/test_sys.py Misc/NEWS
	Objects/intobject.c Objects/longobject.c Objects/typeobject.c
	Python/bltinmodule.c
In-Reply-To: <4860F3FE.8090307@gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com>
Message-ID: <ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com>

On Tue, Jun 24, 2008 at 6:17 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Eric Smith wrote:
>>
>> Note that in
>> http://mail.python.org/pipermail/python-dev/2008-February/077062.html, Guido
>> advocates not adding the __bin__ machinery, which is what lead to the simple
>> implementation of bin() just calling PyNumber_ToBase and relying on
>> __index__.
>>
>> I don't think __bin__ should be added to 2.6.  I don't see the point in
>> adding a feature in 2.6 that's implemented so differently in 3.0.  It's just
>> asking for porting hassles.
>>
>> Instead, I think the approach used in 3.0 (r64451) should be used instead.
>>  That is, if this feature exist at all.  I'm -0 on adding bin(), etc. to
>> floats.

I missed the decision, but it's been added, at least to 2.6. However,
it was added after beta 1, so unless there was consensus about this,
IMO it should be rolled back.

> The 3.0 approach means that non-float floating point types still can't be
> displayed properly by bin()/oct()/hex().

Nor can float, AFAICT from the current 3.0 tree.

> The current 2.6 approach means
> every such class has to implement its own equivalent of PyNumber_ToBase.

But that's not supposed to be a burden -- any type purporting to
implement some kind of integer should implement __index__ anyway.
(Unless I misunderstand what you mean by "equivalent of"?)

> Both are lousy solutions to a relative non-problem (IMO) that can easily be
> implemented using a separate display function for those applications which
> happen to need it.
>
> I'd prefer to see a proper PEP for this proposing a new slot that lets any
> class return an (integer_part, fraction_part) tuple of integers, and  have
> PyNumber_ToBase take care of the actual string formatting.

I'd prefer not to see new features added at all, except for things
that were being planned before the beta and specifically got
permission to be added post-beta-1.

> Introducing such a gratuitous incompatibility between 2.6 and 3.0 at this
> late stage of the process certainly seems highly undesirable to me.

Same here.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ggpolo at gmail.com  Tue Jun 24 19:15:53 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Tue, 24 Jun 2008 14:15:53 -0300
Subject: [Python-Dev] setitimer, getitimer wrapper (issue2240)
Message-ID: <ac2200130806241015g267a16ddu33a2c0852b666200@mail.gmail.com>

Hello,

It has been some time since some discussion happened on
http://bugs.python.org/issue2240, and I intended to compose this email
several days ago but just did it now.

This is an attempt to find someone who might have some
information/idea that could help solving the current problem regarding
the setitimer/getitimer wrapper under FreeBSD (and maybe somewhere
else?). The current problem is that the virtual and prof itimers, in
some occasions, may never finish, and as I remember from talking with
Trent Nelson, the problem showed up when running all the tests but not
when running the test_signal alone.

So, that is it.. any solutions ? :)

Regards,

-- 
-- Guilherme H. Polo Goncalves

From eric+python-dev at trueblade.com  Tue Jun 24 19:07:16 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Tue, 24 Jun 2008 13:07:16 -0400
Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:
 Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c
 Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c
In-Reply-To: <ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	
	<4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com>
	<ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com>
Message-ID: <486129C4.4010700@trueblade.com>

Guido van Rossum wrote:
>> The 3.0 approach means that non-float floating point types still can't be
>> displayed properly by bin()/oct()/hex().
> 
> Nor can float, AFAICT from the current 3.0 tree.

$ ./python
Python 3.0b1+ (py3k:64491:64497M, Jun 24 2008, 07:14:03)
[GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> bin(1.0)
'0b1 * 2.0 ** 0'
 >>>

>> The current 2.6 approach means
>> every such class has to implement its own equivalent of PyNumber_ToBase.
> 
> But that's not supposed to be a burden -- any type purporting to
> implement some kind of integer should implement __index__ anyway.
> (Unless I misunderstand what you mean by "equivalent of"?)

But apparently there's a desire to have bin(), oct(), and hex() support 
non-integer types.  See above, and issue 3008.  In 3.0, it's not 
possible to extend these to other types, because of the lack of __bin__, 
etc.  The test for float is hard coded in PyNumber_ToBase().

Eric.


From martin at v.loewis.de  Tue Jun 24 20:21:46 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Jun 2008 20:21:46 +0200
Subject: [Python-Dev] Python Windows Installers
In-Reply-To: <4860D168.9050305@voidspace.org.uk>
References: <4860D168.9050305@voidspace.org.uk>
Message-ID: <48613B3A.9040805@v.loewis.de>

> I'm just doing some housekeeping on a Windows install, and notice that
> the 'Publisher' of my Python 2.4 and 2.5 installs is shown as "Martin v.
> Lowis". Whilst I *personally* find this very reassuring I wonder if this
> is intended / ideal?

I certainly intended it: the field really is called the "author" file in
MSI, and is (was) intended to indicate the "manufacturer" of the MSI,
see

http://msdn.microsoft.com/en-us/library/aa367808(VS.85).aspx

That Vista calls it "publisher" is somewhat ambiguous.

In any case, bug #1737210 complained about it, and I changed it to
"Python Software Foundation". I can't retroactively change it for the
releases you are looking at.

Regards,
Martin

From fuzzyman at voidspace.org.uk  Tue Jun 24 20:22:13 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 24 Jun 2008 19:22:13 +0100
Subject: [Python-Dev] Python Windows Installers
In-Reply-To: <48613B3A.9040805@v.loewis.de>
References: <4860D168.9050305@voidspace.org.uk> <48613B3A.9040805@v.loewis.de>
Message-ID: <48613B55.4040505@voidspace.org.uk>

Martin v. L?wis wrote:
>> I'm just doing some housekeeping on a Windows install, and notice that
>> the 'Publisher' of my Python 2.4 and 2.5 installs is shown as "Martin v.
>> Lowis". Whilst I *personally* find this very reassuring I wonder if this
>> is intended / ideal?
>>     
>
> I certainly intended it: the field really is called the "author" file in
> MSI, and is (was) intended to indicate the "manufacturer" of the MSI,
> see
>
> http://msdn.microsoft.com/en-us/library/aa367808(VS.85).aspx
>
> That Vista calls it "publisher" is somewhat ambiguous.
>
> In any case, bug #1737210 complained about it, and I changed it to
> "Python Software Foundation". I can't retroactively change it for the
> releases you are looking at.
>   

No problem. I was more concerned about future releases.

All the best,

Michael Foord

> Regards,
> Martin
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.theotherdelia.co.uk/
http://www.resolverhacks.net/


From guido at python.org  Tue Jun 24 20:23:57 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Jun 2008 11:23:57 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:
	Include/object.h Lib/test/test_sys.py Misc/NEWS
	Objects/intobject.c Objects/longobject.c Objects/typeobject.c
	Python/bltinmodule.c
In-Reply-To: <486129C4.4010700@trueblade.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com>
	<ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com>
	<486129C4.4010700@trueblade.com>
Message-ID: <ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>

On Tue, Jun 24, 2008 at 10:07 AM, Eric Smith
<eric+python-dev at trueblade.com> wrote:
> Guido van Rossum wrote:
>>>
>>> The 3.0 approach means that non-float floating point types still can't be
>>> displayed properly by bin()/oct()/hex().
>>
>> Nor can float, AFAICT from the current 3.0 tree.
>
> $ ./python
> Python 3.0b1+ (py3k:64491:64497M, Jun 24 2008, 07:14:03)
> [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> bin(1.0)
> '0b1 * 2.0 ** 0'
>>>>

Oops.

>>> The current 2.6 approach means
>>> every such class has to implement its own equivalent of PyNumber_ToBase.
>>
>> But that's not supposed to be a burden -- any type purporting to
>> implement some kind of integer should implement __index__ anyway.
>> (Unless I misunderstand what you mean by "equivalent of"?)
>
> But apparently there's a desire to have bin(), oct(), and hex() support
> non-integer types.  See above, and issue 3008.  In 3.0, it's not possible to
> extend these to other types, because of the lack of __bin__, etc.  The test
> for float is hard coded in PyNumber_ToBase().

There are lots of desires. This particular one goes straight against
an earlier decision to simplify bin(), oct() and hex() by removing the
__oct__ and __hex__ special methods and not adding __bin__. Certainly
not something to be added without a PEP, certainly not after beta1 was
released already.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Jun 24 21:05:12 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Jun 2008 12:05:12 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:
	Include/object.h Lib/test/test_sys.py Misc/NEWS
	Objects/intobject.c Objects/longobject.c Objects/typeobject.c
	Python/bltinmodule.c
In-Reply-To: <DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com>
	<ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com>
	<486129C4.4010700@trueblade.com>
	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>
	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>
Message-ID: <ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>

I'm curious why the addition of a new feature, past beta 1, was
constrained entirely to a single tracker issue. Clearly not enough
people were aware of it (or there wouldn't be the discussion about it
here).

Following the discussion in the issue tracker is really hard (since
most of the discussion apparently refers to earlier versions). I also
don't see any doc changes.

If you want to keep the feature, at least write some documentation for
it so everyone knows what is actually implemented, and why, without
having to go to the source.

--Guido

On Tue, Jun 24, 2008 at 11:44 AM, Raymond Hettinger <python at rcn.com> wrote:
>>>>>> bin(1.0)
>>>
>>> '0b1 * 2.0 ** 0'
>>>>>>
>>
>> Oops.
>
> This was checked-in for Issue 3008 which has been under discussion and
> development for a while. It was intentionally held until after the beta was
> released so that the beta itself would be as stable as possible (in the
> final week and a half most of the bots were red and it made no sense to
> commit a feature request at that time).
>
> The patch allows a eval-able representation of a float that works across
> platforms and is not dependent of %.17g, atof() or ftoa() routines.  The
> tool makes it much easier to examine or explain what floating point ops
> really do.  It was invaluable in the development of math.sum().  I've spent
> several days of development time on this and would be really bummed if it
> got reversed-out at this point.
>
> The use of __bin__ is not essential to its implementation so that
> implementation detail can be changed if needed.
>
>
> Raymond
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Tue Jun 24 21:43:54 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 24 Jun 2008 20:43:54 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:
	Include/object.h Lib/test/test_sys.py Misc/NEWS
	Objects/intobject.c Objects/longobject.c Objects/typeobject.c
	Python/bltinmodule.c
In-Reply-To: <ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com>
	<ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com>
	<486129C4.4010700@trueblade.com>
	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>
	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
Message-ID: <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>

On Tue, Jun 24, 2008 at 8:05 PM, Guido van Rossum <guido at python.org> wrote:

>
> Following the discussion in the issue tracker is really hard (since
> most of the discussion apparently refers to earlier versions). I also
> don't see any doc changes.


I think there may also still be room for some additional discussion
on the output format;  while being eval-able is nice, one not-so-nice
aspect of the representation is that nearby floats can have
entirely distinct-looking representations;  as in, for example:

>>> hex(126.38)
'0x3f30a3d70a3d7 * 2.0 ** -43'
>>> hex(126.39)
'0x1f98f5c28f5c29 * 2.0 ** -46'

bin() suffers much less from this, but on balance I think I'd
still prefer to see a non eval-able output that's normalized with
respect to the most-significant bit instead of the least.

I realize that (a) this has little to do with implementation
details and use or not of __bin__, and (b) I should have
said this in the issue tracker a few days ago.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080624/1feeeb92/attachment.htm>

From musiccomposition at gmail.com  Tue Jun 24 21:45:49 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 24 Jun 2008 14:45:49 -0500
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <ca471dc20806240942l571c647bx4b56e565eb5405ab@mail.gmail.com>
References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>
	<4860CDAF.2020806@gmail.com>
	<1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com>
	<4860F1DF.5080204@gmail.com>
	<1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com>
	<ca471dc20806240942l571c647bx4b56e565eb5405ab@mail.gmail.com>
Message-ID: <1afaf6160806241245x3ed56863kc05030f1c6950ab8@mail.gmail.com>

On Tue, Jun 24, 2008 at 11:42 AM, Guido van Rossum <guido at python.org> wrote:
> The same generic warning against sweeping changes applies here though.
> You have to manually review each change. The stdlib and especially the
> test suite is likely to break if you just let 2to3 run over it, even
> just a single fixer like fix_has_key.

So be it. We'll go over each module individually as with #3189.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From martin at v.loewis.de  Tue Jun 24 21:51:39 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 24 Jun 2008 21:51:39 +0200
Subject: [Python-Dev] [Python-checkins] r64424 - in
 python/trunk:	Include/object.h Lib/test/test_sys.py
 Misc/NEWS	Objects/intobject.c Objects/longobject.c Objects/typeobject.c
 Python/bltinmodule.c
In-Reply-To: <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<4860ED9F.8030305@trueblade.com>
	<4860F3FE.8090307@gmail.com>	<ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com>	<486129C4.4010700@trueblade.com>	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
Message-ID: <4861504B.209@v.loewis.de>

> I think there may also still be room for some additional discussion
> on the output format;

If so, I think the change should be reverted, and the feature deferred
to 2.7/3.1.

Regards,
Martin

From guido at python.org  Tue Jun 24 22:13:55 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Jun 2008 13:13:55 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:
	Include/object.h Lib/test/test_sys.py Misc/NEWS
	Objects/intobject.c Objects/longobject.c Objects/typeobject.c
	Python/bltinmodule.c
In-Reply-To: <4861504B.209@v.loewis.de>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com>
	<ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com>
	<486129C4.4010700@trueblade.com>
	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>
	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
	<4861504B.209@v.loewis.de>
Message-ID: <ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>

[Mark Dickinson]
>> I think there may also still be room for some additional discussion
>> on the output format;

[Martin v. L?wis]
> If so, I think the change should be reverted, and the feature deferred
> to 2.7/3.1.

It looks like pretty much every aspect of this change was discussed /
reviewed insufficiently. +1 on rolling back and taking the time to get
it right. -1 on rushing it in.

Since it's (potentially) a pretty small feature I might be convinced
to accept it in beta2, but I don't want the fact that it was committed
to force our hand. Raymond, if you care about this feature, please
roll it back so we can have a proper discussion without pressure.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Tue Jun 24 23:48:04 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 24 Jun 2008 17:48:04 -0400
Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:
	Include/object.h Lib/test/test_sys.py Misc/NEWS
	Objects/intobject.c Objects/longobject.c Objects/typeobject.c
	Python/bltinmodule.c
In-Reply-To: <ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com>
	<ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com>
	<486129C4.4010700@trueblade.com>
	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>
	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
Message-ID: <5E3B218C-F52E-410F-B5C8-AE64174BE416@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 24, 2008, at 4:13 PM, Guido van Rossum wrote:

> [Mark Dickinson]
>>> I think there may also still be room for some additional discussion
>>> on the output format;
>
> [Martin v. L?wis]
>> If so, I think the change should be reverted, and the feature  
>> deferred
>> to 2.7/3.1.
>
> It looks like pretty much every aspect of this change was discussed /
> reviewed insufficiently. +1 on rolling back and taking the time to get
> it right. -1 on rushing it in.
>
> Since it's (potentially) a pretty small feature I might be convinced
> to accept it in beta2, but I don't want the fact that it was committed
> to force our hand. Raymond, if you care about this feature, please
> roll it back so we can have a proper discussion without pressure.

FWIW, I concur.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGFrlnEjvBPtnXfVAQITxwP/T16pGX+aTV5O/lhmwYWDtv8bA0S2l+Kl
6paSWTz8HHD0cpAhMAKKN7XqnJ6A0uIebCEJSPFdir8Jl0AMSbBZq88l+D93eoC0
eiqryogRTUzwrs2zC3A/nMjpNe21MErQNUEqmpU7XgdOpBRFQF8UMOnRIoSV8ggx
SKwy+cRHnEk=
=olIq
-----END PGP SIGNATURE-----

From ncoghlan at gmail.com  Wed Jun 25 00:05:21 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Jun 2008 08:05:21 +1000
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <1afaf6160806241245x3ed56863kc05030f1c6950ab8@mail.gmail.com>
References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>	
	<4860CDAF.2020806@gmail.com>	
	<1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com>	
	<4860F1DF.5080204@gmail.com>	
	<1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com>	
	<ca471dc20806240942l571c647bx4b56e565eb5405ab@mail.gmail.com>
	<1afaf6160806241245x3ed56863kc05030f1c6950ab8@mail.gmail.com>
Message-ID: <48616FA1.8060500@gmail.com>

Benjamin Peterson wrote:
> On Tue, Jun 24, 2008 at 11:42 AM, Guido van Rossum <guido at python.org> wrote:
>> The same generic warning against sweeping changes applies here though.
>> You have to manually review each change. The stdlib and especially the
>> test suite is likely to break if you just let 2to3 run over it, even
>> just a single fixer like fix_has_key.
> 
> So be it. We'll go over each module individually as with #3189.

We need to be especially careful with the unit test suite itself - 
changing the test code to avoid the warning will normally be the right 
answer, but when the code is actually setting out to test the deprecated 
feature we need to suppress the warning in the test suite instead.

I should have a decent amount of Python time available later this 
week/weekend, so I should be able to help with this once I get the 
updated docs for special method lookup knocked over as well as checking 
in the somewhat related fix for the following 2.5 behaviour:

 >>> print unicode(Exception)
<type 'exceptions.Exception'>
 >>> print unicode(u"\xa3")
?
 >>> print unicode(Exception(u"\xa3"))
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in 
position 0: ordinal not in range(128)

(The idea being to get that last line printing the same thing as the 
second line, just as unicode(Exception("A")) produces the answer u"A", 
but without breaking the current behaviour of the first line)

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From glyph at divmod.com  Wed Jun 25 01:03:33 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Tue, 24 Jun 2008 23:03:33 -0000
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <48616FA1.8060500@gmail.com>
References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com>
	<4860CDAF.2020806@gmail.com>
	<1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com>
	<4860F1DF.5080204@gmail.com>
	<1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com>
	<ca471dc20806240942l571c647bx4b56e565eb5405ab@mail.gmail.com>
	<1afaf6160806241245x3ed56863kc05030f1c6950ab8@mail.gmail.com>
	<48616FA1.8060500@gmail.com>
Message-ID: <20080624230333.25821.838405170.divmod.xquotient.10140@joule.divmod.com>

On 10:05 pm, ncoghlan at gmail.com wrote:
>We need to be especially careful with the unit test suite itself - 
>changing the test code to avoid the warning will normally be the right 
>answer, but when the code is actually setting out to test the 
>deprecated feature we need to suppress the warning in the test suite 
>instead.

This is a dangerous road to go down.  If you suppress warnings in the 
test suite, you might suppress additional warnings which should actually 
be reported.  Or, if the API gets modified in some way that the warning 
is supposed to be emitted but isn't any longer, it will be silent.  It's 
easy to accidentally suppress too much or not enough.

The way we've dealt with this in Twisted is adding an 'assertWarns' 
method so that we can invoke an API that is supposed to generate a 
warning, and (A) that warning and only that *specific* warning will not 
be emitted, and (B) if the API stops emitting the warning in the future, 
the test will fail and we will notice.

It's also nice to have this facility in the test harness itself, so that 
you don't run the additional risk of accidentally (and silently) leaving 
warning suppression in place for subsequent tests.

From exarkun at divmod.com  Wed Jun 25 04:24:55 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 24 Jun 2008 22:24:55 -0400
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <20080624230333.25821.838405170.divmod.xquotient.10140@joule.divmod.com>
Message-ID: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>

On Tue, 24 Jun 2008 23:03:33 -0000, glyph at divmod.com wrote:
>On 10:05 pm, ncoghlan at gmail.com wrote:
>>We need to be especially careful with the unit test suite itself - changing 
>>the test code to avoid the warning will normally be the right answer, but 
>>when the code is actually setting out to test the deprecated feature we 
>>need to suppress the warning in the test suite instead.
>
>This is a dangerous road to go down.  If you suppress warnings in the test 
>suite, you might suppress additional warnings which should actually be 
>reported.  Or, if the API gets modified in some way that the warning is 
>supposed to be emitted but isn't any longer, it will be silent.  It's easy 
>to accidentally suppress too much or not enough.
>
>The way we've dealt with this in Twisted is adding an 'assertWarns' method 
>so that we can invoke an API that is supposed to generate a warning, and (A) 
>that warning and only that *specific* warning will not be emitted, and (B) 
>if the API stops emitting the warning in the future, the test will fail and 
>we will notice.
>
>It's also nice to have this facility in the test harness itself, so that you 
>don't run the additional risk of accidentally (and silently) leaving warning 
>suppression in place for subsequent tests.

It would be *extra* nice to have this facility added to the standard
library, since assertWarns in Twisted is broken by changes in Python
2.6 (ie, our tests for warnings all fail with trunk at HEAD).  For now,
we will probably address this by switching to a different warning
API.  In the long term, it'd be better for us, other Python developers,
and the standard library if there were an API in the standard library
which facilitated testing of warnings.

Jean-Paul

From python at rcn.com  Wed Jun 25 05:20:05 2008
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 24 Jun 2008 20:20:05 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
References: <20080620041816.4D5E81E4002@bag.python.org><4860ED9F.8030305@trueblade.com>
	<4860F3FE.8090307@gmail.com><ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com><486129C4.4010700@trueblade.com><ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com><DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1><ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com><5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com><4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
Message-ID: <FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>

Issue 3008 has been re-opened for more commentary.

Raymond

----- Original Message ----- 
From: "Guido van Rossum" <guido at python.org>
Since it's (potentially) a pretty small feature I might be convinced
to accept it in beta2, but I don't want the fact that it was committed
to force our hand. Raymond, if you care about this feature, please
roll it back so we can have a proper discussion without pressure.


From guido at python.org  Wed Jun 25 06:05:56 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Jun 2008 21:05:56 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806240956t212c6fd9h78115bc2a34bd049@mail.gmail.com>
	<486129C4.4010700@trueblade.com>
	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>
	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
Message-ID: <ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>

I don't want the discussion in the issue. I want to have it right here.

While I think it's fine to have some function that reveals the binary
representation of floats, I don't think that overlaying this on
hex/oct/bin is worth the problems it causes. This API appears to be
purely for educational purposes; why not implement something in pure
Python using the struct module that reveals the lay-out of the
floating-point value?

(There are also several things wrong with the specific patch, apart
from its lack of docs; #1 is the introduction of an externaly visible
symbol that doesn't start with _Py.)

--Guido

On Tue, Jun 24, 2008 at 8:20 PM, Raymond Hettinger <python at rcn.com> wrote:
> Issue 3008 has been re-opened for more commentary.
>
> Raymond
>
> ----- Original Message ----- From: "Guido van Rossum" <guido at python.org>
> Since it's (potentially) a pretty small feature I might be convinced
> to accept it in beta2, but I don't want the fact that it was committed
> to force our hand. Raymond, if you care about this feature, please
> roll it back so we can have a proper discussion without pressure.
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Wed Jun 25 12:24:06 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Jun 2008 20:24:06 +1000
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>
References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>
Message-ID: <48621CC6.2030801@gmail.com>

Jean-Paul Calderone wrote:
> On Tue, 24 Jun 2008 23:03:33 -0000, glyph at divmod.com wrote:
>> It's also nice to have this facility in the test harness itself, so 
>> that you don't run the additional risk of accidentally (and silently) 
>> leaving warning suppression in place for subsequent tests.
> 
> It would be *extra* nice to have this facility added to the standard
> library, since assertWarns in Twisted is broken by changes in Python
> 2.6 (ie, our tests for warnings all fail with trunk at HEAD).  For now,
> we will probably address this by switching to a different warning
> API.  In the long term, it'd be better for us, other Python developers,
> and the standard library if there were an API in the standard library
> which facilitated testing of warnings.

I forgot this had already been added to the Python regression test 
machinery, so it will just be a matter of updating the relevant tests to 
use it:

http://docs.python.org/dev/library/test.html#module-test.test_support

test.test_support.catch_warning(record=True)?

     Return a context manager that guards the warnings filter from being 
permanently changed and records the data of the last warning that has 
been issued. The record argument specifies whether any raised warnings 
are captured by the object returned by warnings.catch_warning() or 
allowed to propagate as normal.

     The context manager is typically used like this:

     with catch_warning() as w:
         warnings.warn("foo")
         assert str(w.message) == "foo"

     New in version 2.6.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From andrew-pythondev at puzzling.org  Wed Jun 25 15:08:57 2008
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Wed, 25 Jun 2008 23:08:57 +1000
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <48621CC6.2030801@gmail.com>
References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>
	<48621CC6.2030801@gmail.com>
Message-ID: <20080625130857.GA13663@steerpike.home.puzzling.org>

Nick Coghlan wrote:
[...]
>
> I forgot this had already been added to the Python regression test  
> machinery, so it will just be a matter of updating the relevant tests to  
> use it:

That's a nice surprise!  I'm glad the standard library is growing facilities
like this.

I think it could be improved a little, though:

> http://docs.python.org/dev/library/test.html#module-test.test_support
>
> test.test_support.catch_warning(record=True)?
>
>     Return a context manager that guards the warnings filter from being  
> permanently changed and records the data of the last warning that has  
> been issued. The record argument specifies whether any raised warnings  
> are captured by the object returned by warnings.catch_warning() or  
> allowed to propagate as normal.

The description doesn't really make the record=False case clear.  This context
manager is doing two different jobs: 1) restore the filters list and showwarning
function to their original state when done, and 2) optionally (if record=True)
record the last warning in the "as" target.  That feels a bit weird.

I think a clearer way to provide that functionality would be with two separate
context managers: one that copies and finally restores the filters list and
showwarnning function called protect_warnings_module, and then catch_warnings to
record the warnings.  The catch_warnings context manager could reuse the
protect_warnings_module one.  "with protect_warnings_module:" seems easier to
understand (and document) than "with catch_warning(record=False)".

Should I file a bug for this?

>     The context manager is typically used like this:
>
>     with catch_warning() as w:
>         warnings.warn("foo")
>         assert str(w.message) == "foo"
>

Maybe this is a YAGNI, but what if an expression triggers multiple warnings?
Perhaps the WarningMessage object ought to keep a list rather than just the last
warning.

Another thought: if the warnings module had a public, documented way to
manipulate the filter list (e.g. warnings.get_all_filters() and
warnings.set_all_filters(...)), then people could build these sorts of test
facilities for themselves if the test_support one turns out to be too limiting
or otherwise a poor fit. 

-Andrew.


From brett at python.org  Wed Jun 25 19:18:55 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 25 Jun 2008 10:18:55 -0700
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <20080625130857.GA13663@steerpike.home.puzzling.org>
References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>
	<48621CC6.2030801@gmail.com>
	<20080625130857.GA13663@steerpike.home.puzzling.org>
Message-ID: <bbaeab100806251018w42bd1b78qb1a98144ac453779@mail.gmail.com>

On Wed, Jun 25, 2008 at 6:08 AM, Andrew Bennetts
<andrew-pythondev at puzzling.org> wrote:
> Nick Coghlan wrote:
> [...]
>>
>> I forgot this had already been added to the Python regression test
>> machinery, so it will just be a matter of updating the relevant tests to
>> use it:
>
> That's a nice surprise!  I'm glad the standard library is growing facilities
> like this.
>
> I think it could be improved a little, though:
>
>> http://docs.python.org/dev/library/test.html#module-test.test_support
>>
>> test.test_support.catch_warning(record=True)?
>>
>>     Return a context manager that guards the warnings filter from being
>> permanently changed and records the data of the last warning that has
>> been issued. The record argument specifies whether any raised warnings
>> are captured by the object returned by warnings.catch_warning() or
>> allowed to propagate as normal.
>
> The description doesn't really make the record=False case clear.  This context
> manager is doing two different jobs: 1) restore the filters list and showwarning
> function to their original state when done, and 2) optionally (if record=True)
> record the last warning in the "as" target.  That feels a bit weird.
>

The 'record=False' functionality was only added a couple of months
ago. The context manager was originally written to help test the
warnings module itself, so it was not meant to allow warnings to
propagate. I tossed in the 'record' argument so that I could mutate
the warnings filter.

> I think a clearer way to provide that functionality would be with two separate
> context managers: one that copies and finally restores the filters list and
> showwarnning function called protect_warnings_module, and then catch_warnings to
> record the warnings.  The catch_warnings context manager could reuse the
> protect_warnings_module one.  "with protect_warnings_module:" seems easier to
> understand (and document) than "with catch_warning(record=False)".
>
> Should I file a bug for this?
>

If you want, but Benjamin plans to undocument this for users along
with all other test.support stuff (which I agree with). Most of the
APIs in test.support were just quickly written and have not
necessarily been thought through in order to make sure that the APIs
makes sense (like in this case).

-Brett

From faassen at startifact.com  Wed Jun 25 20:45:11 2008
From: faassen at startifact.com (Martijn Faassen)
Date: Wed, 25 Jun 2008 20:45:11 +0200
Subject: [Python-Dev] doctest, exec and __module__
Message-ID: <g3u3no$h76$1@ger.gmane.org>

Hi there,

I've just witnessed an interesting consequence of the way doctest works.

I ran into an issue when doctesting an aspect of SQLAlchemy, where the 
following guard clause tripped me up:

     # In the normal call flow, a request for any of the 3 basic collection
     # types is transformed into one of our trivial subclasses
     # (e.g. InstrumentedList).  Catch anything else that sneaks in here...
     if cls.__module__ == '__builtin__':
         raise sa_exc.ArgumentError(
             "Can not instrument a built-in type. Use a "
             "subclass, even a trivial one.")

My class, coming in as cls here, was defined in a doctest, like this:

   >>> class Foo(object):
   ...    pass

It turns out that doctest compiles and executes this bit of code using a 
line like this:

          # Don't blink!  This is where the user's code gets run.
         exec compile(example.source, filename, "single",
                      compileflags, 1) in test.globs

This places new key/value pairs into a dictionary, in this case 
test.globs. Unfortunately when the execution results in a class 
definition, it'll have its __module__ attribute set to '__builtin__'. 
Try as I might, I couldn't convince exec to do it any differently.

I can't think of an nice way to work around this problem either. The 
ugly workaround in the doctest itself works:

   >>> Foo.__module__ = 'whatever'

That isn't very nice though. You could also iterate through all the 
values in the dictionary after each exec, and then check whether it's a 
class, and if so, set its __module__ to something else than __builtin__, 
but that doesn't feel very pretty (or efficient) either.

Any ideas? Am I missing something? Is there really no way to control 
this behavior with exec?

I'd like to somehow fix doctest.py so it doesn't set the __module__ to 
'__builtin__' for everything. '__main__' would be nicer, as that's what 
the interpreter shell does, and a doctest example already looks like the 
interpreter shell. While the above SQLAlchemy code is hardly pretty, I 
can't think of any better way to put in a safeguard like that either.

Regards,

Martijn


From fdrake at acm.org  Wed Jun 25 21:37:28 2008
From: fdrake at acm.org (Fred Drake)
Date: Wed, 25 Jun 2008 15:37:28 -0400
Subject: [Python-Dev] doctest, exec and __module__
In-Reply-To: <g3u3no$h76$1@ger.gmane.org>
References: <g3u3no$h76$1@ger.gmane.org>
Message-ID: <C8D488D9-C8B0-481A-8DB9-659DC5D044E5@acm.org>

On Jun 25, 2008, at 2:45 PM, Martijn Faassen wrote:
> This places new key/value pairs into a dictionary, in this case  
> test.globs. Unfortunately when the execution results in a class  
> definition, it'll have its __module__ attribute set to  
> '__builtin__'. Try as I might, I couldn't convince exec to do it any  
> differently.

The zope.testing package has a way to work around this from setup/ 
teardown functions passed to the DocFileSuite (or whatever wrapper  
you're using).  See the module zope.testing.module for the functions.

Essentially, the __name__ needs to be set in test.globs; class  
construction uses the global value for __name__ to provide __module__;  
doctests normally don't have one, so it acquires the value from  
__builtin__.__name__ instead.  Which is, of course, horribly wrong.

The solution in zope.testing.module allows the class to be used with  
pickle within the test as well.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From hall.jeff at gmail.com  Wed Jun 25 21:56:11 2008
From: hall.jeff at gmail.com (Jeff Hall)
Date: Wed, 25 Jun 2008 15:56:11 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <loom.20080624T093235-422@post.gmane.org>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>
	<485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol>
	<485D5CFC.5020705@v.loewis.de>
	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>
	<485F1ADA.6090107@v.loewis.de>
	<loom.20080623T102812-837@post.gmane.org>
	<486010CD.2000904@v.loewis.de>
	<loom.20080624T093235-422@post.gmane.org>
Message-ID: <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com>

It seems to me that the root problem is allocation spikes of legitimate,
useful data. Perhaps then we need some sort of "test" to determine if those
are legitimate. Perhaps checking every nth (with n decreasing as allocation
bytes increases) object allocated during a "spike" could be useful. Then
delay garbage collection until x consecutive objects are found to be
garbage?

It seems like we should be attacking the root cause rather than finding some
convoluted math that attempts to work for all scenarios.

On a side note, the information about not GCing on string objects is
interesting? Is there a way to override this behavior? I've found that re.py
chokes on large text files (4MB plus) without line ends (don't ask, they're
not our files but we have to parse them). I wonder if this isn't the
reason...

If the answer to that is, "no, strings are always ignored" I'd recommend
rethinking this (or providing an option to override somehow.

-- 
Haikus are easy
Most make very little sense
Refrigerator
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080625/52291f16/attachment.htm>

From musiccomposition at gmail.com  Wed Jun 25 22:00:17 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 25 Jun 2008 15:00:17 -0500
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
	DeprecationWarning in stdlib)
Message-ID: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>

On Wed, Jun 25, 2008 at 12:18 PM, Brett Cannon <brett at python.org> wrote:
> If you want, but Benjamin plans to undocument this for users along
> with all other test.support stuff (which I agree with). Most of the
> APIs in test.support were just quickly written and have not
> necessarily been thought through in order to make sure that the APIs
> makes sense (like in this case).

Thanks for bringing that up, Brett. Is making test_support non-public
a API in 3.0 fine with everybody?


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From facundobatista at gmail.com  Wed Jun 25 22:16:43 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 25 Jun 2008 17:16:43 -0300
Subject: [Python-Dev] Last "bug weekend"
Message-ID: <e04bdf310806251316v2e74da4dq28613992314b53b8@mail.gmail.com>

Hi all!

In Python Argentina we get together in two places, Buenos Aires and
Santa Fe, and worked out around 10-12 bugs, which is around the half
of the total for both days, so I'm very happy, :)

We even had a Python logo shaped cake, see, see [1].

Did you participate? What do you think it's the outcome for both days?

Regards,

[1] http://www.taniquetil.com.ar/facundo/imgs/tortalogopy.jpeg

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From faassen at startifact.com  Wed Jun 25 22:26:27 2008
From: faassen at startifact.com (Martijn Faassen)
Date: Wed, 25 Jun 2008 22:26:27 +0200
Subject: [Python-Dev] doctest, exec and __module__
In-Reply-To: <C8D488D9-C8B0-481A-8DB9-659DC5D044E5@acm.org>
References: <g3u3no$h76$1@ger.gmane.org>
	<C8D488D9-C8B0-481A-8DB9-659DC5D044E5@acm.org>
Message-ID: <g3u9lj$5me$1@ger.gmane.org>

Fred Drake wrote:
> On Jun 25, 2008, at 2:45 PM, Martijn Faassen wrote:
>> This places new key/value pairs into a dictionary, in this case 
>> test.globs. Unfortunately when the execution results in a class 
>> definition, it'll have its __module__ attribute set to '__builtin__'. 
>> Try as I might, I couldn't convince exec to do it any differently.
> 
> The zope.testing package has a way to work around this from 
> setup/teardown functions passed to the DocFileSuite (or whatever wrapper 
> you're using).  See the module zope.testing.module for the functions.

Thanks, I should've checked that!

> Essentially, the __name__ needs to be set in test.globs; class 
> construction uses the global value for __name__ to provide __module__; 
> doctests normally don't have one, so it acquires the value from 
> __builtin__.__name__ instead.  Which is, of course, horribly wrong.

Okay, so is this mystery mechanism that I was missing. Sorry to bother 
people here on the list.

Regards,

Martijn


From guido at python.org  Wed Jun 25 22:49:24 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Jun 2008 13:49:24 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>
	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
Message-ID: <ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>

[Adding back the list.]

On Tue, Jun 24, 2008 at 9:53 PM, Raymond Hettinger <python at rcn.com> wrote:
>> While I think it's fine to have some function that reveals the binary
>> representation of floats, I don't think that overlaying this on
>> hex/oct/bin is worth the problems it causes.
>
> What problems?  The patch is clean.

Problems like no two people on python-dev agreeing on how exactly the
feature should be implemented. Problems like whether this goes way
beyond the philosophical underpinnings of bin/oct/hex. Problems like
what to do about other types that might want to overload hex/oct/bin.
See Kevin Jacobs' response.

>> This API appears to be
>> purely for educational purposes; why not implement something in pure
>> Python using the struct module that reveals the lay-out of the
>> floating-point value?
>
> This is not the internal floating point layout.  It is the real value
> expressed in exponential form.  It is more than educational --  it is a
> platform independent representation (look at Terry's reference -- it is the
> usual way to precisely specify a float value and it does not depend on
> atof() or vice versa).

Possibly, but it is only readable by a Python expression parser. For
all practical purposes "%.17g" % x works just as well. And this
bypasses the question "why overload this functionality on bin/hex/oct
rather than adding e.g. a new function to math or a new method to
float."

>> (There are also several things wrong with the specific patch, apart
>> from its lack of docs; #1 is the introduction of an externaly visible
>> symbol that doesn't start with _Py.)
>
> Will change the global symbol to _Py.  I already added docs to the patch.
>  Did you see the one that was uploaded a few hours ago (float6.diff)?

I don't care about the details of the patch until we have agreement
about which form the feature should take. We don't have that agreement
yet. I mentioned the flaws in the patch to point out that it was
apparently a rush job.

> I re-opened the discussion at your behest. [...]

I'm very glad you're giving the discussion a second chance. Please
give it a few days at least. My expectation is that the outcome will
be not to overload bin/hex/oct but to add a custom function to math or
a custom method to float, whose output can be further massaged to
create the platform-independent representation you're after. (I doubt
that it's worth changing pickle or marshal though, they are doing fine
with their respective current approaches.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Wed Jun 25 22:55:32 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 25 Jun 2008 22:55:32 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com>	<485D231D.9030806@v.loewis.de>
	<1214075410.5874.26.camel@fsol>	<485D5CFC.5020705@v.loewis.de>	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>	<485F1ADA.6090107@v.loewis.de>	<loom.20080623T102812-837@post.gmane.org>	<486010CD.2000904@v.loewis.de>	<loom.20080624T093235-422@post.gmane.org>
	<1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com>
Message-ID: <4862B0C4.2080901@v.loewis.de>

> It seems to me that the root problem is allocation spikes of legitimate,
> useful data. Perhaps then we need some sort of "test" to determine if
> those are legitimate. Perhaps checking every nth (with n decreasing as
> allocation bytes increases) object allocated during a "spike" could be
> useful. Then delay garbage collection until x consecutive objects are
> found to be garbage?
> 
> It seems like we should be attacking the root cause rather than finding
> some convoluted math that attempts to work for all scenarios.

I think exactly the other way 'round. The timing of thing should not
matter at all, only the exact sequence of allocations and deallocations.

I trust provable maths much more than I trust ad-hoc heuristics, even
if you think the math is convoluted.

> On a side note, the information about not GCing on string objects is
> interesting? Is there a way to override this behavior?

I think you misunderstand. Python releases unused string objects just
fine, and automatically. It doesn't even need GC for that.

Regards,
Martin


From guido at python.org  Wed Jun 25 23:00:44 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Jun 2008 14:00:44 -0700
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
	DeprecationWarning in stdlib)
In-Reply-To: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>
Message-ID: <ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>

I'm a little worried about making stuff undocumented that every core
developer needs to use -- everyone writing tests needs to continue to
use test_support (now test.support?). I imagine people writing unit
test suites for 3rd party libraries might want to use its services
too.

In general I'm not a big fan of having undocumented APIs; 9 out of 10
times someone finds a genuine use case for such an API, and then
you're worse off than if it was documented to begin with, since if
people start using undocumented APIs, they necessarily
reverse-engineer how it works, and then you can never change it. If it
was documented, at least you may be able to get away with modifying
the undocumented parts, assuming people actually read the docs.
(Though we've had cases where the docs and implementation were
inconsistent for years, and eventually we ended up fixing the docs...)

--Guido

On Wed, Jun 25, 2008 at 1:00 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Wed, Jun 25, 2008 at 12:18 PM, Brett Cannon <brett at python.org> wrote:
>> If you want, but Benjamin plans to undocument this for users along
>> with all other test.support stuff (which I agree with). Most of the
>> APIs in test.support were just quickly written and have not
>> necessarily been thought through in order to make sure that the APIs
>> makes sense (like in this case).
>
> Thanks for bringing that up, Brett. Is making test_support non-public
> a API in 3.0 fine with everybody?
>
>
> --
> Cheers,
> Benjamin Peterson
> "There's no place like 127.0.0.1."
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From hall.jeff at gmail.com  Wed Jun 25 23:08:58 2008
From: hall.jeff at gmail.com (Jeff Hall)
Date: Wed, 25 Jun 2008 17:08:58 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <4862B0C4.2080901@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de>
	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>
	<485F1ADA.6090107@v.loewis.de>
	<loom.20080623T102812-837@post.gmane.org>
	<486010CD.2000904@v.loewis.de>
	<loom.20080624T093235-422@post.gmane.org>
	<1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com>
	<4862B0C4.2080901@v.loewis.de>
Message-ID: <1bc395c10806251408w700f0791y3474fcb1f639ee6e@mail.gmail.com>

On Wed, Jun 25, 2008 at 4:55 PM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> > It seems to me that the root problem is allocation spikes of legitimate,
> > useful data. Perhaps then we need some sort of "test" to determine if
> > those are legitimate. Perhaps checking every nth (with n decreasing as
> > allocation bytes increases) object allocated during a "spike" could be
> > useful. Then delay garbage collection until x consecutive objects are
> > found to be garbage?
> >
> > It seems like we should be attacking the root cause rather than finding
> > some convoluted math that attempts to work for all scenarios.
>
> I think exactly the other way 'round. The timing of thing should not
> matter at all, only the exact sequence of allocations and deallocations.
>
> I trust provable maths much more than I trust ad-hoc heuristics, even
> if you think the math is convoluted.
>

I probably chose my wording poorly (particularly for a newcomer/outsider).
What I meant was that the numbers used in GC currently appear arbitrary. The
idea of three "groups" (youngest, oldest and middle) is also arbitrary.
Would it not be better to tear that system apart and create a "sliding"
scale. If the timing method is undesirable then make it slide based on the
allocation/deallocation difference. In this way, the current breakpoints and
number of groups (all of which are arbitrary and fixed) could be replaced by
one coefficient (and yes, I recognize that it would also be arbitrary but it
would be one, tweakable number rather than several).

My gut tells me that your current fix is going to work just fine for now but
we're going to end up tweaking it (or at least discussing tweaking it) every
6-12 months.


> > On a side note, the information about not GCing on string objects is
> > interesting? Is there a way to override this behavior?
>
> I think you misunderstand. Python releases unused string objects just
> fine, and automatically. It doesn't even need GC for that.
>

I took the statement, "Current GC only takes into account container objects,
which, most significantly, ignores string objects (of which most
applications create plenty)" to mean that strings were ignored for deciding
when to do garbage collection. I mistakenly thought that was because they
were assumed to be small. It sounds like they're ignored because they're
automatically collected and so they SHOULD be ignored for object garbage
collection. Thanks for the clarification... Back to the drawing board on my
other problem ;)

-- 
Haikus are easy
Most make very little sense
Refrigerator
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080625/447557f4/attachment-0001.htm>

From brett at python.org  Thu Jun 26 00:27:33 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 25 Jun 2008 15:27:33 -0700
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
	DeprecationWarning in stdlib)
In-Reply-To: <ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>
	<ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>
Message-ID: <bbaeab100806251527y4b54ee6ck83905abec53c70e6@mail.gmail.com>

On Wed, Jun 25, 2008 at 2:00 PM, Guido van Rossum <guido at python.org> wrote:
> I'm a little worried about making stuff undocumented that every core
> developer needs to use -- everyone writing tests needs to continue to
> use test_support

Right, but I would think all core developers know about test.support
and are capable of reading the code and docstring.

> (now test.support?).

In 3.0, yes.

> I imagine people writing unit
> test suites for 3rd party libraries might want to use its services
> too.
>
> In general I'm not a big fan of having undocumented APIs; 9 out of 10
> times someone finds a genuine use case for such an API, and then
> you're worse off than if it was documented to begin with, since if
> people start using undocumented APIs, they necessarily
> reverse-engineer how it works, and then you can never change it. If it
> was documented, at least you may be able to get away with modifying
> the undocumented parts, assuming people actually read the docs.
> (Though we've had cases where the docs and implementation were
> inconsistent for years, and eventually we ended up fixing the docs...)
>

My worry is that the module has had stuff tossed in ad-hoc in such a
way that maintaining the code for public consumption is a pain. If we
took the time to clean up the APIs then I would be fine with
documenting the module.

-Brett

From martin at v.loewis.de  Thu Jun 26 00:39:37 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jun 2008 00:39:37 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <1bc395c10806251408w700f0791y3474fcb1f639ee6e@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	<1214075410.5874.26.camel@fsol>
	<485D5CFC.5020705@v.loewis.de>	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>	<485F1ADA.6090107@v.loewis.de>	<loom.20080623T102812-837@post.gmane.org>	<486010CD.2000904@v.loewis.de>	<loom.20080624T093235-422@post.gmane.org>	<1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com>	<4862B0C4.2080901@v.loewis.de>
	<1bc395c10806251408w700f0791y3474fcb1f639ee6e@mail.gmail.com>
Message-ID: <4862C929.6070909@v.loewis.de>

> I took the statement, "Current GC only takes into account container
> objects, which, most significantly, ignores string objects (of which
> most applications create plenty)" to mean that strings were ignored for
> deciding when to do garbage collection. I mistakenly thought that was
> because they were assumed to be small. It sounds like they're ignored
> because they're automatically collected and so they SHOULD be ignored
> for object garbage collection. 

More precisely, a string object can never participate in a cycle (it
can be referenced from a cycle, but not be *in* the cycle, as it
has no references to other objects). GC in Python is only about
container objects (which potentially can be cyclic); non-container
objects are released when the refcount says they are no longer
referenced.

Whether or not allocation of definitely-non-cyclic objects should
still trigger cyclic GC (in the hope that some objects hang on a
garbage cycle) is a question that is open to debate; I'd prefer an
analysis of existing applications before making decisions.

Regards,
Martin

From musiccomposition at gmail.com  Thu Jun 26 00:42:37 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 25 Jun 2008 17:42:37 -0500
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
	DeprecationWarning in stdlib)
In-Reply-To: <ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>
	<ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>
Message-ID: <1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com>

On Wed, Jun 25, 2008 at 4:00 PM, Guido van Rossum <guido at python.org> wrote:
> I'm a little worried about making stuff undocumented that every core
> developer needs to use -- everyone writing tests needs to continue to
> use test_support (now test.support?). I imagine people writing unit
> test suites for 3rd party libraries might want to use its services
> too.

I think undocumented is a little unspecific here. What I mean is
"reserved for core Python tests and no promise is made to retain
compatibility." Of course, we would keep docs for them (perhaps in
Lib/test/README), so new core developers could write their tests.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From alexandre at peadrop.com  Thu Jun 26 01:35:15 2008
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Wed, 25 Jun 2008 19:35:15 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <4862B0C4.2080901@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de>
	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>
	<485F1ADA.6090107@v.loewis.de>
	<loom.20080623T102812-837@post.gmane.org>
	<486010CD.2000904@v.loewis.de>
	<loom.20080624T093235-422@post.gmane.org>
	<1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com>
	<4862B0C4.2080901@v.loewis.de>
Message-ID: <acd65fa20806251635k249044acg93a667ccab017a2f@mail.gmail.com>

On Wed, Jun 25, 2008 at 4:55 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I think exactly the other way 'round. The timing of thing should not
> matter at all, only the exact sequence of allocations and deallocations.

I would it be possible, if not a good idea, to only track object
deallocations as the GC traversal trigger? As far as I know, dangling
cyclic references cannot be formed when allocating objects. So, this
could potentially mitigate the quadratic behavior during allocation
bursts.

-- Alexandre

From jmp at MIT.EDU  Wed Jun 25 23:13:49 2008
From: jmp at MIT.EDU (Justin Mazzola Paluska)
Date: Wed, 25 Jun 2008 17:13:49 -0400
Subject: [Python-Dev] Disable tests in unittest (issue3202)
Message-ID: <20080625211349.GA471@mit.edu>

Hi,

I just reported issue3202 in the bugracker
(http://bugs.python.org/issue3202) with a patch that implements a way
to disable unittest.TestCases using a decorator.  Benjamin Peterson
suggested that I bring up the issue on python-dev.

The idea behind the patch is that it?s sometimes useful to disable
tests without removing them from the TestCase.  For example, a
co-worker and I have a module with a few tests that will fail for the
forseeable future because we haven?t had a chance to implement the
features the tests are testing.  The tracebacks for these ?not
implemented yet? tests obscure real failures in other tests.

Normally, we?d just rename the failing methods from something starting
with ?test? to something like ?_test?, so unittest ignores them.
However, doing so removes all traces of the tests when you re-run the
test suite, so the disabled tests are easy to forget.

Instead, issue3202 allows you to write something like:

  from unittest import main, TestCase, disabled
  
  class SampleTest(TestCase):
  
      def test_addition(self):
  
          self.assertEqual(2, 1+1)
  
      def test_broken(self):

          # this is a real failure in our code      
          self.assertEqual(5, 2+2)

      @disabled
      def test_not_implemented(self):
  
          # Test of a feature not implemented yet.
          doit()
  if __name__ == '__main__':
      main()

which has one test disabled.  Running the test suite outputs

    %python sample_tests2.py 
    .FD
    ======================================================================
    FAIL: test_broken (__main__.SampleTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "sample_tests2.py", line 12, in test_broken
        self.assertEqual(5, 2+2)
    AssertionError: 5 != 4
    
    ----------------------------------------------------------------------
    Ran 3 tests in 0.001s
    
    FAILED (failures=1)

showing only the real failure and a simple ?D? for the disabled test.
The disabled test is still there, but doesn?t get in your way.

JUnit4 has a similar decorator for its tests, @Ignore.

The patch in issue3202 implements the decorator @disabled and a few
modifications to the classes in the unittest module.

What does the Python hivemind think?  I?m also open to enhancing it if
the list has any ideas.
	?Justin

From jml at mumak.net  Thu Jun 26 02:51:49 2008
From: jml at mumak.net (Jonathan Lange)
Date: Thu, 26 Jun 2008 10:51:49 +1000
Subject: [Python-Dev] Disable tests in unittest (issue3202)
In-Reply-To: <20080625211349.GA471@mit.edu>
References: <20080625211349.GA471@mit.edu>
Message-ID: <d06a5cd30806251751ie8bc983w9aabb292ab903845@mail.gmail.com>

On Thu, Jun 26, 2008 at 7:13 AM, Justin Mazzola Paluska <jmp at mit.edu> wrote:
> Hi,
>
...
> The idea behind the patch is that it's sometimes useful to disable
> tests without removing them from the TestCase.  For example, a
> co-worker and I have a module with a few tests that will fail for the
> forseeable future because we haven't had a chance to implement the
> features the tests are testing.  The tracebacks for these "not
> implemented yet" tests obscure real failures in other tests.
...
> The patch in issue3202 implements the decorator @disabled and a few
> modifications to the classes in the unittest module.
>
> What does the Python hivemind think?  I'm also open to enhancing it if
> the list has any ideas.

I think it's really worth looking at the approach that bzrlib's tests
take here (see bzrlib.tests.ExtendedTestResult and the
out-of-date-but-helpful http://bazaar-vcs.org/BzrExtendTestSuite)

Instead of disabling the tests, their practice is to run the tests but
have them raise KnownFailure. When they write a test that they know
will fail, they have it raise this exception. The extended TestResult
object catches this exception in addFailure and reports it
appropriately.[1]

So, when there's a test that is expected to fail, they do something like this:

    def test_foo(self):
          x = get_some_value()
          self.expectFailure('get_some_value is bogus', self.assertEqual, x, 42)

Using KnownFailure is better than disabling because you'll be able to
tell if the test suddenly and mysteriously passes.

I can see that disabling still has some value, but I'd rather have
KnownFailure first.

jml

[1] They use a similar strategy for skipping tests based on platform.
(You can't test Bazaar's symlink support on Windows, for example).

From facundobatista at gmail.com  Thu Jun 26 02:52:11 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 25 Jun 2008 21:52:11 -0300
Subject: [Python-Dev] Disable tests in unittest (issue3202)
In-Reply-To: <20080625211349.GA471@mit.edu>
References: <20080625211349.GA471@mit.edu>
Message-ID: <e04bdf310806251752m6f1fce1dwafb7a2ce48c6f99d@mail.gmail.com>

2008/6/25 Justin Mazzola Paluska <jmp at mit.edu>:

> The idea behind the patch is that it's sometimes useful to disable
> tests without removing them from the TestCase.  For example, a
> co-worker and I have a module with a few tests that will fail for the
> forseeable future because we haven't had a chance to implement the
> features the tests are testing.  The tracebacks for these "not
> implemented yet" tests obscure real failures in other tests.

What about TestSkipped? I thought that raising
test_support.TestSkipped should behave like this: you're saying that
you're actually NOT executing the tests, but you say that they are
there.

But a few days ago I needed to comment out a test that previously was
raising this, because it made a buildbot to go red.

How this should behave?

Thanks!

-- 
. Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From musiccomposition at gmail.com  Thu Jun 26 02:55:37 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Wed, 25 Jun 2008 19:55:37 -0500
Subject: [Python-Dev] Disable tests in unittest (issue3202)
In-Reply-To: <e04bdf310806251752m6f1fce1dwafb7a2ce48c6f99d@mail.gmail.com>
References: <20080625211349.GA471@mit.edu>
	<e04bdf310806251752m6f1fce1dwafb7a2ce48c6f99d@mail.gmail.com>
Message-ID: <1afaf6160806251755y1979651l29b6d7ca4d0d9e@mail.gmail.com>

On Wed, Jun 25, 2008 at 7:52 PM, Facundo Batista
<facundobatista at gmail.com> wrote:
>
> What about TestSkipped? I thought that raising
> test_support.TestSkipped should behave like this: you're saying that
> you're actually NOT executing the tests, but you say that they are
> there.

I think he's talking about outside core Python development (putting it
into unittest).

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From martin at v.loewis.de  Thu Jun 26 06:01:14 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 26 Jun 2008 06:01:14 +0200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <acd65fa20806251635k249044acg93a667ccab017a2f@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>	
	<1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de>	
	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>	
	<485F1ADA.6090107@v.loewis.de>	
	<loom.20080623T102812-837@post.gmane.org>	
	<486010CD.2000904@v.loewis.de>	
	<loom.20080624T093235-422@post.gmane.org>	
	<1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com>	
	<4862B0C4.2080901@v.loewis.de>
	<acd65fa20806251635k249044acg93a667ccab017a2f@mail.gmail.com>
Message-ID: <4863148A.3050304@v.loewis.de>

> I would it be possible, if not a good idea, to only track object
> deallocations as the GC traversal trigger? As far as I know, dangling
> cyclic references cannot be formed when allocating objects.

Not sure what you mean by that.

x = []
x.append(x)
del x

creates a cycle with no deallocation occurring.

Regards,
Martin

From alexandre at peadrop.com  Thu Jun 26 07:31:53 2008
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Thu, 26 Jun 2008 01:31:53 -0400
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <4863148A.3050304@v.loewis.de>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>
	<485F1ADA.6090107@v.loewis.de>
	<loom.20080623T102812-837@post.gmane.org>
	<486010CD.2000904@v.loewis.de>
	<loom.20080624T093235-422@post.gmane.org>
	<1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com>
	<4862B0C4.2080901@v.loewis.de>
	<acd65fa20806251635k249044acg93a667ccab017a2f@mail.gmail.com>
	<4863148A.3050304@v.loewis.de>
Message-ID: <acd65fa20806252231n518c7fd0w4968168b6d06391d@mail.gmail.com>

On Thu, Jun 26, 2008 at 12:01 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I would it be possible, if not a good idea, to only track object
>> deallocations as the GC traversal trigger? As far as I know, dangling
>> cyclic references cannot be formed when allocating objects.
>
> Not sure what you mean by that.
>
> x = []
> x.append(x)
> del x
>
> creates a cycle with no deallocation occurring.
>

Oh... never mind then.

-- Alexandre

From andrew-pythondev at puzzling.org  Thu Jun 26 07:49:16 2008
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Thu, 26 Jun 2008 15:49:16 +1000
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <bbaeab100806251018w42bd1b78qb1a98144ac453779@mail.gmail.com>
References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>
	<48621CC6.2030801@gmail.com>
	<20080625130857.GA13663@steerpike.home.puzzling.org>
	<bbaeab100806251018w42bd1b78qb1a98144ac453779@mail.gmail.com>
Message-ID: <20080626054916.GA24366@steerpike.home.puzzling.org>

Brett Cannon wrote:
> On Wed, Jun 25, 2008 at 6:08 AM, Andrew Bennetts
[...]
> >
> > Should I file a bug for this?
> >
> 
> If you want, but Benjamin plans to undocument this for users along
> with all other test.support stuff (which I agree with). Most of the
> APIs in test.support were just quickly written and have not
> necessarily been thought through in order to make sure that the APIs
> makes sense (like in this case).

Ok, then we're back to there being no supported way to write tests that need to
intercept warnings.  Twisted has already suffered from this (JP reports that
Twisted's assertWarns is broken in 2.6), and I doubt it's alone.

So I guess I am filing a bug after all... :)

-Andrew.


From hodgestar+pythondev at gmail.com  Thu Jun 26 10:58:03 2008
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Thu, 26 Jun 2008 10:58:03 +0200
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
	DeprecationWarning in stdlib)
In-Reply-To: <fb73205e0806260155k5ea418adr2e3bf5216c8e01c5@mail.gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>
	<ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>
	<bbaeab100806251527y4b54ee6ck83905abec53c70e6@mail.gmail.com>
	<fb73205e0806260155k5ea418adr2e3bf5216c8e01c5@mail.gmail.com>
Message-ID: <fb73205e0806260158x575a9435wd0b8c4df56a70854@mail.gmail.com>

On Thu, Jun 26, 2008 at 12:27 AM, Brett Cannon <brett at python.org> wrote:
> Right, but I would think all core developers know about test.support
> and are capable of reading the code and docstring.

Docstrings don't help for things other than Modules, Classes and
Functions.  So, for example, are constants like FUZZ intended to be
part of the external API provided by the module or not? Also, there
are already a fair number of functions in test_support that don't have
docstrings. Are these intended to be part of the API?

Obviously one can grep around the existing tests to see what they do,
but then one ends up perpetuating any errors others have made.

While one could possibly attempt to make a case for "all core
developers know" [1] it certainly isn't true that "all future core
developers know" [2] or that "all people who could put patches into
the issue tracker know" [3].

[1] Although personally I would bet against it until proven --
especially given a sensible definition of "core developer" to mean, in
this case, anyone with the right to commit tests to the repository.
[2] Since many of these future developers probably don't even use Python yet.
[3] And we do want patches to issues to include tests, right?

From python at rcn.com  Thu Jun 26 13:24:43 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 26 Jun 2008 04:24:43 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>
	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
Message-ID: <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>

From: "Guido van Rossum" <guido at python.org>
> I don't care about the details of the patch until we have agreement
> about which form the feature should take. We don't have that agreement
> yet.

Updated to the patch to address everyone's review comments:
       http://bugs.python.org/file10742/float8.diff

* Alexander Belopolsky requested exponential notation instead of potentially very long strings of bits.  Done

* Alexander Belopolsky requested  true mathematical radix 2 representation of a float rather than its 64-bit memory layout.  Done

* Antoine Pitrou requested that hex() and oct() be supported as well as bin().  Terry J. Reedy also requested support for hex(). 
Done.

* Alexander Belopolsky and Alexandre Vassalotti requested that the output be a two-way street -- something that can be round-tripped 
through eval().  Done.

* Amaury Forgeot d'Arc requested that the implementation not manipulate C strings inplace.  Fixed -- used  PyUnicode_FromFormat() 
instead.

* Amaury Forgeot d'Arc requested that tests should check if negative numbers have the same representation as their absolute value. 
Done.

* Mark Dickinson requested sign preserving output for bin(-0.0).  We couldn't find a clean way to do this without a special cased 
output format.

* Mark Dickinson reviewed the NaN/Inf handling.  Done.

* Eric Smith requested that the routine be attached to _PyFloat_to_base() instead of attaching to __bin__, __oct__, and __hex__. 
Done.

* Guido requested that the docs be updated. Done.

* Guido requested that the globally visible C API function name be prefixed with _Py.  Done.

* Mark Dickinson requested normalizing output to start with a 1 so that nearby values have similar reprs.  Done.


Raymond 


From p.f.moore at gmail.com  Thu Jun 26 14:16:33 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 26 Jun 2008 13:16:33 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
Message-ID: <79990c6b0806260516n5dd10c5bmde03f35fcb8e2156@mail.gmail.com>

On 26/06/2008, Raymond Hettinger <python at rcn.com> wrote:
> From: "Guido van Rossum" <guido at python.org>
> > I don't care about the details of the patch until we have agreement
> > about which form the feature should take. We don't have that agreement
> > yet.
> >
>
> Updated to the patch to address everyone's review comments:
>      http://bugs.python.org/file10742/float8.diff

Just as a contrary point, I'm not particularly keen on the output
format (which takes the form '0b1 * 2.0 ** 0' as far as I can see),
and I'm definitely not keen on the fact that it's overloaded on the
hex/bin/oct builtins.

Can't it be a separate function? If it is, I don't much care about the
output format (as I have no particular need for the feature) but would
it not be better if it were machine-parseable, rather than executable?

Paul

From eric+python-dev at trueblade.com  Thu Jun 26 14:23:49 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Thu, 26 Jun 2008 08:23:49 -0400
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>	<4861504B.209@v.loewis.de>	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
Message-ID: <48638A55.1030104@trueblade.com>

Actually, after saying I was opposed to __bin__ in 2.6, I said:
"Instead, I think the approach used in 3.0 (r64451) should be used 
instead.  That is, if this feature exist at all.  I'm -0 on adding 
bin(), etc. to floats."

My last sentence is a little unclear.  I meant I'm -0 on adding floats 
as arguments to bin(), oct(), and hex().  Primarily because a) it's not 
extensible in 3.0, and b) I find it surprising, in that I'd expect those 
functions to throw an error for non-integral types (that is, those not 
having __index__).  I think adding a "float_as_binary_expression()" 
(with a better name) in some module would get the functionality you 
seek.  What is gained by adding this to bin() and friends?

Raymond Hettinger wrote:
> From: "Guido van Rossum" <guido at python.org>
>> I don't care about the details of the patch until we have agreement
>> about which form the feature should take. We don't have that agreement
>> yet.
> 
> Updated to the patch to address everyone's review comments:
>       http://bugs.python.org/file10742/float8.diff
> 
> * Alexander Belopolsky requested exponential notation instead of 
> potentially very long strings of bits.  Done
> 
> * Alexander Belopolsky requested  true mathematical radix 2 
> representation of a float rather than its 64-bit memory layout.  Done
> 
> * Antoine Pitrou requested that hex() and oct() be supported as well as 
> bin().  Terry J. Reedy also requested support for hex(). Done.
> 
> * Alexander Belopolsky and Alexandre Vassalotti requested that the 
> output be a two-way street -- something that can be round-tripped 
> through eval().  Done.
> 
> * Amaury Forgeot d'Arc requested that the implementation not manipulate 
> C strings inplace.  Fixed -- used  PyUnicode_FromFormat() instead.
> 
> * Amaury Forgeot d'Arc requested that tests should check if negative 
> numbers have the same representation as their absolute value. Done.
> 
> * Mark Dickinson requested sign preserving output for bin(-0.0).  We 
> couldn't find a clean way to do this without a special cased output format.
> 
> * Mark Dickinson reviewed the NaN/Inf handling.  Done.
> 
> * Eric Smith requested that the routine be attached to 
> _PyFloat_to_base() instead of attaching to __bin__, __oct__, and 
> __hex__. Done.
> 
> * Guido requested that the docs be updated. Done.
> 
> * Guido requested that the globally visible C API function name be 
> prefixed with _Py.  Done.
> 
> * Mark Dickinson requested normalizing output to start with a 1 so that 
> nearby values have similar reprs.  Done.
> 
> 
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/eric%2Bpython-dev%40trueblade.com 
> 
> 


From musiccomposition at gmail.com  Thu Jun 26 14:45:44 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 26 Jun 2008 07:45:44 -0500
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
	DeprecationWarning in stdlib)
In-Reply-To: <fb73205e0806260158x575a9435wd0b8c4df56a70854@mail.gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>
	<ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>
	<bbaeab100806251527y4b54ee6ck83905abec53c70e6@mail.gmail.com>
	<fb73205e0806260155k5ea418adr2e3bf5216c8e01c5@mail.gmail.com>
	<fb73205e0806260158x575a9435wd0b8c4df56a70854@mail.gmail.com>
Message-ID: <1afaf6160806260545o60fe1f0cl996ad482c352d0b@mail.gmail.com>

On Thu, Jun 26, 2008 at 3:58 AM, Simon Cross
<hodgestar+pythondev at gmail.com> wrote:
>
> [1] Although personally I would bet against it until proven --
> especially given a sensible definition of "core developer" to mean, in
> this case, anyone with the right to commit tests to the repository.
> [2] Since many of these future developers probably don't even use Python yet.
> [3] And we do want patches to issues to include tests, right?

See my above message. I don't think we need to worry about the docs
part of "undocumented." All I mean by undocumented is that the API is
not guaranteed throughout versions, and test.support really should
only be used for core tests.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From solipsis at pitrou.net  Thu Jun 26 14:49:44 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 26 Jun 2008 12:49:44 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?=5BPython-checkins=5D_r64424_-_in=09python?=
	=?utf-8?q?/trunk=3AInclude/object=2EhLib/test/test=5Fsys=2Epy=09Mi?=
	=?utf-8?q?sc/NEWSObjects/intobject=2EcObjects/longobject=2Ec=09Obj?=
	=?utf-8?q?ects/typeobject=2EcPython/bltinmodule=2Ec?=
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>
	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
Message-ID: <loom.20080626T124537-652@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> 
> * Antoine Pitrou requested that hex() and oct() be supported as well as bin().

Just to qualify this, I said that if bin() were to gain float support, the
same should probably be done for hex() and oct(). That doesn't mean I'm in
favor of bin() support for floats.

Regards

Antoine.



From guido at python.org  Thu Jun 26 15:52:32 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 06:52:32 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
Message-ID: <ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>

Would you mind reading the rest of *this* thread on python-dev and
respond to the discussion about the design of the feature?

On Thu, Jun 26, 2008 at 4:24 AM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Guido van Rossum" <guido at python.org>
>>
>> I don't care about the details of the patch until we have agreement
>> about which form the feature should take. We don't have that agreement
>> yet.
>
> Updated to the patch to address everyone's review comments:
>      http://bugs.python.org/file10742/float8.diff
>
> * Alexander Belopolsky requested exponential notation instead of potentially
> very long strings of bits.  Done
>
> * Alexander Belopolsky requested  true mathematical radix 2 representation
> of a float rather than its 64-bit memory layout.  Done
>
> * Antoine Pitrou requested that hex() and oct() be supported as well as
> bin().  Terry J. Reedy also requested support for hex(). Done.
>
> * Alexander Belopolsky and Alexandre Vassalotti requested that the output be
> a two-way street -- something that can be round-tripped through eval().
>  Done.
>
> * Amaury Forgeot d'Arc requested that the implementation not manipulate C
> strings inplace.  Fixed -- used  PyUnicode_FromFormat() instead.
>
> * Amaury Forgeot d'Arc requested that tests should check if negative numbers
> have the same representation as their absolute value. Done.
>
> * Mark Dickinson requested sign preserving output for bin(-0.0).  We
> couldn't find a clean way to do this without a special cased output format.
>
> * Mark Dickinson reviewed the NaN/Inf handling.  Done.
>
> * Eric Smith requested that the routine be attached to _PyFloat_to_base()
> instead of attaching to __bin__, __oct__, and __hex__. Done.
>
> * Guido requested that the docs be updated. Done.
>
> * Guido requested that the globally visible C API function name be prefixed
> with _Py.  Done.
>
> * Mark Dickinson requested normalizing output to start with a 1 so that
> nearby values have similar reprs.  Done.
>
>
> Raymond
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Thu Jun 26 14:50:31 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 26 Jun 2008 05:50:31 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<79990c6b0806260516n5dd10c5bmde03f35fcb8e2156@mail.gmail.com>
Message-ID: <C0FF892B14B84E57802D34560B17048E@RaymondLaptop1>

> Just as a contrary point, I'm not particularly keen on the output
> format (which takes the form '0b1 * 2.0 ** 0' as far as I can see),

That format was requested by everyone else on the tracker
discussion.  What I originally wanted was something like 0b11.0101.
But that didn't round-trip through eval, it didn't match the style used 
in the numerical papers referenced by Terry Reedy, and it didn't scale
well with inputs like 1.1E+100.

> and I'm definitely not keen on the fact that it's overloaded on the
> hex/bin/oct builtins.
> 
> Can't it be a separate function? 

Simplicity.  bin/oct/hex have the job of giving alternate base representations for numbers.
Nothing is gained by adding a duplicate set of functions in the math module for float inputs.

> would
> it not be better if it were machine-parseable, rather than executable?

We already have struct.pack for machine-parseable output.
This is supposed to be human readable as well as providing
an exact, platform indepent way of specifying a particular
float value (something that's useful in testing algorithms like that in math.sum()).


Raymond



From ncoghlan at gmail.com  Thu Jun 26 15:56:23 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Jun 2008 23:56:23 +1000
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <20080626054916.GA24366@steerpike.home.puzzling.org>
References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>
	<48621CC6.2030801@gmail.com>
	<20080625130857.GA13663@steerpike.home.puzzling.org>
	<bbaeab100806251018w42bd1b78qb1a98144ac453779@mail.gmail.com>
	<20080626054916.GA24366@steerpike.home.puzzling.org>
Message-ID: <4863A007.60804@gmail.com>

Andrew Bennetts wrote:
> Brett Cannon wrote:
>> On Wed, Jun 25, 2008 at 6:08 AM, Andrew Bennetts
> [...]
>>> Should I file a bug for this?
>>>
>> If you want, but Benjamin plans to undocument this for users along
>> with all other test.support stuff (which I agree with). Most of the
>> APIs in test.support were just quickly written and have not
>> necessarily been thought through in order to make sure that the APIs
>> makes sense (like in this case).
> 
> Ok, then we're back to there being no supported way to write tests that need to
> intercept warnings.  Twisted has already suffered from this (JP reports that
> Twisted's assertWarns is broken in 2.6), and I doubt it's alone.
> 
> So I guess I am filing a bug after all... :)

Yeah - Brett's correct that everything under "test.test_support" should 
really be formally undocumented. It's mostly a place for code that 
reflects "things we do a lot in our unit tests and are tired of 
repeating" rather than "this is a good API that we want to support 
forever and encourage other people to use".

However, if other folks turn out to have similar needs, then it may be 
possible to add something to unittest to support it. However, given that 
the beta deadline has already passed, you may need to use similar 
hackery to that used by catch_warning and replace warnings.showwarning 
with a test function that saves the raised exception (it also wouldn't 
be hard to enhance it a bit to handle more than a single warning).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Thu Jun 26 16:08:02 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 00:08:02 +1000
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
 DeprecationWarning in stdlib)
In-Reply-To: <1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>	
	<ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>
	<1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com>
Message-ID: <4863A2C2.8000202@gmail.com>

Benjamin Peterson wrote:
> On Wed, Jun 25, 2008 at 4:00 PM, Guido van Rossum <guido at python.org> wrote:
>> I'm a little worried about making stuff undocumented that every core
>> developer needs to use -- everyone writing tests needs to continue to
>> use test_support (now test.support?). I imagine people writing unit
>> test suites for 3rd party libraries might want to use its services
>> too.
> 
> I think undocumented is a little unspecific here. What I mean is
> "reserved for core Python tests and no promise is made to retain
> compatibility." Of course, we would keep docs for them (perhaps in
> Lib/test/README), so new core developers could write their tests.

Alternatively, we could just leave them out of the docs table of 
contents, and stick a big warning at the top saying that these are 
internal APIs and subject to change without warning between releases.

(This discussion actually argues somewhat in favour of having _test as 
the top-level package name for the regression test suite - after all, we 
make *zero* promises about keeping the API of any of the test modules 
the same from release to release, so we should really be using the 
standard leading underscore convention to flag internal implementation 
details that are not really intended for use by third parties)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Thu Jun 26 16:08:24 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 07:08:24 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <C0FF892B14B84E57802D34560B17048E@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<79990c6b0806260516n5dd10c5bmde03f35fcb8e2156@mail.gmail.com>
	<C0FF892B14B84E57802D34560B17048E@RaymondLaptop1>
Message-ID: <ca471dc20806260708t365f326as5aa671485d69e6b8@mail.gmail.com>

On Thu, Jun 26, 2008 at 5:50 AM, Raymond Hettinger <python at rcn.com> wrote:
>> Just as a contrary point, I'm not particularly keen on the output
>> format (which takes the form '0b1 * 2.0 ** 0' as far as I can see),
>
> That format was requested by everyone else on the tracker
> discussion.  What I originally wanted was something like 0b11.0101.
> But that didn't round-trip through eval, it didn't match the style used in
> the numerical papers referenced by Terry Reedy, and it didn't scale
> well with inputs like 1.1E+100.
>
>> and I'm definitely not keen on the fact that it's overloaded on the
>> hex/bin/oct builtins.
>>
>> Can't it be a separate function?
>
> Simplicity.  bin/oct/hex have the job of giving alternate base
> representations for numbers.
> Nothing is gained by adding a duplicate set of functions in the math module
> for float inputs.

I disagree, and others here have disagreed too. We made a consicous
decision to *remove* the overloading of hex/oct/bin via
__hex__/__oct__/__bin__ in 3.0, in order to simplify these functions,
which only work for integers, not for any other style of numbers. If
bin(3.4) works, why not bin() of a Fraction, or of a complex number?
Or for that matter, why not hex() of a string? All these have some use
case. But is that use case important enough to put it in the
bin()/hex()/oct() built-in functions? Why not hex() of a dictionary?
Where does it end? We drew a line in the sand -- these are only for
ints.

>> would
>> it not be better if it were machine-parseable, rather than executable?
>
> We already have struct.pack for machine-parseable output.
> This is supposed to be human readable as well as providing
> an exact, platform indepent way of specifying a particular
> float value (something that's useful in testing algorithms like that in
> math.sum()).

The only use cases you bring up appear to be in testing and education.
This is not a strong enough motivation for adding a wart to the
bin/oct/hex builtins. I'm sure you can write the same thing in pure
Python -- wouldn't that be good enough for testing? And if you then
put it somewhere in a module in the stdlib, wouldn't that be good
enough for education?

There's a strong movement for keeping the language small and clean.
Adding more overloaded functionality to built-in functions goes
counter to that ideal. A new stdlib function causes no overhead to the
*language* definition (and built-ins *are* part of the language).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Thu Jun 26 16:12:17 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 26 Jun 2008 07:12:17 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>
	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
Message-ID: <FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>

> Would you mind reading the rest of *this* thread on python-dev and
> respond to the discussion about the design of the feature?

The last four entries were from this thread. I don't know what else you want me to do. I can update the patch as people make 
suggestions. That's pretty much it.

I recapped the earlier discussion from the tracker so the participants in this thread would be aware of the requests that were made 
there and why.  I originally wanted a different output format but it evolved to the one that's there now to meet the various needs 
of the posters.  This is important background for someone just joining the thread and thinking a different output format would be 
better.

There's a part of this thread that says basically, "fine, but stick it somewhere else."  To me, it doesn't make any sense at all to 
create a parallel set of functions in the math module. To convert a number to binary, it makes sense to use the bin() function.  I 
don't understand this notion that bin() is a sacred cow of intergerdom and would philosophically corrupt if it handled floats also.

Raymond 


From ncoghlan at gmail.com  Thu Jun 26 16:19:10 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 00:19:10 +1000
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <C0FF892B14B84E57802D34560B17048E@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>	<4861504B.209@v.loewis.de>	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>	<79990c6b0806260516n5dd10c5bmde03f35fcb8e2156@mail.gmail.com>
	<C0FF892B14B84E57802D34560B17048E@RaymondLaptop1>
Message-ID: <4863A55E.1050102@gmail.com>

Raymond Hettinger wrote:
>> and I'm definitely not keen on the fact that it's overloaded on the
>> hex/bin/oct builtins.
>>
>> Can't it be a separate function? 
> 
> Simplicity.  bin/oct/hex have the job of giving alternate base 
> representations for numbers.
> Nothing is gained by adding a duplicate set of functions in the math 
> module for float inputs.

I'd place additional requirements on using bin/oct/hex for this:

1. The new feature must be available to floating point types other than 
float (such as Decimal) in both 2.6 and 3.0 (keeping in mind that 3.0 
does not support __bin__, __hex__, or __oct__ methods - it uses only 
__index__ to implement bin(), hex() and oct()

2. Other classes (such as Decimal) should be able to leverage the 
formatting functionality provided for floats.

If it was just a new method on float objects or a new function in the 
math module, neither of those additional requirements would apply - I 
would be completely fine with the function only working for actual float 
objects.

However, in either case, I think this also runs afoul of the "we're in 
beta" argument - yes, it's a nice feature, but I don't think it's one 
that will cause any great dramas if users don't get their hands on it 
until 2.7/3.1.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Thu Jun 26 16:40:49 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 00:40:49 +1000
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>	<4861504B.209@v.loewis.de>	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
Message-ID: <4863AA71.5020901@gmail.com>

Raymond Hettinger wrote:
> There's a part of this thread that says basically, "fine, but stick it 
> somewhere else."  To me, it doesn't make any sense at all to create a 
> parallel set of functions in the math module. To convert a number to 
> binary, it makes sense to use the bin() function.  I don't understand 
> this notion that bin() is a sacred cow of intergerdom and would 
> philosophically corrupt if it handled floats also.

It isn't the extension to something other than integers that bothers me, 
it's the extension to floats in a way that can't be easily supported for 
other types such as decimal.Decimal and fractions.Fraction.

Well, that and the beta deadline (we have to draw the line somewhere, or 
we'll be stuck in an eternal spiral of "just one more feature")

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From exarkun at divmod.com  Thu Jun 26 16:46:46 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 26 Jun 2008 10:46:46 -0400
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <4863A007.60804@gmail.com>
Message-ID: <20080626144646.4714.191332201.divmod.quotient.13422@ohm>

On Thu, 26 Jun 2008 23:56:23 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> [snip]
>>
>>Ok, then we're back to there being no supported way to write tests that 
>>need to
>>intercept warnings.  Twisted has already suffered from this (JP reports 
>>that
>>Twisted's assertWarns is broken in 2.6), and I doubt it's alone.
>>
>>So I guess I am filing a bug after all... :)
>
>Yeah - Brett's correct that everything under "test.test_support" should 
>really be formally undocumented. It's mostly a place for code that reflects 
>"things we do a lot in our unit tests and are tired of repeating" rather 
>than "this is a good API that we want to support forever and encourage other 
>people to use".
>
>However, if other folks turn out to have similar needs, then it may be 
>possible to add something to unittest to support it. However, given that the 
>beta deadline has already passed, you may need to use similar hackery to 
>that used by catch_warning and replace warnings.showwarning with a test 
>function that saves the raised exception (it also wouldn't be hard to 
>enhance it a bit to handle more than a single warning).

We don't use showwarning because in order to reliably catch warnings that
way, it's necessary to rely on even more private implementation details of
the warning system.

Consider this:

    from warnings import warn
    from test.test_support import catch_warning

    def f():
        warn("foo")

    def test():
        with catch_warning() as w:
            f()
            assert str(w.message) == "foo", "%r != %r" % (w.message, "foo")

    test()
    test()

The first assertion passes (by the way, I don't understand why w.message
isn't the message passed to warn, but is instead an instance of UserWarning)
but the second assertion fails.  A more subtle example might include two
functions, the first of which is deprecated and called by the second, and
one test for each of them.  Now the test for the warning will only pass if
it runs before the other test; if they accidentally run in the other order,
you won't see the warning, so as far as I can tell, you can't reliably write
a unit test for warnings using catch_warning.

The real problem with testing many uses of the warning system is that it
doesn't expose enough public APIs for this to be possible.  You *have*
to use APIs which are, apparently, private (such as warn_explicit).

Jean-Paul

From ncoghlan at gmail.com  Thu Jun 26 16:52:57 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 00:52:57 +1000
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <35D8067A7D90402BB2A4DF125ADD5036@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>	<4861504B.209@v.loewis.de>	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>	<79990c6b0806260516n5dd10c5bmde03f35fcb8e2156@mail.gmail.com>
	<C0FF892B14B84E57802D34560B17048E@RaymondLaptop1>
	<4863A55E.1050102@gmail.com>
	<35D8067A7D90402BB2A4DF125ADD5036@RaymondLaptop1>
Message-ID: <4863AD49.4050202@gmail.com>

Raymond Hettinger wrote:
> FYI, I had already updated the patch to incorporate your suggestion.
> It checks __index__ first and then __float__ if index doesn't exist.
> The result makes it work with Decimal inputs.

Only sort of - the domain of the output is then constrained by what the 
builtin float type can represent, which strikes me as fairly 
inappropriate for what should be a lossless display operation (both 
fractions.Fraction and decimal.Decimal can represent a lot of numbers 
that float can't deal with properly). To use the old faithful example of 
this, given:

 >>> float(Decimal("1.1"))
1.1000000000000001

what happens for bin(Decimal("1.1")) with the patch version that uses 
__float__?

As I've said elsewhere in this thread, I'm certainly not averse to the 
basic idea of letting bin/oct/hex handle rational numbers as well as 
integers, I just don't see it as something we have to have in the 
current release cycle, and nor do I see it really working properly 
without adding a __rational__ special method that returns a 
numerator/denominator 2-tuple (which would obviously require a PEP).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From eric+python-dev at trueblade.com  Thu Jun 26 16:59:34 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Thu, 26 Jun 2008 10:59:34 -0400
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <48638A55.1030104@trueblade.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<ca471dc20806241123p36cbc770s46a80197b2d71f01@mail.gmail.com>	<DDF1D4053561439D96F25FBFF3462ECE@RaymondLaptop1>	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>	<4861504B.209@v.loewis.de>	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<48638A55.1030104@trueblade.com>
Message-ID: <4863AED6.2020702@trueblade.com>

Eric Smith wrote:
> Actually, after saying I was opposed to __bin__ in 2.6, I said:
> "Instead, I think the approach used in 3.0 (r64451) should be used 
> instead.  That is, if this feature exist at all.  I'm -0 on adding 
> bin(), etc. to floats."
> 
> My last sentence is a little unclear.  I meant I'm -0 on adding floats 
> as arguments to bin(), oct(), and hex().  Primarily because a) it's not 
> extensible in 3.0, and b) I find it surprising, in that I'd expect those 
> functions to throw an error for non-integral types (that is, those not 
> having __index__).  I think adding a "float_as_binary_expression()" 
> (with a better name) in some module would get the functionality you 
> seek.  What is gained by adding this to bin() and friends?

And to clarify myself, yet again (from private email with Raymond):

I actually think it's a useful feature, just not in bin().  I'm sure it 
will land somewhere, and I'm also sure I'll use it, at least from the 
interactive prompt.

And if bin() were generally extensible for all types, I wouldn't really 
even care all that much if this feature landed in bin().  But 3.0 is 
going in the opposite direction, which is what much of my concern is 
based on, and why I commented at the outset on the 2.6 approach 
differing from the 3.0 approach.

Eric.

From guido at python.org  Thu Jun 26 17:02:09 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 08:02:09 -0700
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <4863A007.60804@gmail.com>
References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>
	<48621CC6.2030801@gmail.com>
	<20080625130857.GA13663@steerpike.home.puzzling.org>
	<bbaeab100806251018w42bd1b78qb1a98144ac453779@mail.gmail.com>
	<20080626054916.GA24366@steerpike.home.puzzling.org>
	<4863A007.60804@gmail.com>
Message-ID: <ca471dc20806260802y6b8a6cf0t81201ca252d1c45f@mail.gmail.com>

On Thu, Jun 26, 2008 at 6:56 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Andrew Bennetts wrote:
>>
>> Brett Cannon wrote:
>>>
>>> On Wed, Jun 25, 2008 at 6:08 AM, Andrew Bennetts
>>
>> [...]
>>>>
>>>> Should I file a bug for this?
>>>>
>>> If you want, but Benjamin plans to undocument this for users along
>>> with all other test.support stuff (which I agree with). Most of the
>>> APIs in test.support were just quickly written and have not
>>> necessarily been thought through in order to make sure that the APIs
>>> makes sense (like in this case).
>>
>> Ok, then we're back to there being no supported way to write tests that
>> need to
>> intercept warnings.  Twisted has already suffered from this (JP reports
>> that
>> Twisted's assertWarns is broken in 2.6), and I doubt it's alone.
>>
>> So I guess I am filing a bug after all... :)
>
> Yeah - Brett's correct that everything under "test.test_support" should
> really be formally undocumented. It's mostly a place for code that reflects
> "things we do a lot in our unit tests and are tired of repeating" rather
> than "this is a good API that we want to support forever and encourage other
> people to use".

I still have a big problem with this attitude. If it's valuable enough
to share between our tests, it should be properly written for reuse
and documented. A collection of hacks remains a collection of hacks.
We can do better.

> However, if other folks turn out to have similar needs, then it may be
> possible to add something to unittest to support it. However, given that the
> beta deadline has already passed, you may need to use similar hackery to
> that used by catch_warning and replace warnings.showwarning with a test
> function that saves the raised exception (it also wouldn't be hard to
> enhance it a bit to handle more than a single warning).

I don't think the beta should be used as an argument to prevent
refactoring and documenting code. I think Andrew's complaint is
justified and should be addressed. I don't expect the test suite to
remain unchanged while we're in beta anyways...


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Thu Jun 26 17:12:40 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Jun 2008 11:12:40 -0400
Subject: [Python-Dev] [Python-checkins] r64424 -
	in	python/trunk:Include/object.h
	Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
	Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <4863AA71.5020901@gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>	<4861504B.209@v.loewis.de>	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
Message-ID: <328D5F1E-E082-4002-9A65-38A31625C161@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 26, 2008, at 10:40 AM, Nick Coghlan wrote:

> Well, that and the beta deadline (we have to draw the line  
> somewhere, or we'll be stuck in an eternal spiral of "just one more  
> feature")

Guido wanted to get the beta out when we did exactly so we could draw  
this line in the sand.  I'd much rather people be spending time  
getting what features we do have tested, stabilized, bug fixed, and  
turning the buildbots green across the board.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGOx6HEjvBPtnXfVAQK/IwP+MM75Kw0pMtzQ0wND51D6Gkyx/anr6xOf
uLQ41AhcPd//XDTfMQOO9DYsks1sxt8UnDtajyAFGKK9tZmymhwttplJpBdFDSM7
4km2tV9TZZJQUI/VUwV/d6IT80xfdct6c+lkOjeOl2KMnkPEuu+AoLnG0Me8FKiX
heC8FnyXut4=
=pG60
-----END PGP SIGNATURE-----

From ncoghlan at gmail.com  Thu Jun 26 17:14:36 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 01:14:36 +1000
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <20080626144646.4714.191332201.divmod.quotient.13422@ohm>
References: <20080626144646.4714.191332201.divmod.quotient.13422@ohm>
Message-ID: <4863B25C.5070706@gmail.com>

Jean-Paul Calderone wrote:
> The first assertion passes (by the way, I don't understand why w.message
> isn't the message passed to warn, but is instead an instance of 
> UserWarning)
> but the second assertion fails.

See Brett's comment elsewhere in this thread about the level of thought 
(or lack thereof) that went into the design of some of the test_support 
APIs (including catch_warning).


> A more subtle example might include two
> functions, the first of which is deprecated and called by the second, and
> one test for each of them.  Now the test for the warning will only pass if
> it runs before the other test; if they accidentally run in the other order,
> you won't see the warning, so as far as I can tell, you can't reliably 
> write
> a unit test for warnings using catch_warning.

Yeah, the Python regression tests actually cheat quite a bit when it 
comes to testing warnings - several of the tests clear the warning cache 
just in case the warning has already been triggered by other code.

I suspect what some of those tests *should* be doing (aside from those 
testing the warning module itself) is configuring the warning system to 
convert the relevant warnings to exceptions, then catching the resulting 
exception.

> The real problem with testing many uses of the warning system is that it
> doesn't expose enough public APIs for this to be possible.  You *have*
> to use APIs which are, apparently, private (such as warn_explicit).

Hmm, I think the bigger problem is that there is no documented way to 
save the warnings filter and restore it to a previous state - the 
'filters' attribute (which holds the actual list of filters) isn't 
documented and isn't included in __all__. This makes it hard to write an 
officially supported test that fiddles with the warning settings then 
puts them back the way they were.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From glyph at divmod.com  Thu Jun 26 17:18:55 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 26 Jun 2008 15:18:55 -0000
Subject: [Python-Dev] Community buildbots and Python release quality metrics
Message-ID: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>

Today on planetpython.org, Doug Hellman announced the June issue of 
Python magazine.  The cover story this month is about Pybots, "the 
fantastic automation system that has been put in place to make sure new 
releases of Python software are as robust and stable as possible".

Last week, there was a "beta" release of Python which, according to the 
community buildbots, cannot run any existing python software.  Normally 
I'd be complaining here about Twisted, but in fact Twisted is doing 
relatively well right now; only 80 failing tests.  Django apparently 
cannot even be imported.

The community buildbots have been in a broken state for months now[1]. 
I've been restraining myself from whinging about this, but now that it's 
getting close to release, it's time to get these into shape, or it's 
time to get rid of them.

There are a few separate issues here.  One is, to what extent should 
Python actually be compatible between releases?  There's not much point 
in saying "this release is compatible with python 2.5" if all python 2.5 
programs need to be modified in order to run on python 2.6.  Of course, 
it's quite likely that there are some 3rd-party Python programs that 
continue to work, but python.org provides links to 2 projects that don't 
and none that do.

Another way to phrase this question is, "whose responsibility is it to 
make Python 2.5 programs run on Python 2.6"?  Or, "what happens when the 
core team finds out that a change they have made has broken some python 
software 'in the wild'"?

Here are a couple of ways to answer these questions:

  1) It's the python core team's responsibility.  There should be Python 
buildbots for lots of different projects and they should all be green 
before the code can be considered [one of: allowed in trunk, alpha, 
beta, RC].
  2) It's the python core team's responsibility as long as the major 
projects are using public APIs.  The code should not be considered 
[alpha, beta, RC] if there are any known breakages in public APIs.
  3) It's only the python core team's responsibility to notify projects 
of incompatible changes of which they are aware, which may occur in 
public or private APIs.  Major projects with breakages will be given a 
grace period before a [beta, RC, final] to release a forward-compatible 
version.
  4) The status quo: every new Python release can (and will, at least 
speaking in terms of 2.6) break lots of popular software, with no 
warning aside from the beta period.  There are no guarantees.

For obvious reasons, I'd prefer #1, but any of the above is better than 
#4.  I don't think #4 is an intentional choice, either; I think it's the 
result of there being no clear consequence for community buildbots 
failing.  Or, for that matter, of core buildbots failing.  Investigating 
the history is tricky (I see a bunch of emails from Barry here, but I'm 
not sure which is the final state) but I believe the beta went out with 
a bunch of buildbots offline or failing.

A related but independent issue is: should the community buildbots 
continue to exist?  An automated test is only as good as the 
consequences of its failure.  As it stands, the community buildbots seem 
to be nothing but an embarrassment.  I don't mean this to slam Grig, or 
the Python team - I mean, literally, if there is no consequence of their 
failure then the only use I can see for the data the buildbots currently 
provide (i.e. months and months of failures) is to embarrass the Python 
core developers.  python.org's purpose should not be to provide negative 
press for Python ;), so if this continues to be the case, they probably 
shouldn't be linked.  This isn't just an issue with changes to Python 
breaking stuff; the pybots' configuration is apparently not well 
maintained, since the buildbots which say "2.5" aren't passing either, 
and that's not a problem with the code, it's just that the slaves are 
actually running against trunk at HEAD.

Ultimately these tools only exist to ensure the quality of Python 
releases.  The really critical question here is, what does it mean to 
have a Python release that is high-quality?  There are some obvious 
things: it shouldn't crash, it should conform to its own documentation. 
Personally I think "it passes all of its own tests" and "it runs 
existing Python code" are important metrics too, but the most important 
thing I'm trying to get across here is that there should be a clear 
understanding of which goals the release / QA / buildbot / test process 
is trying to accomplish.  For me, personally, it really needs to be 
clear when I can say "you guys screwed up and broke stuff", and when I 
just have to suck it up and deal with the consequences of a new version 
of Python in Twisted.

It's definitely bad for all of us if the result is that new releases of 
Python just break everything.  Users don't care where the responsibility 
lies, they just know that stuff doesn't work, so we should decide on a 
process which allows Twisted (and other popular projects, like Django, 
Plone, pytz, PyGTK, pylons, ...) to (mostly) run when new versions of 
Python are released.

[1]: http://mail.python.org/pipermail/python-dev/2008-May/079212.html

From guido at python.org  Thu Jun 26 17:19:31 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 08:19:31 -0700
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
	DeprecationWarning in stdlib)
In-Reply-To: <4863A2C2.8000202@gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>
	<ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>
	<1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com>
	<4863A2C2.8000202@gmail.com>
Message-ID: <ca471dc20806260819h4293b840v1149806cf81c376c@mail.gmail.com>

On Thu, Jun 26, 2008 at 7:08 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Benjamin Peterson wrote:
>>
>> On Wed, Jun 25, 2008 at 4:00 PM, Guido van Rossum <guido at python.org>
>> wrote:
>>>
>>> I'm a little worried about making stuff undocumented that every core
>>> developer needs to use -- everyone writing tests needs to continue to
>>> use test_support (now test.support?). I imagine people writing unit
>>> test suites for 3rd party libraries might want to use its services
>>> too.
>>
>> I think undocumented is a little unspecific here. What I mean is
>> "reserved for core Python tests and no promise is made to retain
>> compatibility." Of course, we would keep docs for them (perhaps in
>> Lib/test/README), so new core developers could write their tests.
>
> Alternatively, we could just leave them out of the docs table of contents,
> and stick a big warning at the top saying that these are internal APIs and
> subject to change without warning between releases.
>
> (This discussion actually argues somewhat in favour of having _test as the
> top-level package name for the regression test suite - after all, we make
> *zero* promises about keeping the API of any of the test modules the same
> from release to release, so we should really be using the standard leading
> underscore convention to flag internal implementation details that are not
> really intended for use by third parties)

That would also remove the problem users encounter from time to time
with module files named "test.py".

But I think we should think about this more. I don't think anyone
expects the code inside any particular test_foo.py to have a stable
public interface. But quite a bit of the test support infrastructure
is reused by third party test frameworks. I think we should
acknowledge and support such reuse.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Jun 26 17:21:36 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 01:21:36 +1000
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <328D5F1E-E082-4002-9A65-38A31625C161@python.org>
References: <20080620041816.4D5E81E4002@bag.python.org>	<ca471dc20806241205p6fbbae00v41e4d6b055e586c1@mail.gmail.com>	<5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com>	<4861504B.209@v.loewis.de>	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
Message-ID: <4863B400.4090401@gmail.com>

Barry Warsaw wrote:
> On Jun 26, 2008, at 10:40 AM, Nick Coghlan wrote:
> 
>> Well, that and the beta deadline (we have to draw the line somewhere, 
>> or we'll be stuck in an eternal spiral of "just one more feature")
> 
> Guido wanted to get the beta out when we did exactly so we could draw 
> this line in the sand.  I'd much rather people be spending time getting 
> what features we do have tested, stabilized, bug fixed, and turning the 
> buildbots green across the board.

Having been caught by the 2.5 beta deadline with the changes that 
eventually became PEP 361 (and I think were significantly improved by 
the additional attention that was available due to the delay) I 
understand completely.

(And to everyone with features that get bumped to 2.7/3.1 because of 
this... while a number of you no doubt know this already, it really is 
astonishing how soon the next release seems to roll around, even with 
our fairly leisurely release schedules!)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From p.f.moore at gmail.com  Thu Jun 26 17:27:01 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 26 Jun 2008 16:27:01 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <4863AED6.2020702@trueblade.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4861504B.209@v.loewis.de>
	<ca471dc20806241313t721571e0s25b259a25c8e2228@mail.gmail.com>
	<FC36FA3B393C49248083D063A0FF89FE@RaymondLaptop1>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<48638A55.1030104@trueblade.com> <4863AED6.2020702@trueblade.com>
Message-ID: <79990c6b0806260827g39b10270g4ccfb06ba4640cbf@mail.gmail.com>

On 26/06/2008, Eric Smith <eric+python-dev at trueblade.com> wrote:
> I actually think it's a useful feature, just not in bin().  I'm sure it will
> land somewhere, and I'm also sure I'll use it, at least from the interactive
> prompt.

Can you give an example of its use? Maybe there are such examples in
the tracker discussion, but I'm not following the tracker, and I think
it would add some weight to the discussion here if use cases were
noted.

At the moment, the impression I get is that most of the python-dev
comments are negative, but (without looking to confirm) most of the
tracker comments are positive. If so, getting everything in one place
will make it easier to see a balanced discussion.

Having said all this, I agree with the point that this should be
deferred to 2.7/3.1 now we're in beta. So there's hardly a rush.

Paul.

From exarkun at divmod.com  Thu Jun 26 17:28:01 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 26 Jun 2008 11:28:01 -0400
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <4863B25C.5070706@gmail.com>
Message-ID: <20080626152801.4714.909118230.divmod.quotient.13430@ohm>

On Fri, 27 Jun 2008 01:14:36 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
>Jean-Paul Calderone wrote:
> [snip]
>>The real problem with testing many uses of the warning system is that it
>>doesn't expose enough public APIs for this to be possible.  You *have*
>>to use APIs which are, apparently, private (such as warn_explicit).
>
>Hmm, I think the bigger problem is that there is no documented way to save 
>the warnings filter and restore it to a previous state - the 'filters' 
>attribute (which holds the actual list of filters) isn't documented and 
>isn't included in __all__. This makes it hard to write an officially 
>supported test that fiddles with the warning settings then puts them back 
>the way they were.

It sounds like you're agreeing that there aren't enough public APIs.
Making warn_explicit public addresses this particular problem in one
way - by letting applications hook into the warning system before
filters are applied.  Making the filter list public is another way,
since it would let applications clear and then restore the filters.

I don't particularly care about the details, I just want some public
API for this.  Making warn_explicit public seems better to me, since
it was already there in previous versions of Python, and it lets you
completely ignore both the filters list and the global registry, but
if others would rather make the filters and global registry a part of
the public API, that's fine by me as well.

Jean-Paul

From guido at python.org  Thu Jun 26 17:28:36 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 08:28:36 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <4863B400.4090401@gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
Message-ID: <ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>

On Thu, Jun 26, 2008 at 8:21 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Barry Warsaw wrote:
>>
>> On Jun 26, 2008, at 10:40 AM, Nick Coghlan wrote:
>>
>>> Well, that and the beta deadline (we have to draw the line somewhere, or
>>> we'll be stuck in an eternal spiral of "just one more feature")
>>
>> Guido wanted to get the beta out when we did exactly so we could draw this
>> line in the sand.  I'd much rather people be spending time getting what
>> features we do have tested, stabilized, bug fixed, and turning the buildbots
>> green across the board.
>
> Having been caught by the 2.5 beta deadline with the changes that eventually
> became PEP 361 (and I think were significantly improved by the additional
> attention that was available due to the delay) I understand completely.
>
> (And to everyone with features that get bumped to 2.7/3.1 because of this...
> while a number of you no doubt know this already, it really is astonishing
> how soon the next release seems to roll around, even with our fairly
> leisurely release schedules!)

I'd like to separate the concerns though.

I personally don't want to see the feature in its current form added
to 2.7/3.1 either. As others pointed out, it's a wart on the
bin/oct/hex functions.

So as far as the feature design goes, I offer some suggestions: a new
module; or a new function in math; or a new method on float. Since
Raymond is the champion for the feature let him choose the API from
those alternatives.

Regarding the addition to 2.6/3.0 post beta 1, I think a new module
has the most chance of success, especially if it's written in Python
in such a way as to need minimal changes between 2.6 and 3.0.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Jun 26 17:32:08 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 01:32:08 +1000
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
 DeprecationWarning in stdlib)
In-Reply-To: <ca471dc20806260819h4293b840v1149806cf81c376c@mail.gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>	
	<ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>	
	<1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com>	
	<4863A2C2.8000202@gmail.com>
	<ca471dc20806260819h4293b840v1149806cf81c376c@mail.gmail.com>
Message-ID: <4863B678.5040205@gmail.com>

Guido van Rossum wrote:
> But I think we should think about this more. I don't think anyone
> expects the code inside any particular test_foo.py to have a stable
> public interface. But quite a bit of the test support infrastructure
> is reused by third party test frameworks. I think we should
> acknowledge and support such reuse.

I'm personally fine with that approach, but some of the new items in 
there for 2.6 could probably use a bit of cleaning up to improve the 
APIs and/or the way they work.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Thu Jun 26 17:33:23 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 08:33:23 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
Message-ID: <ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>

Too verbose, Glyph. :-)

It needs to be decided case-by-case. The beta is called beta because,
well, it may break stuff and we may want to fix it. But there are also
likely many frameworks that use undefined behavior. I'm not
particularly impressed by statistics like "all tests are red" -- this
may all be caused by a single issue. For example, I'd much rather read
an explanation about *why* Django cannot even be imported than a
blanket complaint that this is a disgrace. So why is it?

--Guido

On Thu, Jun 26, 2008 at 8:18 AM,  <glyph at divmod.com> wrote:
> Today on planetpython.org, Doug Hellman announced the June issue of Python
> magazine.  The cover story this month is about Pybots, "the fantastic
> automation system that has been put in place to make sure new releases of
> Python software are as robust and stable as possible".
>
> Last week, there was a "beta" release of Python which, according to the
> community buildbots, cannot run any existing python software.  Normally I'd
> be complaining here about Twisted, but in fact Twisted is doing relatively
> well right now; only 80 failing tests.  Django apparently cannot even be
> imported.
>
> The community buildbots have been in a broken state for months now[1]. I've
> been restraining myself from whinging about this, but now that it's getting
> close to release, it's time to get these into shape, or it's time to get rid
> of them.
>
> There are a few separate issues here.  One is, to what extent should Python
> actually be compatible between releases?  There's not much point in saying
> "this release is compatible with python 2.5" if all python 2.5 programs need
> to be modified in order to run on python 2.6.  Of course, it's quite likely
> that there are some 3rd-party Python programs that continue to work, but
> python.org provides links to 2 projects that don't and none that do.
>
> Another way to phrase this question is, "whose responsibility is it to make
> Python 2.5 programs run on Python 2.6"?  Or, "what happens when the core
> team finds out that a change they have made has broken some python software
> 'in the wild'"?
>
> Here are a couple of ways to answer these questions:
>
>  1) It's the python core team's responsibility.  There should be Python
> buildbots for lots of different projects and they should all be green before
> the code can be considered [one of: allowed in trunk, alpha, beta, RC].
>  2) It's the python core team's responsibility as long as the major projects
> are using public APIs.  The code should not be considered [alpha, beta, RC]
> if there are any known breakages in public APIs.
>  3) It's only the python core team's responsibility to notify projects of
> incompatible changes of which they are aware, which may occur in public or
> private APIs.  Major projects with breakages will be given a grace period
> before a [beta, RC, final] to release a forward-compatible version.
>  4) The status quo: every new Python release can (and will, at least
> speaking in terms of 2.6) break lots of popular software, with no warning
> aside from the beta period.  There are no guarantees.
>
> For obvious reasons, I'd prefer #1, but any of the above is better than #4.
>  I don't think #4 is an intentional choice, either; I think it's the result
> of there being no clear consequence for community buildbots failing.  Or,
> for that matter, of core buildbots failing.  Investigating the history is
> tricky (I see a bunch of emails from Barry here, but I'm not sure which is
> the final state) but I believe the beta went out with a bunch of buildbots
> offline or failing.
>
> A related but independent issue is: should the community buildbots continue
> to exist?  An automated test is only as good as the consequences of its
> failure.  As it stands, the community buildbots seem to be nothing but an
> embarrassment.  I don't mean this to slam Grig, or the Python team - I mean,
> literally, if there is no consequence of their failure then the only use I
> can see for the data the buildbots currently provide (i.e. months and months
> of failures) is to embarrass the Python core developers.  python.org's
> purpose should not be to provide negative press for Python ;), so if this
> continues to be the case, they probably shouldn't be linked.  This isn't
> just an issue with changes to Python breaking stuff; the pybots'
> configuration is apparently not well maintained, since the buildbots which
> say "2.5" aren't passing either, and that's not a problem with the code,
> it's just that the slaves are actually running against trunk at HEAD.
>
> Ultimately these tools only exist to ensure the quality of Python releases.
>  The really critical question here is, what does it mean to have a Python
> release that is high-quality?  There are some obvious things: it shouldn't
> crash, it should conform to its own documentation. Personally I think "it
> passes all of its own tests" and "it runs existing Python code" are
> important metrics too, but the most important thing I'm trying to get across
> here is that there should be a clear understanding of which goals the
> release / QA / buildbot / test process is trying to accomplish.  For me,
> personally, it really needs to be clear when I can say "you guys screwed up
> and broke stuff", and when I just have to suck it up and deal with the
> consequences of a new version of Python in Twisted.
>
> It's definitely bad for all of us if the result is that new releases of
> Python just break everything.  Users don't care where the responsibility
> lies, they just know that stuff doesn't work, so we should decide on a
> process which allows Twisted (and other popular projects, like Django,
> Plone, pytz, PyGTK, pylons, ...) to (mostly) run when new versions of Python
> are released.
>
> [1]: http://mail.python.org/pipermail/python-dev/2008-May/079212.html
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jun 26 17:34:25 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 08:34:25 -0700
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
	DeprecationWarning in stdlib)
In-Reply-To: <4863B678.5040205@gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>
	<ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>
	<1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com>
	<4863A2C2.8000202@gmail.com>
	<ca471dc20806260819h4293b840v1149806cf81c376c@mail.gmail.com>
	<4863B678.5040205@gmail.com>
Message-ID: <ca471dc20806260834v18a6fe25k5daa51c4551e7f85@mail.gmail.com>

On Thu, Jun 26, 2008 at 8:32 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>>
>> But I think we should think about this more. I don't think anyone
>> expects the code inside any particular test_foo.py to have a stable
>> public interface. But quite a bit of the test support infrastructure
>> is reused by third party test frameworks. I think we should
>> acknowledge and support such reuse.
>
> I'm personally fine with that approach, but some of the new items in there
> for 2.6 could probably use a bit of cleaning up to improve the APIs and/or
> the way they work.

So get crackin'!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Jun 26 17:42:56 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 01:42:56 +1000
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
Message-ID: <4863B900.1040808@gmail.com>

glyph at divmod.com wrote:
> Today on planetpython.org, Doug Hellman announced the June issue of 
> Python magazine.  The cover story this month is about Pybots, "the 
> fantastic automation system that has been put in place to make sure new 
> releases of Python software are as robust and stable as possible".
> 
> Last week, there was a "beta" release of Python which, according to the 
> community buildbots, cannot run any existing python software.  Normally 
> I'd be complaining here about Twisted, but in fact Twisted is doing 
> relatively well right now; only 80 failing tests.  Django apparently 
> cannot even be imported.

beta 1 has some trouble running *our* test suite - I'd be fairly 
surprised if the community buildbots were in significantly better shape.

> The community buildbots have been in a broken state for months now[1].

Continuously running community buildbots on the maintenance trees makes 
sense, since those trees should always be in a releasable state. For the 
trunk, they're really only interesting when the Python core buildbots 
are reporting all green, but some of the community buildbots are 
reporting red.

One of the problems is what the term "beta" means to different groups - 
for us, this first beta was really about saying "zero new features from 
here on, focus on making what we have now work properly". The relatively 
late landing of a couple of major PEPs (371, 3108) also didn't do any 
favours for trunk stability.

If the community buildbots aren't largely green by the time beta 2 comes 
out, that's when I'll agree we have a problem - they should definitely 
be green by the time first release candidate comes out.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Thu Jun 26 17:47:57 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 01:47:57 +1000
Subject: [Python-Dev] [Python-checkins] r64424 - in
 python/trunk:Include/object.h Lib/test/test_sys.py
 Misc/NEWSObjects/intobject.c Objects/longobject.c
 Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>	
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>	
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>	
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>	
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>	
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>	
	<4863AA71.5020901@gmail.com>	
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>	
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
Message-ID: <4863BA2D.8030805@gmail.com>

Guido van Rossum wrote:
> I personally don't want to see the feature in its current form added
> to 2.7/3.1 either. As others pointed out, it's a wart on the
> bin/oct/hex functions.
> 
> So as far as the feature design goes, I offer some suggestions: a new
> module; or a new function in math; or a new method on float. Since
> Raymond is the champion for the feature let him choose the API from
> those alternatives.
> 
> Regarding the addition to 2.6/3.0 post beta 1, I think a new module
> has the most chance of success, especially if it's written in Python
> in such a way as to need minimal changes between 2.6 and 3.0.

One of the other reasons I'd like to postpone the feature is that I 
think with a clean design behind it it could be an elegant addition to 
those builtins rather than a wart.

But helping Raymond to convince you or anyone else of that is well down 
my to-do list at the moment (which I think just got longer with the 
test_support discussion...)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Thu Jun 26 17:52:18 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 01:52:18 +1000
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <20080626152801.4714.909118230.divmod.quotient.13430@ohm>
References: <20080626152801.4714.909118230.divmod.quotient.13430@ohm>
Message-ID: <4863BB32.3040101@gmail.com>

Jean-Paul Calderone wrote:
> I don't particularly care about the details, I just want some public
> API for this.  Making warn_explicit public seems better to me, since
> it was already there in previous versions of Python, and it lets you
> completely ignore both the filters list and the global registry, but
> if others would rather make the filters and global registry a part of
> the public API, that's fine by me as well.

Why do you say warn_explicit isn't public? It's in both the 2.5 and 2.6 
API docs for the warnings module.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From exarkun at divmod.com  Thu Jun 26 18:01:18 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 26 Jun 2008 12:01:18 -0400
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <4863BB32.3040101@gmail.com>
Message-ID: <20080626160118.4714.38891548.divmod.quotient.13435@ohm>

On Fri, 27 Jun 2008 01:52:18 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
>Jean-Paul Calderone wrote:
>>I don't particularly care about the details, I just want some public
>>API for this.  Making warn_explicit public seems better to me, since
>>it was already there in previous versions of Python, and it lets you
>>completely ignore both the filters list and the global registry, but
>>if others would rather make the filters and global registry a part of
>>the public API, that's fine by me as well.
>
>Why do you say warn_explicit isn't public? It's in both the 2.5 and 2.6 API 
>docs for the warnings module.

Brett told me it was private (on this list several weeks or a month or so
ago).  It's also no longer called in 2.6 by the C implementation of the
warning system.

Jean-Paul

From glyph at divmod.com  Thu Jun 26 18:21:08 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 26 Jun 2008 16:21:08 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
Message-ID: <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>

On 03:33 pm, guido at python.org wrote:
>Too verbose, Glyph. :-)

Mea culpa.  "Glyph" might be a less appropriate moniker than "Altogether 
too many glyphs".
>It needs to be decided case-by-case.

If certain tests are to be ignored on a case-by-case basis, why not 
record that decision by disabling the test in the code?  Otherwise, the 
decision inevitably gets boiled down to "it's okay to release the code 
with a bunch of tests failing, but I don't know which ones".  As it was 
on Twisted when we used to make case-by-case decisions about failures, 
and as it is here now.
>The beta is called beta because, well, it may break stuff and we may 
>want to fix it.

That's also why the alpha is called an alpha.  My informal understanding 
is that a beta should have no (or at least very few) known issues, with 
a medium level of confidence that no further ones will be found.  An RC 
would have absolutely no known issues with a fairly high level of 
confidence that no further ones will be found.  This would include the 
community buildbots basically working for the most part; I would not be 
surprised if there were a few tests that still had issues.

But clearly the reality does not meet my informal expectations, so it 
would be nice to have something written down to check against.  Still, 
I'm bringing this up now because it _is_ a beta, and I think it will be 
too late to talk about dealing with persistent test failures after the 
RCs start coming out.

(Of course, I'm just being sneaky.  I don't actually care if it's 
clearly documented, I just care that I stop having problems with 
incompatibility.  But I believe having it clearly spelled out would 
actually prevent a lot of the problems that I've been having, since I 
don't think anyone would *intentionally* select a policy where every 
release breaks at least one major dependent project.)
>I'm not particularly impressed by statistics like "all tests are red" 
>-- this
>may all be caused by a single issue.

The issue, for me, is not specifically that tests are red.  It's that 
there's no clear policy about what to do about that.  Can a release go 
out with some of the tests being red?  If so, what are the extenuating 
circumstances?  Does this have to be fixed?  If not, why not?  Why are 
we talking about this now?  Shouldn't the change which caused Django to 
become unimportable have been examined at the time, rather than months 
later?  (In other words, if it *is* a single issue, great, it's easy to 
fix: revert that single issue.)  If not, then shouldn't someone in 
Django-land have been notified so they could cope with the change?

Sorry that there are so many questions here; if I had fewer, I'd use 
fewer words to ask them.
>For example, I'd much rather read an explanation about *why* Django 
>cannot even be imported than a blanket complaint that this is a 
>disgrace. So why is it?

I don't know.  JP is already addressing the issues affecting Twisted in 
another thread (incompatible changes in the private-but-necessary-to- 
get-any-testing-done API of the warnings module).  But I really think 
that whoever made the change which broke it should be the one 
investigating it, not me.

From glyph at divmod.com  Thu Jun 26 18:33:46 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 26 Jun 2008 16:33:46 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <4863B900.1040808@gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<4863B900.1040808@gmail.com>
Message-ID: <20080626163346.25821.1622910462.divmod.xquotient.10493@joule.divmod.com>

On 03:42 pm, ncoghlan at gmail.com wrote:
>glyph at divmod.com wrote:

>beta 1 has some trouble running *our* test suite - I'd be fairly 
>surprised if the community buildbots were in significantly better 
>shape.

That's another problem, yes :)
>>The community buildbots have been in a broken state for months now[1].

>If the community buildbots aren't largely green by the time beta 2 
>comes out, that's when I'll agree we have a problem - they should 
>definitely be green by the time first release candidate comes out.

I have mixed feelings about this: on the one hand, if this were a matter 
of public record, I would have known that this was too early to start 
complaining, and we could have saved everyone a lot of time reading 
these messages.  That would be good; I would prefer to just let everyone 
work without bothering about process.

On the other hand, it's much easier to deal with test failures as they 
arise than in one giant chunk of work at the end of the development 
process.  I spoke to a few core developers at PyCon who thought that the 
buildbots should always be green and any change that broke them should 
be reverted.  They may remain nameless unless they wish to raise their 
hands :-) but that's definitely how I feel about it.
>Continuously running community buildbots on the maintenance trees makes 
>sense, since those trees should always be in a releasable state. For 
>the trunk, they're really only interesting when the Python core 
>buildbots are reporting all green, but some of the community buildbots 
>are reporting red.

(... which should be all the time ...)
>One of the problems is what the term "beta" means to different groups - 
>for us, this first beta was really about saying "zero new features from 
>here on, focus on making what we have now work properly". The 
>relatively late landing of a couple of major PEPs (371, 3108) also 
>didn't do any favours for trunk stability.

A big part of why I wrote this message is that I wanted a clear 
understanding of the relationship between the definition of  "alpha", 
"beta" and "RC" and the state of various buildbots.  If that 
relationship exists already, just linking to it from 
http://python.org/download/releases/2.6/ would be good.  By the way, 
that page still says these are "alpha" releases.

From schmir at gmail.com  Thu Jun 26 18:42:33 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Thu, 26 Jun 2008 18:42:33 +0200
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
Message-ID: <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>

On Thu, Jun 26, 2008 at 5:33 PM, Guido van Rossum <guido at python.org> wrote:

> an explanation about *why* Django cannot even be imported than a
> blanket complaint that this is a disgrace. So why is it?
>


  File "/home/ralf/pediapress/appserver/django/db/models/options.py",
line 198, in _many_to_many
self._fill_m2m_cache()
  File "/home/ralf/pediapress/appserver/django/db/models/options.py",
line 221, in _fill_m2m_cache
cache[field] = None
  File "/home/ralf/pediapress/appserver/django/utils/datastructures.py",
line 75, in __setitem__
super(SortedDict, self).__setitem__(key, value)
TypeError: unhashable type: 'ManyToManyField'


same for sqlalchemy:
egg/sqlalchemy/schema.py", line 113, in __call__
return type.__call__(self, name, metadata, *args, **kwargs)
  File "/home/ralf/py26/lib/python2.6/site-packages/SQLAlchemy-0.5.0beta1-py2.6.egg/sqlalchemy/schema.py",
line 207, in __init__
self.primary_key = PrimaryKeyConstraint()
  File "/home/ralf/py26/lib/python2.6/site-packages/SQLAlchemy-0.5.0beta1-py2.6.egg/sqlalchemy/schema.py",
line 294, in _set_primary_key
self.constraints.add(pk)
TypeError: unhashable type: 'PrimaryKeyConstraint'


and already discussed:
http://mail.python.org/pipermail/python-dev/2008-April/078421.html

- Ralf

From scott+python-dev at scottdial.com  Thu Jun 26 18:52:58 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Thu, 26 Jun 2008 12:52:58 -0400
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
Message-ID: <4863C96A.9030802@scottdial.com>

Ralf Schmitt wrote:
> TypeError: unhashable type: 'ManyToManyField'
> TypeError: unhashable type: 'PrimaryKeyConstraint'
> 
> and already discussed:
> http://mail.python.org/pipermail/python-dev/2008-April/078421.html

Following the discussion to it's conclusions leads one to a tracker 
issue [1] that was supposed to be a blocker for the beta but (I assume) 
was forgotten about. Clearly this should be promoted to being a blocker 
again and some decision made.

[1] http://bugs.python.org/issue2235

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From glyph at divmod.com  Thu Jun 26 18:54:49 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 26 Jun 2008 16:54:49 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
Message-ID: <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>

On 04:42 pm, schmir at gmail.com wrote:
>On Thu, Jun 26, 2008 at 5:33 PM, Guido van Rossum <guido at python.org> 
>wrote:
>>an explanation about *why* Django cannot even be imported than a
>>blanket complaint that this is a disgrace. So why is it?

>and already discussed:
>http://mail.python.org/pipermail/python-dev/2008-April/078421.html

Following that trail of breadcrumbs, I ended up here:

    http://bugs.python.org/issue2235

with a message from some guy named "Barry Warsaw" (anyone know him?) 
that says:

  """
  Guido, can you comment on Amaury's latest patch?  I'm going to bump 
this
  back to critical so as not to hold up 2.6 alpha, but it should be 
marked
  as a release blocker for the first beta.
  """

I don't know if this "Barry" guy has the appropriate permissions on the 
bugtracker to increase priorities, so I've taken the liberty of 
upgrading it as a release blocker for the _second_ beta ... ;-).  So, at 
least there's been one productive consequence of this discussion.

From guido at python.org  Thu Jun 26 19:03:59 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 10:03:59 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>
Message-ID: <ca471dc20806261003y140f1b29sba2cdd11ef657835@mail.gmail.com>

On Thu, Jun 26, 2008 at 9:21 AM,  <glyph at divmod.com> wrote:
> On 03:33 pm, guido at python.org wrote:
>> It needs to be decided case-by-case.
>
> If certain tests are to be ignored on a case-by-case basis, why not record
> that decision by disabling the test in the code?  Otherwise, the decision
> inevitably gets boiled down to "it's okay to release the code with a bunch
> of tests failing, but I don't know which ones".  As it was on Twisted when
> we used to make case-by-case decisions about failures, and as it is here
> now.

No, I just meant that we need to figure out for each 3rd party test
that fails whether the failure is our fault (too incompatibile) or
theirs (relied on undefined behavior) and what the best fix is (change
our code or theirs -- note that even if it's there fault there are
cases where the best fix is to change our code.

>> The beta is called beta because, well, it may break stuff and we may want
>> to fix it.
>
> That's also why the alpha is called an alpha.  My informal understanding is
> that a beta should have no (or at least very few) known issues, with a
> medium level of confidence that no further ones will be found.  An RC would
> have absolutely no known issues with a fairly high level of confidence that
> no further ones will be found.  This would include the community buildbots
> basically working for the most part; I would not be surprised if there were
> a few tests that still had issues.

I guess we differ in interpretation. I see alphas as early outreach to
the community. I see betas as a line in the sand where we stop
adding/changing functionality -- but certainly not the moment where we
expect everything to be perfect. During the beta period we fix bugs
only -- those could be problems with new features, but those could
also be problems where we accidentally introduced a backwards
incompatibility.

I'm sorry if your interpretation of the terminology is different, but
this is mine and this is what we've always used, and it's not likely
to change. (At least not for the 2.6/3.0 release.)

> But clearly the reality does not meet my informal expectations, so it would
> be nice to have something written down to check against.  Still, I'm
> bringing this up now because it _is_ a beta, and I think it will be too late
> to talk about dealing with persistent test failures after the RCs start
> coming out.

Absolutely correct. The RCs are hoped to be as good as the final
release. *Now* is the time to bring up issue.

But please bring up specific issues -- I don't want to have an
extended discussion about process or quality or expectations. I just
want the code to be fixed.

> (Of course, I'm just being sneaky.  I don't actually care if it's clearly
> documented, I just care that I stop having problems with incompatibility.
>  But I believe having it clearly spelled out would actually prevent a lot of
> the problems that I've been having, since I don't think anyone would
> *intentionally* select a policy where every release breaks at least one
> major dependent project.)

And here we seem to be parting our ways. We have a large amount of
process already. I don't want more.

>> I'm not particularly impressed by statistics like "all tests are red" --
>> this
>> may all be caused by a single issue.
>
> The issue, for me, is not specifically that tests are red.  It's that
> there's no clear policy about what to do about that.  Can a release go out
> with some of the tests being red?  If so, what are the extenuating
> circumstances?  Does this have to be fixed?  If not, why not?  Why are we
> talking about this now?  Shouldn't the change which caused Django to become
> unimportable have been examined at the time, rather than months later?  (In
> other words, if it *is* a single issue, great, it's easy to fix: revert that
> single issue.)  If not, then shouldn't someone in Django-land have been
> notified so they could cope with the change?
>
> Sorry that there are so many questions here; if I had fewer, I'd use fewer
> words to ask them.

If you're talking about community buildbots (which I presume are
testing 3rd party packages against the core release) being red, that's
out of scope for the core developers. We have enough trouble keeping
our own (core) buildbots green, as you may have noticed. Some of the
core buildbots are red because, well, frankly, they run on a cranky
old version of an OS that few people care about. And then sometimes we
leave them red, or turn them off.

I see the core buildbots as a tool for developers to have their
changes tested on a wide variety of platforms. They are an internal
tool that gives us guidance about whether we're ready.

I hope the community buildbots can be used the same way: a red bot
means someone needs to look into an issue. The issue could be with the
core or with the 3rd party package being tested. I don't think a
policy like "no community buildbots should be red" makes any sense.

>> For example, I'd much rather read an explanation about *why* Django cannot
>> even be imported than a blanket complaint that this is a disgrace. So why is
>> it?
>
> I don't know.  JP is already addressing the issues affecting Twisted in
> another thread (incompatible changes in the private-but-necessary-to-
> get-any-testing-done API of the warnings module).  But I really think that
> whoever made the change which broke it should be the one investigating it,
> not me.

Whoever made what change? You can't seriously expect core developers
investigating issues with 3rd party packages, no matter what the
cause. The correct process is that someone who cares about the 3rd
party package (could be an end user, a developer of that package, or a
core developer who happens to care) looks into the issue enough to
diagnose it, and then either proposes a fix or files a bug with the
likely culprit, which could be the core or the 3rd party package. If
nobody cares, well, that's open source too.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jun 26 19:07:12 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 10:07:12 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
Message-ID: <ca471dc20806261007h5e1d456eyce74b48542e0b262@mail.gmail.com>

And just to make my position perfectly clear, I've unassigned it,
since I don't foresee to be able to give this issue the quality time
it clearly needs. Mind you, I agree it's a release blocker. But I
don't have time to personally investigate it. Sorry.

On Thu, Jun 26, 2008 at 9:54 AM,  <glyph at divmod.com> wrote:
> On 04:42 pm, schmir at gmail.com wrote:
>>
>> On Thu, Jun 26, 2008 at 5:33 PM, Guido van Rossum <guido at python.org>
>> wrote:
>>>
>>> an explanation about *why* Django cannot even be imported than a
>>> blanket complaint that this is a disgrace. So why is it?
>
>> and already discussed:
>> http://mail.python.org/pipermail/python-dev/2008-April/078421.html
>
> Following that trail of breadcrumbs, I ended up here:
>
>   http://bugs.python.org/issue2235
>
> with a message from some guy named "Barry Warsaw" (anyone know him?) that
> says:
>
>  """
>  Guido, can you comment on Amaury's latest patch?  I'm going to bump this
>  back to critical so as not to hold up 2.6 alpha, but it should be marked
>  as a release blocker for the first beta.
>  """
>
> I don't know if this "Barry" guy has the appropriate permissions on the
> bugtracker to increase priorities, so I've taken the liberty of upgrading it
> as a release blocker for the _second_ beta ... ;-).  So, at least there's
> been one productive consequence of this discussion.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Thu Jun 26 19:14:32 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jun 2008 19:14:32 +0200
Subject: [Python-Dev] Community buildbots and Python
	release	quality	metrics
In-Reply-To: <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>
Message-ID: <4863CE78.7010108@v.loewis.de>

> That's also why the alpha is called an alpha.  My informal understanding
> is that a beta should have no (or at least very few) known issues

No, that's not the purpose. Instead, it is a promise that no further
features will be added, i.e. that the code is stable from a feature
point of view.

It certainly will contain bugs. The final release will certainly contain
bugs, just look at the long list of open bug reports.

> The issue, for me, is not specifically that tests are red.  It's that
> there's no clear policy about what to do about that.  Can a release go
> out with some of the tests being red?  If so, what are the extenuating
> circumstances?

The final release, no. The beta release: sure (in particular if it's the
first beta release).

Also make a difference between the Python buildbots, and the community
buildbots. I think few if any core committers ever look at the status
of the community buildbots - the community has to bring breakage to
our attention (as you just did).

> Does this have to be fixed?  If not, why not?

Here, I'm confused what specifically you talk about. That Django doesn't
import? It is certainly not part of the release procedure to verify that
Django can still be imported.

> Why are
> we talking about this now?  Shouldn't the change which caused Django to
> become unimportable have been examined at the time, rather than months
> later?

Somebody should have pointed it out when it happened. Unfortunately,
nobody did, so apparently, the community doesn't really care.

> I don't know.  JP is already addressing the issues affecting Twisted in
> another thread (incompatible changes in the private-but-necessary-to-
> get-any-testing-done API of the warnings module).  But I really think
> that whoever made the change which broke it should be the one
> investigating it, not me.

How could they have known that they broke it?

Regards,
Martin

From martin at v.loewis.de  Thu Jun 26 19:23:47 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jun 2008 19:23:47 +0200
Subject: [Python-Dev] Community buildbots and Python
	release	quality	metrics
In-Reply-To: <20080626163346.25821.1622910462.divmod.xquotient.10493@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<4863B900.1040808@gmail.com>
	<20080626163346.25821.1622910462.divmod.xquotient.10493@joule.divmod.com>
Message-ID: <4863D0A3.50600@v.loewis.de>

> A big part of why I wrote this message is that I wanted a clear
> understanding of the relationship between the definition of  "alpha",
> "beta" and "RC" and the state of various buildbots.  If that
> relationship exists already, just linking to it from
> http://python.org/download/releases/2.6/ would be good.  By the way,
> that page still says these are "alpha" releases.

I think its significantly simpler to answer specific questions on the
mailing list, to educate the specific set of people who have this
question, than to put an official answer on the web page.

So I'm skeptical that anybody will change the web page - just continue
asking questions as you encounter them.

Regards,
Martin

From schmir at gmail.com  Thu Jun 26 19:40:25 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Thu, 26 Jun 2008 19:40:25 +0200
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
Message-ID: <932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com>

On Thu, Jun 26, 2008 at 6:54 PM,  <glyph at divmod.com> wrote:
> On 04:42 pm, schmir at gmail.com wrote:
>>
>> On Thu, Jun 26, 2008 at 5:33 PM, Guido van Rossum <guido at python.org>
>> wrote:
>>>
>>> an explanation about *why* Django cannot even be imported than a
>>> blanket complaint that this is a disgrace. So why is it?
>
>> and already discussed:
>> http://mail.python.org/pipermail/python-dev/2008-April/078421.html
>
> Following that trail of breadcrumbs, I ended up here:
>
>   http://bugs.python.org/issue2235
>
> with a message from some guy named "Barry Warsaw" (anyone know him?) that
> says:
>
>  """
>  Guido, can you comment on Amaury's latest patch?  I'm going to bump this
>  back to critical so as not to hold up 2.6 alpha, but it should be marked
>  as a release blocker for the first beta.
>  """
>
> I don't know if this "Barry" guy has the appropriate permissions on the
> bugtracker to increase priorities, so I've taken the liberty of upgrading it
> as a release blocker for the _second_ beta ... ;-).  So, at least there's
> been one productive consequence of this discussion.

this patch even make things worse for me. now py.test also dies.

From guido at python.org  Thu Jun 26 19:55:30 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 10:55:30 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
	<932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com>
Message-ID: <ca471dc20806261055g5219d34fi23b84f9462af8962@mail.gmail.com>

On Thu, Jun 26, 2008 at 10:40 AM, Ralf Schmitt <schmir at gmail.com> wrote:
> this patch even make things worse for me. now py.test also dies.

Please add details to the tracker.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From schmir at gmail.com  Thu Jun 26 20:07:41 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Thu, 26 Jun 2008 20:07:41 +0200
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <ca471dc20806261055g5219d34fi23b84f9462af8962@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
	<932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com>
	<ca471dc20806261055g5219d34fi23b84f9462af8962@mail.gmail.com>
Message-ID: <932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com>

On Thu, Jun 26, 2008 at 7:55 PM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Jun 26, 2008 at 10:40 AM, Ralf Schmitt <schmir at gmail.com> wrote:
>> this patch even make things worse for me. now py.test also dies.
>
> Please add details to the tracker.
>

Well, I think most probably the patch is correct and it just catches
another class which defines __eq__ but does not define __hash__. So
nothing to add there.
The question is if this behaviour should be kept or not. I guess
there's no bug tracking that...

From python at rcn.com  Thu Jun 26 20:07:55 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 26 Jun 2008 11:07:55 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>
	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
Message-ID: <DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>

From: "Guido van Rossum" <guido at python.org>
> So as far as the feature design goes, I offer some suggestions: a new
> module; or a new function in math; or a new method on float. Since
> Raymond is the champion for the feature let him choose the API from
> those alternatives.

I choose bin/hex/oct methods on floatobjects.
Will work-up a patch.


Raymond


From guido at python.org  Thu Jun 26 20:45:06 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 11:45:06 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
	<932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com>
	<ca471dc20806261055g5219d34fi23b84f9462af8962@mail.gmail.com>
	<932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com>
Message-ID: <ca471dc20806261145m164374bnb4dcdf770c2be0fa@mail.gmail.com>

On Thu, Jun 26, 2008 at 11:07 AM, Ralf Schmitt <schmir at gmail.com> wrote:
> On Thu, Jun 26, 2008 at 7:55 PM, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Jun 26, 2008 at 10:40 AM, Ralf Schmitt <schmir at gmail.com> wrote:
>>> this patch even make things worse for me. now py.test also dies.
>>
>> Please add details to the tracker.

> Well, I think most probably the patch is correct and it just catches
> another class which defines __eq__ but does not define __hash__. So
> nothing to add there.

So you're saying py.test needs to be fixed? Fine with me, but then I
don't understand why you bothered bringing it up here instead of
fixing it (or reporting to the py.test maintainers, I don't know if
you're one or not).

> The question is if this behaviour should be kept or not. I guess
> there's no bug tracking that...

The best place is that very same issue. If this feature is really too
annoying something needs to be done. Detailed reports about real code
that gets broken without a good cause help making that case.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Jun 26 20:50:08 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 11:50:08 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
Message-ID: <ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>

On Thu, Jun 26, 2008 at 11:07 AM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Guido van Rossum" <guido at python.org>
>>
>> So as far as the feature design goes, I offer some suggestions: a new
>> module; or a new function in math; or a new method on float. Since
>> Raymond is the champion for the feature let him choose the API from
>> those alternatives.
>
> I choose bin/hex/oct methods on floatobjects.
> Will work-up a patch.

Let's step back and discuss the API some more.

- Do we need all three?

- If so, why not .tobase(N)? (Even if N is restricted to 2, 8 and 16.)

- What should the output format be? I know you originally favored
0b10101.010101 etc. Now that it's not overloaded on the bin/oct/hex
builtins, the constraint that it needs to be an eval() able expression
may be dropped (unless you see a use case for that too).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jmp at MIT.EDU  Thu Jun 26 20:57:27 2008
From: jmp at MIT.EDU (Justin Mazzola Paluska)
Date: Thu, 26 Jun 2008 14:57:27 -0400
Subject: [Python-Dev] Disable tests in unittest (issue3202)
In-Reply-To: <d06a5cd30806251751ie8bc983w9aabb292ab903845@mail.gmail.com>
References: <20080625211349.GA471@mit.edu>
	<d06a5cd30806251751ie8bc983w9aabb292ab903845@mail.gmail.com>
Message-ID: <20080626185727.GA29900@mit.edu>

On Thu, Jun 26, 2008 at 10:51:49AM +1000, Jonathan Lange wrote:
> On Thu, Jun 26, 2008 at 7:13 AM, Justin Mazzola Paluska <jmp at mit.edu> wrote:
> I think it's really worth looking at the approach that bzrlib's tests
> take here (see bzrlib.tests.ExtendedTestResult and the
> out-of-date-but-helpful http://bazaar-vcs.org/BzrExtendTestSuite)
> 
> Instead of disabling the tests, their practice is to run the tests but
> have them raise KnownFailure. When they write a test that they know
> will fail, they have it raise this exception. The extended TestResult
> object catches this exception in addFailure and reports it
> appropriately.[1]

I wasn?t aware of bzrlib?s extentions to unittest ? many of them,
including KnownFailure, TestSkipped,
bzrlib.test.ExtendedTestResult.expectFailure, and the sundry list of
bzrlib.test.TestCase.assert* look useful enough to belong in the
standard library.

<snip ? example>
> I can see that disabling still has some value, but I'd rather have
> KnownFailure first.

I think disabling and the bzr extensions have different use cases.
KnownFailure lets someone with an intimate knowledge of the the
failing test case write down that they expect the test to fail.  The
disabled decorator lets someone less familiar with the test cases to
easily disable tests that are getting in their way, without needing to
understand the body of the test function (or worse, mis-understanding
it and breaking it in a hard-to-understand way).

I agree that in the ideal world, we?d like to have both.  In fact, if
we had the bzr extensions, it?d be easy to write the disabled
decorator in terms of their KnownFailure:

def disabled(func):
    @wraps(func):
    def wrapper(*args, **kwargs):
        raise KnownFailure("%s disabled" % func.__name__)
    return wrapper

However, unless Python?s unittest is going to incorporate the useful
features of bzrlib?s TestCase object[*], it probably makes sense to
implement the disabled feature separately.
	?Justin

[*] Perhaps in the long term, this is the right thing to do as
unittest seems to have fallen behind in useful feature count as its
?conceptual? parent JUnit.

From g.brandl at gmx.net  Thu Jun 26 21:33:58 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 26 Jun 2008 21:33:58 +0200
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <ca471dc20806260802y6b8a6cf0t81201ca252d1c45f@mail.gmail.com>
References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>	<48621CC6.2030801@gmail.com>	<20080625130857.GA13663@steerpike.home.puzzling.org>	<bbaeab100806251018w42bd1b78qb1a98144ac453779@mail.gmail.com>	<20080626054916.GA24366@steerpike.home.puzzling.org>	<4863A007.60804@gmail.com>
	<ca471dc20806260802y6b8a6cf0t81201ca252d1c45f@mail.gmail.com>
Message-ID: <g40qos$85s$1@ger.gmane.org>

Guido van Rossum schrieb:

>>> Ok, then we're back to there being no supported way to write tests that
>>> need to
>>> intercept warnings.  Twisted has already suffered from this (JP reports
>>> that
>>> Twisted's assertWarns is broken in 2.6), and I doubt it's alone.
>>>
>>> So I guess I am filing a bug after all... :)
>>
>> Yeah - Brett's correct that everything under "test.test_support" should
>> really be formally undocumented. It's mostly a place for code that reflects
>> "things we do a lot in our unit tests and are tired of repeating" rather
>> than "this is a good API that we want to support forever and encourage other
>> people to use".
> 
> I still have a big problem with this attitude. If it's valuable enough
> to share between our tests, it should be properly written for reuse
> and documented. A collection of hacks remains a collection of hacks.
> We can do better.

Since this is part of Benjamin's project, we will make sure that the
test_support that emerges from it will be properly documented, stable
and usable.

Georg


From glyph at divmod.com  Thu Jun 26 21:32:10 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 26 Jun 2008 19:32:10 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <ca471dc20806261003y140f1b29sba2cdd11ef657835@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>
	<ca471dc20806261003y140f1b29sba2cdd11ef657835@mail.gmail.com>
Message-ID: <20080626193210.25821.328171947.divmod.xquotient.10633@joule.divmod.com>


I do tend to ramble on, so here's an executive summary of my response:

I want python developers to pay attention to the community buildbots and 
to treat breakages of existing projects as a serious issue.  However, I 
don't think that maintaining those projects is the core team's job, so 
all I'm asking for is for core developers to:

  * treat breakages of 3rd party packages as a potentially serious issue,
  * if possible (i.e. if they find out about the breakage soon enough, 
which should be the case in any pybots failure) revert the change that 
caused the problem until the problem can be fixed, and
  * notify 3rd party maintainers when it's decided that the breakage will 
not be fixed.

This only applies to breakages that the core developers find out about, 
which for all practical purposes means the ones on the community 
builders page.

Those of you looking for point-by-point responses and some repetition of 
the above points, enjoy :).

On 05:03 pm, guido at python.org wrote:
>On Thu, Jun 26, 2008 at 9:21 AM,  <glyph at divmod.com> wrote:
>>On 03:33 pm, guido at python.org wrote:
>>>It needs to be decided case-by-case.
>>(misunderstanding)
>No, I just meant that we need to figure out for each 3rd party test
>that fails whether the failure is our fault (too incompatibile) or
>theirs (relied on undefined behavior) and what the best fix is (change
>our code or theirs -- note that even if it's there fault there are
>cases where the best fix is to change our code.

This is basically fine, as far as I'm concerned.

I would like to suggest, however, that these issues be dealt with as 
soon as possible, rather than waiting for the release process to begin. 
A lot of decisions are made on this mailing list about the supposed 
properties of "average" python code, without any actual survey of said 
code.  Sometimes the results of that survey can be really surprising. 
The end goal of any particular compatibility policy, of a distinction 
between "public" and "private" APIs, and so on, is to keep code working.
>I'm sorry if your interpretation of the terminology is different, but
>this is mine and this is what we've always used, and it's not likely
>to change. (At least not for the 2.6/3.0 release.)

I have no problem with your definitions of these terms.  I think that 
they should probably be in PEP 101 though.  Would you accept a patch 
that added an edited / expanded version of this paragraph?
>>Still, I'm bringing this up now because it _is_ a beta,

>Absolutely correct. The RCs are hoped to be as good as the final
>release. *Now* is the time to bring up issue.

Well, that's good, at least :)
>But please bring up specific issues -- I don't want to have an
>extended discussion about process or quality or expectations. I just
>want the code to be fixed.

Well, one specific issue has been bumped in priority as a result of this 
thread, and others are under discussion.  The code is getting fixed.
>>(I just care that I stop having problems with incompatibility.)
>
>And here we seem to be parting our ways. We have a large amount of
>process already. I don't want more.

Looking at it from my perspective, I'm proposing a reduction in process. 
Under the current process, if a buildbot goes red, the developer makes a 
judgment call, the release manager makes a judgment call, there's 
discussion on a ticket, a ticket gets filed, it gets promoted, it gets 
demoted, the RM forgets to re-promote it...

My suggestion is that the process be, simply: if a buildbot (community 
or otherwise) goes red, the change that broke it gets reverted.  No 
questions asked!  It's still there in the revision history, ready to be 
re-applied once the issues get worked out.  Discussion can then take 
place and case-by-case judgments can be applied.
>If you're talking about community buildbots (which I presume are
>testing 3rd party packages against the core release) being red, that's
>out of scope for the core developers.

I don't necessarily think that keeping the community buildbots red is 
the core developers' responsibility, but I don't think it should be 
entirely out of scope, either.  The python test suite is, frankly, poor 
- and I hope I'm not surprising you by saying that.  It's full of race 
conditions, tends to fail intermittently, and is frequently ignored. 
Not only that, but it is quite often changed, so tests for issues that 
affect real code are quite often removed.  So, the community buildbots 
are not just making sure that 3rd-party code still works, they are an 
expanded, independently developed test suite to make sure that *python 
itself* still works.  Sometimes they will not fill that role 
particularly well, but they are worth paying attention to.

If python had a good, exhaustive regression test suite that was 
immutable between major versions, I'd probably feel differently.  But 
that's not the world we live in.

Right now, apparently, the *default* policy is that if the community 
buildbots go red, later, before a release, someone will maybe take a 
look at it.  I'd suggest that the *default* policy ought to be that if a 
particular changeset breaks a community buildbot, it needs further 
examination before being merged to trunk.

However, this is just the way I prefer to do development; if you think 
that would slow things down too much, the only thing I'm _really_ asking 
for is a clear statement that says "there should be no test failures on 
community buildbots that have not been explicitly accepted before a 
final release".  I'm not even sure what "explicitly accepted" means - 
you have to sign off?  the release manager, maybe?  A discussion on this 
list?  I don't really care, as long as _somebody_ does.

Right now, my impression of the process is this:

  * The community buildbot goes red; no core developer looks at it.
    * If the project is Twisted, JP fixes the bug on Twisted's end.
    * If the project is Django, nobody notices.
  * Months later, a beta goes out.  A few people try it out and report 
some bugs, but don't really understand the output.  A good number go un- 
triaged.
  * A little while later, a final release comes out.  Many projects are 
broken as a result.

This is not a hypothetical concern.  This is what happened with 2.5; 
Twisted was broken for months, and Zope *to this day* does not support 
Python 2.5.  2.6 looks like it's headed for the same trajectory.  To be 
clear: this is with all of Python's _own_ tests passing, so it is 
specific to paying attention to community buildbots.  (And the community 
buildbots only build django and twisted right now.  I'm not talking 
about a massive pan-galactic survey of all possible python projects. 
I'm only talking about those popular enough to make this select list. 
Which should still be a slightly longer list, but I digress...)
>Some of the
>core buildbots are red because, well, frankly, they run on a cranky
>old version of an OS that few people care about.

On Twisted, we have a distinction between "supported" and "unsupported" 
platforms, to provide the ability to run on platforms which aren't 
really supported and don't really run the whole suite, but we are 
nevertheless interested in.  I don't believe the setup is too hard and 
we'll definitely help out with that if you want to do it.  (I believe 
Thomas Herve volunteered to do this at PyCon...)
>I hope the community buildbots can be used the same way: a red bot
>means someone needs to look into an issue. The issue could be with the
>core or with the 3rd party package being tested. I don't think a
>policy like "no community buildbots should be red" makes any sense.

These bots have been red for months.  The issues exist, but have not 
been looked into.  As a result, Barry made a specific commitment on a 
ticket (i.e. "this should block beta 1") which was not met.  I think 
_something_ has to be changed to encourage people to do this more 
immediately or more seriously.
>Whoever made what change? You can't seriously expect core developers
>investigating issues with 3rd party packages, no matter what the
>cause. The correct process is that someone who cares about the 3rd
>party package (could be an end user, a developer of that package, or a
>core developer who happens to care) looks into the issue enough to
>diagnose it, and then either proposes a fix or files a bug with the
>likely culprit, which could be the core or the 3rd party package. If
>nobody cares, well, that's open source too.

If the breakage is calculated and expected, and the benefits clearly 
oughtweigh the costs... oh well, too bad for the 3rd party people.  It 
would be nice if the core developers would notify the third party if 
they find out about it so that it can be verified that the change in 
question wasn't obscuring some *other* problem, but from what I've seen, 
the breakages that I have been concerned about have not been 
intentional, calculated changes, but side-effects of other things.

I'm talking about the case where the breakage reveals either a bug in 
Python, or an unintentional / side-effect change in behavior, which is 
surprisingly frequent.

From glyph at divmod.com  Thu Jun 26 21:35:58 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 26 Jun 2008 19:35:58 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <4863CE78.7010108@v.loewis.de>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>
	<4863CE78.7010108@v.loewis.de>
Message-ID: <20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com>


On 05:14 pm, martin at v.loewis.de wrote:
>>I don't know.  JP is already addressing the issues affecting Twisted 
>>in
>>another thread (incompatible changes in the private-but-necessary-to-
>>get-any-testing-done API of the warnings module).  But I really think
>>that whoever made the change which broke it should be the one
>>investigating it, not me.
>
>How could they have known that they broke it?

Because the relevant community buildbot turned red with that revision of 
trunk.  Keep in mind I'm not talking about every piece of Python code in 
the universe; just the ones selected for the community buildbots.  It 
would be nice if there were a dozen or so projects on there, but paying 
attention to the two builders that do actually run right now would be a 
good start.

From g.brandl at gmx.net  Thu Jun 26 21:44:22 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 26 Jun 2008 21:44:22 +0200
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
Message-ID: <g40rcc$bd9$1@ger.gmane.org>

glyph at divmod.com schrieb:

> Another way to phrase this question is, "whose responsibility is it to 
> make Python 2.5 programs run on Python 2.6"?  Or, "what happens when the 
> core team finds out that a change they have made has broken some python 
> software 'in the wild'"?
> 
> Here are a couple of ways to answer these questions:
> 
>   1) It's the python core team's responsibility.  There should be Python 
> buildbots for lots of different projects and they should all be green 
> before the code can be considered [one of: allowed in trunk, alpha, 
> beta, RC].
>   2) It's the python core team's responsibility as long as the major 
> projects are using public APIs.  The code should not be considered 
> [alpha, beta, RC] if there are any known breakages in public APIs.
 >
>   3) It's only the python core team's responsibility to notify projects 
> of incompatible changes of which they are aware, which may occur in 
> public or private APIs.  Major projects with breakages will be given a 
> grace period before a [beta, RC, final] to release a forward-compatible 
> version.
>   4) The status quo: every new Python release can (and will, at least 
> speaking in terms of 2.6) break lots of popular software, with no 
> warning aside from the beta period.  There are no guarantees.

Python's backwards compatibility has never extended to non-public APIs.
That rules out 1). 2) would be nice, but to be honest, 2) and 3) are
unrealistic given the number of Python core developers and the tasks
already at hand. 4) is not acceptable either.

However, you seem to be overlooking that there's a 3.5) in there: it's
the Python core team's responsibility as long as the major projects are
using public APIs; but to reduce the workload every project should watch
its own buildbots and notify the core team using the issue tracker in
order to get the issue resolved.

At no time will a policy "the community buildbots must be green" be
useful: the tests that run on these buildbots are not under our control,
so if the tests test things we deem non-public we can't do anything
about it. (And we may have a hard time convincing project authors to
change the tests if we promised to make them run anyway.)

Georg


From g.brandl at gmx.net  Thu Jun 26 21:46:48 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 26 Jun 2008 21:46:48 +0200
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626163346.25821.1622910462.divmod.xquotient.10493@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<4863B900.1040808@gmail.com>
	<20080626163346.25821.1622910462.divmod.xquotient.10493@joule.divmod.com>
Message-ID: <g40rgv$bv4$1@ger.gmane.org>

glyph at divmod.com schrieb:
> On 03:42 pm, ncoghlan at gmail.com wrote:
>>glyph at divmod.com wrote:
> 
>>beta 1 has some trouble running *our* test suite - I'd be fairly 
>>surprised if the community buildbots were in significantly better 
>>shape.
> 
> That's another problem, yes :)
>>>The community buildbots have been in a broken state for months now[1].
> 
>>If the community buildbots aren't largely green by the time beta 2 
>>comes out, that's when I'll agree we have a problem - they should 
>>definitely be green by the time first release candidate comes out.
> 
> I have mixed feelings about this: on the one hand, if this were a matter 
> of public record, I would have known that this was too early to start 
> complaining, and we could have saved everyone a lot of time reading 
> these messages.  That would be good; I would prefer to just let everyone 
> work without bothering about process.

The problem is that not enough people care about the community buildbots,
and this isn't likely to improve since we have not enough manpower to do
it.

> On the other hand, it's much easier to deal with test failures as they 
> arise than in one giant chunk of work at the end of the development 
> process.  I spoke to a few core developers at PyCon who thought that the 
> buildbots should always be green and any change that broke them should 
> be reverted.  They may remain nameless unless they wish to raise their 
> hands :-) but that's definitely how I feel about it.

I think no core developer will not tell you that the (core) buildbots should be
green all the time. As for reverting changes that break, I'd support this
only for changes that break *all* of them. For example, I only use one
platform to develop on (and I guess it's the same for many others), having
the buildbots go red on another platform means I can try to fix the issue.

Georg


From hall.jeff at gmail.com  Thu Jun 26 21:48:10 2008
From: hall.jeff at gmail.com (Jeff Hall)
Date: Thu, 26 Jun 2008 15:48:10 -0400
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>
	<4863CE78.7010108@v.loewis.de>
	<20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com>
Message-ID: <1bc395c10806261248k42f49d43va4d09a7251ee523a@mail.gmail.com>

To me (and I'm an outsider not a direct developer), it's Python developers
responsibility to handle the Python problems and the Python build bots. The
community build bots are the responsibility of their authors. If JP is
handling the Twisted one then great. It's got a gatekeeper. If no one is
handling the Django build bot that's not the Python Dev Team's problem but
is a problem on the Django side and someone involved in that should be
tasked with monitoring.

Having said all that, I agree that the community bots ought to at least be
looked at. If I check in a patch in and I notice that a community bot went
from green to red then I probably should double check my code. The problem
is that, as you said, the community bots haven't been well maintained...
They've been red for awhile. So asking the developers to then go do the leg
work to find the original error, fix it and then back check everything
between then and now that might have ALSO caused a problem is alot of
effort.

It's not the Bank's responsibility to balance my check book. They give me
the tools to do it and then I have to check. A similar principle applies
here, I believe.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080626/7fe0dca2/attachment.htm>

From exarkun at divmod.com  Thu Jun 26 21:48:41 2008
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 26 Jun 2008 15:48:41 -0400
Subject: [Python-Dev] Community buildbots and Python release quality
 metrics
In-Reply-To: <g40rgv$bv4$1@ger.gmane.org>
Message-ID: <20080626194841.4714.1180228744.divmod.quotient.13515@ohm>

On Thu, 26 Jun 2008 21:46:48 +0200, Georg Brandl <g.brandl at gmx.net> wrote:
> [snip]
>
>As for reverting changes that break, I'd support this
>only for changes that break *all* of them. For example, I only use one
>platform to develop on (and I guess it's the same for many others), having
>the buildbots go red on another platform means I can try to fix the issue.

BuildBot has two ways to let you run your code on all builders before you
commit it to trunk.  You can force a build on a branch or you can try a
build with a patch.  I don't know if these options are enabled on Python's
buildmaster.  If they are, then if you want, you can use them to make sure
your code works on all platforms before you put it into trunk, where it may
cause problems for someone else.

Jean-Paul

From python at rcn.com  Thu Jun 26 21:52:25 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 26 Jun 2008 12:52:25 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
Message-ID: <6F8B0560F400452A934A174137F0F227@RaymondLaptop1>

From: "Guido van Rossum" <guido at python.org>
> Let's step back and discuss the API some more.
> 
> - Do we need all three?

I think so -- see the the reasons below.  Of course, my first 
choice was not on your list.  To me, the one obvious way to 
convert a number to a eval-able string in a different base is 
to use bin(), oct(), or hex().  But that appears to be off the 
table for reasons that I've read but don't make any sense to me.
It seems simple enough, extendable enough, and clean enough
for bin/oct/hex to use __index__ if present and __float__ if not.


> - If so, why not .tobase(N)? (Even if N is restricted to 2, 8 and 16.)

I don't think it's user-friendly to have the float-to-bin API
fail to parallel the int-to-bin API.  IMO, it should be done
the same way in both places.

I don't find it attractive in appearance.  Any use case I can
imagine involves multiple calls using the same base and I 
would likely end-up using functools.partial or somesuch
to factor-out the repeated use of the same variable.  In particular,
it's less usable with a series of numbers at the interactive prompt. 
That is one of the primary use cases since it allows you to see
exactly what is happening with float arithmetic:

>>> .6 + .7
1.2999999999999998
>>> bin(.6)
'0b10011001100110011001100110011001100110011001100110011 * 2.0 ** -53'
>>> bin(.7)
'0b1011001100110011001100110011001100110011001100110011 * 2.0 ** -52'
>>> bin(.6 + .7)
'0b101001100110011001100110011001100110011001100110011 * 2.0 ** -50'
>>> bin(1.3)
'0b10100110011001100110011001100110011001100110011001101 * 2.0 ** -52'

Or checking whether a number is exactly representable:

>>> bin(3.375)
'0b11011 * 2.0 ** -3'

Both of those bits of analysis become awkward with the tobase() method:

>>> (.6).tobase(2)
      ...



> - What should the output format be? I know you originally favored
> 0b10101.010101 etc. Now that it's not overloaded on the bin/oct/hex
> builtins, the constraint that it needs to be an eval() able expression
> may be dropped (unless you see a use case for that too).

The other guys convinced me that round tripping was important
and that there is a good use case for being able to read/write
precisely specified floats in a platform independent manner.
Also, my original idea didn't scale well without exponential
notation -- i.e.  bin(125E-100) would have a heckofa lot
of leading zeroes.   Terry and Mark also pointed-out that
the hex with exponential notation was the normal notation
used in papers on floating point arithmetic.  Lastly, once I
changed over to the new way, it dramatically simplified the
implementation.


Raymond

From martin at v.loewis.de  Thu Jun 26 21:55:40 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jun 2008 21:55:40 +0200
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>	<ca471dc20806242105j18fc8681u314771bff750b11a@mail.gmail.com>	<85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1>	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>	<4863AA71.5020901@gmail.com>	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>	<4863B400.4090401@gmail.com>	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
Message-ID: <4863F43C.2080904@v.loewis.de>

>> So as far as the feature design goes, I offer some suggestions: a new
>> module; or a new function in math; or a new method on float. Since
>> Raymond is the champion for the feature let him choose the API from
>> those alternatives.
> 
> I choose bin/hex/oct methods on floatobjects.
> Will work-up a patch.

I think the feature is misguided in the first place. Why do you want
a hex representation of floating point numbers? Can't you use
struct.pack for that? And, if bin/hex/oct are useful, why not base
6 (say)?

Regards,
Martin

From martin at v.loewis.de  Thu Jun 26 22:00:30 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jun 2008 22:00:30 +0200
Subject: [Python-Dev] Community buildbots and Python
	release	quality	metrics
In-Reply-To: <20080626193210.25821.328171947.divmod.xquotient.10633@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>	<20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>	<ca471dc20806261003y140f1b29sba2cdd11ef657835@mail.gmail.com>
	<20080626193210.25821.328171947.divmod.xquotient.10633@joule.divmod.com>
Message-ID: <4863F55E.4010200@v.loewis.de>

> I want python developers to pay attention to the community buildbots

I don't think that goal is realistic. Instead, somebody who has actual
interest in this matter should pay this attention, and then bring it up
on python-dev when something breaks.

Regards,
Martin

From tjreedy at udel.edu  Thu Jun 26 22:02:19 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 26 Jun 2008 16:02:19 -0400
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
Message-ID: <g40sk9$g0s$1@ger.gmane.org>



glyph at divmod.com wrote:

> to what extent should Python actually be compatible between releases?

As I understand things from years of observation, the following are fair 
game to changed in ways possibly backward-incompatible for specific 
code: bugs, detailed float behavior (which may be system-specific 
anyway), error messages, private interfaces, non-enforcement of 
documented rules, and defined implementation-dependent behavior.  But 
Guido has been somewhat conservative about this (least so for bug fixes, 
I think) and sometimes put off changes even when the fault lies with 
questionable usages.

One fly in the ointment is that bugs (a discrepancy between doc and 
code) may be fixed either by changing the doc or the code, according to 
which embodies the design intention.  So neither can be taken as gospel.


From steve at holdenweb.com  Thu Jun 26 22:04:27 2008
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 26 Jun 2008 16:04:27 -0400
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <g40rcc$bd9$1@ger.gmane.org>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<g40rcc$bd9$1@ger.gmane.org>
Message-ID: <4863F64B.9000609@holdenweb.com>

Georg Brandl wrote:
> glyph at divmod.com schrieb:
> 
>> Another way to phrase this question is, "whose responsibility is it to 
>> make Python 2.5 programs run on Python 2.6"?  Or, "what happens when 
>> the core team finds out that a change they have made has broken some 
>> python software 'in the wild'"?
>>
>> Here are a couple of ways to answer these questions:
>>
>>   1) It's the python core team's responsibility.  There should be 
>> Python buildbots for lots of different projects and they should all be 
>> green before the code can be considered [one of: allowed in trunk, 
>> alpha, beta, RC].
>>   2) It's the python core team's responsibility as long as the major 
>> projects are using public APIs.  The code should not be considered 
>> [alpha, beta, RC] if there are any known breakages in public APIs.
>  >
>>   3) It's only the python core team's responsibility to notify 
>> projects of incompatible changes of which they are aware, which may 
>> occur in public or private APIs.  Major projects with breakages will 
>> be given a grace period before a [beta, RC, final] to release a 
>> forward-compatible version.
>>   4) The status quo: every new Python release can (and will, at least 
>> speaking in terms of 2.6) break lots of popular software, with no 
>> warning aside from the beta period.  There are no guarantees.
> 
> Python's backwards compatibility has never extended to non-public APIs.
> That rules out 1). 2) would be nice, but to be honest, 2) and 3) are
> unrealistic given the number of Python core developers and the tasks
> already at hand. 4) is not acceptable either.
> 
> However, you seem to be overlooking that there's a 3.5) in there: it's
> the Python core team's responsibility as long as the major projects are
> using public APIs; but to reduce the workload every project should watch
> its own buildbots and notify the core team using the issue tracker in
> order to get the issue resolved.
> 
> At no time will a policy "the community buildbots must be green" be
> useful: the tests that run on these buildbots are not under our control,
> so if the tests test things we deem non-public we can't do anything
> about it. (And we may have a hard time convincing project authors to
> change the tests if we promised to make them run anyway.)
> 
If, however, the community buildbot goes red because of a regression or 
backwards incompatibility the developers should, when notified, at least 
undertake to verify that the breakage is intentional.

Unfortunately having a buildbot is like installing a firewall. Some 
people assume that the very presence of the buildbot is a guard against 
version incompatibilities, whereas in truth there is always the messy 
business of communication required to ensure the information gets to 
where it is useful.

The bottom line, of course, is that if project X can't be bothered to 
tell the core development team when changes break their build it's 
nobody's fault but their own. If the report it and the core developers 
ignore the reports, that's something that should be rectified.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Thu Jun 26 22:04:27 2008
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 26 Jun 2008 16:04:27 -0400
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <g40rcc$bd9$1@ger.gmane.org>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<g40rcc$bd9$1@ger.gmane.org>
Message-ID: <4863F64B.9000609@holdenweb.com>

Georg Brandl wrote:
> glyph at divmod.com schrieb:
> 
>> Another way to phrase this question is, "whose responsibility is it to 
>> make Python 2.5 programs run on Python 2.6"?  Or, "what happens when 
>> the core team finds out that a change they have made has broken some 
>> python software 'in the wild'"?
>>
>> Here are a couple of ways to answer these questions:
>>
>>   1) It's the python core team's responsibility.  There should be 
>> Python buildbots for lots of different projects and they should all be 
>> green before the code can be considered [one of: allowed in trunk, 
>> alpha, beta, RC].
>>   2) It's the python core team's responsibility as long as the major 
>> projects are using public APIs.  The code should not be considered 
>> [alpha, beta, RC] if there are any known breakages in public APIs.
>  >
>>   3) It's only the python core team's responsibility to notify 
>> projects of incompatible changes of which they are aware, which may 
>> occur in public or private APIs.  Major projects with breakages will 
>> be given a grace period before a [beta, RC, final] to release a 
>> forward-compatible version.
>>   4) The status quo: every new Python release can (and will, at least 
>> speaking in terms of 2.6) break lots of popular software, with no 
>> warning aside from the beta period.  There are no guarantees.
> 
> Python's backwards compatibility has never extended to non-public APIs.
> That rules out 1). 2) would be nice, but to be honest, 2) and 3) are
> unrealistic given the number of Python core developers and the tasks
> already at hand. 4) is not acceptable either.
> 
> However, you seem to be overlooking that there's a 3.5) in there: it's
> the Python core team's responsibility as long as the major projects are
> using public APIs; but to reduce the workload every project should watch
> its own buildbots and notify the core team using the issue tracker in
> order to get the issue resolved.
> 
> At no time will a policy "the community buildbots must be green" be
> useful: the tests that run on these buildbots are not under our control,
> so if the tests test things we deem non-public we can't do anything
> about it. (And we may have a hard time convincing project authors to
> change the tests if we promised to make them run anyway.)
> 
If, however, the community buildbot goes red because of a regression or 
backwards incompatibility the developers should, when notified, at least 
undertake to verify that the breakage is intentional.

Unfortunately having a buildbot is like installing a firewall. Some 
people assume that the very presence of the buildbot is a guard against 
version incompatibilities, whereas in truth there is always the messy 
business of communication required to ensure the information gets to 
where it is useful.

The bottom line, of course, is that if project X can't be bothered to 
tell the core development team when changes break their build it's 
nobody's fault but their own. If the report it and the core developers 
ignore the reports, that's something that should be rectified.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From martin at v.loewis.de  Thu Jun 26 22:05:24 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jun 2008 22:05:24 +0200
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626194841.4714.1180228744.divmod.quotient.13515@ohm>
References: <20080626194841.4714.1180228744.divmod.quotient.13515@ohm>
Message-ID: <4863F684.9050505@v.loewis.de>

> BuildBot has two ways to let you run your code on all builders before you
> commit it to trunk.  You can force a build on a branch or you can try a
> build with a patch.  I don't know if these options are enabled on Python's
> buildmaster.  If they are, then if you want, you can use them to make sure
> your code works on all platforms before you put it into trunk, where it may
> cause problems for someone else.

Even if that's possible, it adds a burden. As likely not all people
agree that such a procedure would be a good thing, it's necessary that
somebody establishes a policy. Guido has already indicated that he is
not interested in further policies; Barry (the canonical other person
to set policies) has indicated in the past that he would like such a
policy, but deems it unrealistic at this point. He gives other
procedural changes higher priority, so that realistically, such a policy
might not be established within the next two years.

Regards,
Martin

From glyph at divmod.com  Thu Jun 26 22:06:38 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 26 Jun 2008 20:06:38 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <g40rcc$bd9$1@ger.gmane.org>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<g40rcc$bd9$1@ger.gmane.org>
Message-ID: <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com>

On 07:44 pm, g.brandl at gmx.net wrote:
>At no time will a policy "the community buildbots must be green" be
>useful: the tests that run on these buildbots are not under our 
>control,
>so if the tests test things we deem non-public we can't do anything
>about it. (And we may have a hard time convincing project authors to
>change the tests if we promised to make them run anyway.)

That's not what I'm suggesting.

If there is a legitimate disagreement between Python developers and 
developers of a project about whether an API should continue to be 
supported, then clearly, the Python developers get to win.  Welcome to 
open source.

However, I believe that the core team is not paying attention to other 
projects breaking.  I'm not saying that you want to make breaking 
changes, or that bug reports are not dealt with, but the problem is that 
dealing with these problems after the fact makes it _much_ more painful. 
When you get to the release, and you have 30 bug reports due to other 
projects breaking, they get triaged, some get left in, and some features 
of lots of different projects are left broken.  And many projects do not 
bother to test with the beta.

If developers were simply presented with the results of their changes 
immediately ("you broke twisted, django doesn't import, zope segfaults 
and mercurial destroys your repository") then perhaps they would weigh 
the benefits of compatibility differently, and do the trivial, obvious 
fix immediately, rather than waiting 6 months and having to figure out 
what the heck change caused the bug.  I accept the fact that python core 
development is done differently than i.e. Java development, and some 
backward compatibility may simply be broken.

Case in point: changes to the warnings module being discussed in a 
separate thread break a significant number of Twisted's tests, because 
they remove functionality which is not public but is required to test 
code that uses the warnings module.  Would Brett have taken this into 
account if he knew about it immediately when revision 62303 was 
committed?  Maybe not, but now that the code is written and done, 
there's significantly less motivation to go back and fix it.  At the 
time it might have only been a few minutes' work to continue supporting 
the use-case which Twisted needs.  Brett wouldn't even have necessarily 
needed to do the work: it would have been easier to convince a Twisted 
developer to do the work it to keep the community buildbot green rather 
than to make it a bit less red.  Now, though, it's much easier to say 
"oh well, that's private, you lose".  I don't ascribe this to malice - 
it really *would* be much harder to fix it now, for us as well as for 
him.

From musiccomposition at gmail.com  Thu Jun 26 22:09:11 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 26 Jun 2008 15:09:11 -0500
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
	DeprecationWarning in stdlib)
In-Reply-To: <ca471dc20806260834v18a6fe25k5daa51c4551e7f85@mail.gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>
	<ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>
	<1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com>
	<4863A2C2.8000202@gmail.com>
	<ca471dc20806260819h4293b840v1149806cf81c376c@mail.gmail.com>
	<4863B678.5040205@gmail.com>
	<ca471dc20806260834v18a6fe25k5daa51c4551e7f85@mail.gmail.com>
Message-ID: <1afaf6160806261309o3d346901jfc38fbda223ee9cb@mail.gmail.com>

On Thu, Jun 26, 2008 at 10:34 AM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Jun 26, 2008 at 8:32 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I'm personally fine with that approach, but some of the new items in there
>> for 2.6 could probably use a bit of cleaning up to improve the APIs and/or
>> the way they work.
>
> So get crackin'!

I'll add to my list. :)

Anyway, if we are going to make test.support public for real why don't
we take Nick's suggestion to rename the test package to _test, but
move test.support to a toplevel position like testutils or such.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From dickinsm at gmail.com  Thu Jun 26 22:17:48 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 26 Jun 2008 21:17:48 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <4863F43C.2080904@v.loewis.de>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
Message-ID: <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>

On Thu, Jun 26, 2008 at 8:55 PM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

>
> I think the feature is misguided in the first place. Why do you want
> a hex representation of floating point numbers?


Answering for myself:  because it gives an exact representation of a
floating-point number in a fairly human-readable format.


> Can't you use struct.pack for that?


struct.pack would only show the bit layout, leaving the user to
manually extract the sign bit, exponent, and fraction, and then make
sense of the whole thing.  The proposed feature works at a higher
level of abstraction, directly describing the *value* of the float rather
than its bit layout.  In particular, this means it'll make sense
across platforms, regardless of variations in bit layout.

And, if bin/hex/oct are useful, why not base
> 6 (say)?
>

I'd say that bin and hex are special:  bin is natural because
floats are usually thought of, and stored as, binary numbers.
hex is special because it gives a compact way of representing
a float, and because there's already a history of using hex
floats in numerical analysis literature and in programming
languages (C99, Java, ...)

I have to admit that I can't see much use for octal floats.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080626/3f79e019/attachment-0001.htm>

From guido at python.org  Thu Jun 26 22:21:49 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 13:21:49 -0700
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <g40qos$85s$1@ger.gmane.org>
References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>
	<48621CC6.2030801@gmail.com>
	<20080625130857.GA13663@steerpike.home.puzzling.org>
	<bbaeab100806251018w42bd1b78qb1a98144ac453779@mail.gmail.com>
	<20080626054916.GA24366@steerpike.home.puzzling.org>
	<4863A007.60804@gmail.com>
	<ca471dc20806260802y6b8a6cf0t81201ca252d1c45f@mail.gmail.com>
	<g40qos$85s$1@ger.gmane.org>
Message-ID: <ca471dc20806261321m119b354dlf4e0117d3470c36f@mail.gmail.com>

On Thu, Jun 26, 2008 at 12:33 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Guido van Rossum schrieb:
>
>>>> Ok, then we're back to there being no supported way to write tests that
>>>> need to
>>>> intercept warnings.  Twisted has already suffered from this (JP reports
>>>> that
>>>> Twisted's assertWarns is broken in 2.6), and I doubt it's alone.
>>>>
>>>> So I guess I am filing a bug after all... :)
>>>
>>> Yeah - Brett's correct that everything under "test.test_support" should
>>> really be formally undocumented. It's mostly a place for code that
>>> reflects
>>> "things we do a lot in our unit tests and are tired of repeating" rather
>>> than "this is a good API that we want to support forever and encourage
>>> other
>>> people to use".
>>
>> I still have a big problem with this attitude. If it's valuable enough
>> to share between our tests, it should be properly written for reuse
>> and documented. A collection of hacks remains a collection of hacks.
>> We can do better.
>
> Since this is part of Benjamin's project, we will make sure that the
> test_support that emerges from it will be properly documented, stable
> and usable.

I'm assuming however that Benjamin's project will land in 2.7 or 3.1,
right? If he's going to be refactoring tests all summer there's no way
that can be merged into the trunk and the py3k branch right before the
final release.


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Thu Jun 26 22:23:19 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jun 2008 22:23:19 +0200
Subject: [Python-Dev] Community buildbots and Python
	release	quality	metrics
In-Reply-To: <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<g40rcc$bd9$1@ger.gmane.org>
	<20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com>
Message-ID: <4863FAB7.6040200@v.loewis.de>

> I don't ascribe this to malice -
> it really *would* be much harder to fix it now, for us as well as for him.

I think I disagree. It's easier to fix it now than it was to fix it back
then. Fixing it back then would have meant to constantly observe the
buildbots, and keep them running, so it would be a huge effort to
maintain the infrastructure just to find a the few changes that
unintentially broke something.

Looking at them now is a lot of effort, also. But this effort is
feasible, once the root cause is identified, the patch causing it might
just get backed out. Maintaining the community buildbots has proven
infeasible.

It's unfortunate that many package authors don't understand that not
all breakage is deliberate, and that their only chance to get undesired
breakage reverted is to report bugs NOW.

Regards,
Martin

From guido at python.org  Thu Jun 26 22:28:20 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 13:28:20 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
Message-ID: <ca471dc20806261328j79c7473en935f88d4d1e92f5@mail.gmail.com>

On Thu, Jun 26, 2008 at 1:17 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> I'd say that bin and hex are special:  bin is natural because
> floats are usually thought of, and stored as, binary numbers.
> hex is special because it gives a compact way of representing
> a float, and because there's already a history of using hex
> floats in numerical analysis literature and in programming
> languages (C99, Java, ...)

Can you show us what APIs and output formats C99 and Java support?
Maybe we can borrow something from there rather than reinventing the
wheel?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Thu Jun 26 22:30:51 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 26 Jun 2008 22:30:51 +0200
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>	<4863AA71.5020901@gmail.com>	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>	<4863B400.4090401@gmail.com>	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
Message-ID: <4863FC7B.6070903@v.loewis.de>

> 
>     I think the feature is misguided in the first place. Why do you want
>     a hex representation of floating point numbers? 
> 
> 
> Answering for myself:  because it gives an exact representation of a
> floating-point number in a fairly human-readable format.

Ok. But

py> binascii.hexlify(struct.pack("d", 3.14))
'1f85eb51b81e0940'

does that already, no? You won't know the precise value, but you won't
know that with hex support, either.

> struct.pack would only show the bit layout, leaving the user to
> manually extract the sign bit, exponent, and fraction, and then make
> sense of the whole thing.

I'd question that the user is able to make sense of a number when
mantissa and exponent is represented in hex.

> I'd say that bin and hex are special:  bin is natural because
> floats are usually thought of, and stored as, binary numbers.
> hex is special because it gives a compact way of representing
> a float, and because there's already a history of using hex
> floats in numerical analysis literature and in programming
> languages (C99, Java, ...)

Then I'd argue that the feature should be symmetric: If there is
support for printing floating point numbers as hex, there should
also be support for hex floating point literals.

Also, to follow C's tradition, it would be better if that was
*not* integrated into the hex function (or a hex method), but
if there was support for %a in string formatting.

Regards,
Martin

From python at rcn.com  Thu Jun 26 22:31:13 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 26 Jun 2008 13:31:13 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
References: <20080620041816.4D5E81E4002@bag.python.org>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
Message-ID: <F56E13631D5942E49259666870142392@RaymondLaptop1>

[Mark Dickinson]
> I have to admit that I can't see much use for octal floats.

Neither do I.  They look weird to me.


Raymond

From g.brandl at gmx.net  Thu Jun 26 22:36:12 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 26 Jun 2008 22:36:12 +0200
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<g40rcc$bd9$1@ger.gmane.org>
	<20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com>
Message-ID: <g40udi$m74$1@ger.gmane.org>

glyph at divmod.com schrieb:
> On 07:44 pm, g.brandl at gmx.net wrote:
>>At no time will a policy "the community buildbots must be green" be
>>useful: the tests that run on these buildbots are not under our 
>>control,
>>so if the tests test things we deem non-public we can't do anything
>>about it. (And we may have a hard time convincing project authors to
>>change the tests if we promised to make them run anyway.)
> 
> That's not what I'm suggesting.

Sure, but I wanted to say that nevertheless :)

> If there is a legitimate disagreement between Python developers and 
> developers of a project about whether an API should continue to be 
> supported, then clearly, the Python developers get to win.  Welcome to 
> open source.
> 
> However, I believe that the core team is not paying attention to other 
> projects breaking.

Quite true. It is their duty to bring the breakage to our attention, if
they want to see results.

> I'm not saying that you want to make breaking 
> changes, or that bug reports are not dealt with, but the problem is that 
> dealing with these problems after the fact makes it _much_ more painful. 
> When you get to the release, and you have 30 bug reports due to other 
> projects breaking, they get triaged, some get left in, and some features 
> of lots of different projects are left broken.  And many projects do not 
> bother to test with the beta.

I thought we're talking about the projects that *do* use the buildbots?

> If developers were simply presented with the results of their changes 
> immediately ("you broke twisted, django doesn't import, zope segfaults 
> and mercurial destroys your repository") then perhaps they would weigh 
> the benefits of compatibility differently, and do the trivial, obvious 
> fix immediately, rather than waiting 6 months and having to figure out 
> what the heck change caused the bug. 

This sounds very nice in theory, but it must be implemented in a way that
does not add a burden to the developer, as Martin said.

 > Case in point: changes to the warnings module being discussed in a
 > separate thread break a significant number of Twisted's tests, because
 > they remove functionality which is not public but is required to test
 > code that uses the warnings module.  Would Brett have taken this into
 > account if he knew about it immediately when revision 62303 was
 > committed?  Maybe not, but now that the code is written and done,
 > there's significantly less motivation to go back and fix it.  At the
 > time it might have only been a few minutes' work to continue supporting
 > the use-case which Twisted needs.  Brett wouldn't even have necessarily
 > needed to do the work: it would have been easier to convince a Twisted
 > developer to do the work it to keep the community buildbot green rather
 > than to make it a bit less red.  Now, though, it's much easier to say
 > "oh well, that's private, you lose".  I don't ascribe this to malice -
 > it really *would* be much harder to fix it now, for us as well as for
 > him.

I must say that I'm increasingly surprised by this discussion since it seems
that not only the Python core devs neglect the community buildbots.
Shouldn't the project authors have at least the same interest that their
code runs on the next major Python version? And while they don't necessarily
have more time to watch the buildbots than we have, the amount of what they
need to keep an eye on.

Georg


From musiccomposition at gmail.com  Thu Jun 26 22:38:22 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Thu, 26 Jun 2008 15:38:22 -0500
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <ca471dc20806261321m119b354dlf4e0117d3470c36f@mail.gmail.com>
References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm>
	<48621CC6.2030801@gmail.com>
	<20080625130857.GA13663@steerpike.home.puzzling.org>
	<bbaeab100806251018w42bd1b78qb1a98144ac453779@mail.gmail.com>
	<20080626054916.GA24366@steerpike.home.puzzling.org>
	<4863A007.60804@gmail.com>
	<ca471dc20806260802y6b8a6cf0t81201ca252d1c45f@mail.gmail.com>
	<g40qos$85s$1@ger.gmane.org>
	<ca471dc20806261321m119b354dlf4e0117d3470c36f@mail.gmail.com>
Message-ID: <1afaf6160806261338o769ace1ag6530632bf3e61b94@mail.gmail.com>

On Thu, Jun 26, 2008 at 3:21 PM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Jun 26, 2008 at 12:33 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>>
>> Since this is part of Benjamin's project, we will make sure that the
>> test_support that emerges from it will be properly documented, stable
>> and usable.
>
> I'm assuming however that Benjamin's project will land in 2.7 or 3.1,
> right? If he's going to be refactoring tests all summer there's no way
> that can be merged into the trunk and the py3k branch right before the
> final release.

Good timing! Georg and I were just talking about this. You're correct;
I've never imagined having it in before 2.6 or 3.0 final.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From g.brandl at gmx.net  Thu Jun 26 22:39:00 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 26 Jun 2008 22:39:00 +0200
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <g40sk9$g0s$1@ger.gmane.org>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<g40sk9$g0s$1@ger.gmane.org>
Message-ID: <g40uiq$m74$2@ger.gmane.org>

Terry Reedy schrieb:
> 
> glyph at divmod.com wrote:
> 
>> to what extent should Python actually be compatible between releases?
> 
> As I understand things from years of observation, the following are fair 
> game to changed in ways possibly backward-incompatible for specific 
> code: bugs, detailed float behavior (which may be system-specific 
> anyway), error messages, private interfaces, non-enforcement of 
> documented rules, and defined implementation-dependent behavior.  But 
> Guido has been somewhat conservative about this (least so for bug fixes, 
> I think) and sometimes put off changes even when the fault lies with 
> questionable usages.

Which is the source of many unresolved open issues in the tracker -- everyone
agrees that "it" is a bug and should be fixed, perhaps there's even a patch,
but people may be using it this way, so nothing is done forever.

(And these bugs usually are the kind of things we don't want to change in
3.0 either since they are subtle things.)

However, I don't have a solution here, so I won't complain about it anymore.

Georg


From guido at python.org  Thu Jun 26 22:41:50 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 13:41:50 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
Message-ID: <ca471dc20806261341t62e8575bk46ad03bca3ac3180@mail.gmail.com>

On Thu, Jun 26, 2008 at 12:52 PM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Guido van Rossum" <guido at python.org>
>>
>> Let's step back and discuss the API some more.
>>
>> - Do we need all three?
>
> I think so -- see the the reasons below.

Sounds like Mark Dickinson only cares about bin and hex.

> Of course, my first choice was not
> on your list.  To me, the one obvious way to convert a number to a eval-able
> string in a different base is to use bin(), oct(), or hex().  But that
> appears to be off the table for reasons that I've read but don't make any
> sense to me.
> It seems simple enough, extendable enough, and clean enough
> for bin/oct/hex to use __index__ if present and __float__ if not.

That's not extendable to types that aren't int or float though. And it
would accept Decimal instances which seems a really odd thing to do.

>> - If so, why not .tobase(N)? (Even if N is restricted to 2, 8 and 16.)
>
> I don't think it's user-friendly to have the float-to-bin API
> fail to parallel the int-to-bin API.  IMO, it should be done
> the same way in both places.

Consistency only goes so far. We have 0b, 0o and 0x notations for
integers, and the bin/oct/hex builtins are meant to invert those. We
don't have base-{2,8,16} literals for floats.

> I don't find it attractive in appearance.  Any use case I can
> imagine involves multiple calls using the same base and I would likely
> end-up using functools.partial or somesuch
> to factor-out the repeated use of the same variable.  In particular,
> it's less usable with a series of numbers at the interactive prompt. That is
> one of the primary use cases since it allows you to see
> exactly what is happening with float arithmetic:
>
>>>> .6 + .7
>
> 1.2999999999999998
>>>>
>>>> bin(.6)
>
> '0b10011001100110011001100110011001100110011001100110011 * 2.0 ** -53'
>>>>
>>>> bin(.7)
>
> '0b1011001100110011001100110011001100110011001100110011 * 2.0 ** -52'
>>>>
>>>> bin(.6 + .7)
>
> '0b101001100110011001100110011001100110011001100110011 * 2.0 ** -50'
>>>>
>>>> bin(1.3)
>
> '0b10100110011001100110011001100110011001100110011001101 * 2.0 ** -52'
>
> Or checking whether a number is exactly representable:
>
>>>> bin(3.375)
>
> '0b11011 * 2.0 ** -3'
>
> Both of those bits of analysis become awkward with the tobase() method:
>
>>>> (.6).tobase(2)

You don't need the parentheses around .6.

I think much fewer than 0.01% of Python users will ever need this.
It's a one-liner helper function if you prefer to say bin(x) instead
of x.bin().

>> - What should the output format be? I know you originally favored
>> 0b10101.010101 etc. Now that it's not overloaded on the bin/oct/hex
>> builtins, the constraint that it needs to be an eval() able expression
>> may be dropped (unless you see a use case for that too).
>
> The other guys convinced me that round tripping was important
> and that there is a good use case for being able to read/write
> precisely specified floats in a platform independent manner.

Can you summarize those reasons? Who are the users of that feature?
I'm still baffled why a feature whose only users are extreme experts
needs to have such a prominent treatment. Surely there are a lot more
Python users who call urlopen() or urlparse() all day long. Should
these be built-in functions then?

> Also, my original idea didn't scale well without exponential
> notation -- i.e.  bin(125E-100) would have a heckofa lot
> of leading zeroes.   Terry and Mark also pointed-out that
> the hex with exponential notation was the normal notation
> used in papers on floating point arithmetic.  Lastly, once I
> changed over to the new way, it dramatically simplified the
> implementation.

I agree that you need to have a notation using an exponent. If it
weren't for the roundtripping, I'd probably have preferred something
which simply showed me the bits of the IEEE floating point number
broken out into mantissa and exponent -- that seems more educational
to me than normalizing things so that the last bit is nonzero.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Thu Jun 26 22:55:54 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 26 Jun 2008 21:55:54 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <ca471dc20806261328j79c7473en935f88d4d1e92f5@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<ca471dc20806261328j79c7473en935f88d4d1e92f5@mail.gmail.com>
Message-ID: <5c6f2a5d0806261355j71041a0dvca7672605c66f74d@mail.gmail.com>

On Thu, Jun 26, 2008 at 9:28 PM, Guido van Rossum <guido at python.org> wrote:

> Can you show us what APIs and output formats C99 and Java support?
> Maybe we can borrow something from there rather than reinventing the
> wheel?
>

Java's toHexString method is documented at:

http://java.sun.com/javase/6/docs/api/java/lang/Double.html#toHexString(double)

It's disadvantage from Python's point of view is that some features are IEEE
754
specific (e.g. treatment of subnormals, which don't exist for most other
floating
point types).

C99s support for hex literals uses a similar format;  the standard is less
specific about the precise output format, but it's still of the form

0x1.<fraction>p<exponent>

Incidentally, the funny 'p' for the exponent instead of 'e' is apparently
there to avoid ambiguity in something like:

0x1e+3

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080626/5c5e70fc/attachment.htm>

From dickinsm at gmail.com  Thu Jun 26 22:46:05 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 26 Jun 2008 21:46:05 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <4863FC7B.6070903@v.loewis.de>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
Message-ID: <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>

On Thu, Jun 26, 2008 at 9:30 PM, "Martin v. L?wis" <martin at v.loewis.de>
wrote:

> > Answering for myself:  because it gives an exact representation of a
> > floating-point number in a fairly human-readable format.
>
> Ok. But
>
> py> binascii.hexlify(struct.pack("d", 3.14))
> '1f85eb51b81e0940'
>
> does that already, no? You won't know the precise value, but you won't
> know that with hex support, either.
>

The output from hex_float(3.14) would be something like:

'0x1.91eb851eb851fp+1'

The exponent is still usually given in decimal;  there's no need
for it to be hexadecimal for exactness.

I'd question that the user is able to make sense of a number when
> mantissa and exponent is represented in hex.
>

I think the above it still a bit easier to understand than
if one has to figure out where the sign/exponent and
exponent/fraction bit boundaries are, unbias the exponent,
and add the extra hidden '1' bit into the mantissa.  That's
a lot of mental work.



> Then I'd argue that the feature should be symmetric: If there is
> support for printing floating point numbers as hex, there should
> also be support for hex floating point literals.
>

I agree with this.  Or at least support for hex floating point
strings, if not literals.


>
> Also, to follow C's tradition, it would be better if that was
> *not* integrated into the hex function (or a hex method), but
> if there was support for %a in string formatting.
>

I'd be delighted with '%a' support.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080626/ef2c536a/attachment.htm>

From barry at python.org  Thu Jun 26 23:06:03 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Jun 2008 17:06:03 -0400
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <4863B900.1040808@gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<4863B900.1040808@gmail.com>
Message-ID: <3B30AF7E-42D9-4D90-AB99-5A675094135F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 26, 2008, at 11:42 AM, Nick Coghlan wrote:

> If the community buildbots aren't largely green by the time beta 2  
> comes out, that's when I'll agree we have a problem - they should  
> definitely be green by the time first release candidate comes out.

Just FTR,

I'll be looking at the Python buildbots and Roundup tracker to decide  
whether I think beta2 is in shape to be released or not.  I definitely  
won't be tracking the non-core buildbots, so if something troublesome  
comes up there, you will have to submit a bug report to the Python  
tracker.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGQEu3EjvBPtnXfVAQIp6QP/bOU+8vImlRkfKCXazYhi3yEXcNwCPpWI
e3AoiOlgKbOhSPraNTyUBjC+OjuqHQttyMJz8TTpexSwhpG/erDIqUjRbkW0Yaas
attvFZBYbJhF3qvHyLzmSp9U3oXY6VyWayL8ZdzIGcukEW7g8DgQNQyoGOE4kEeX
i6+4RIRf+7A=
=X1lh
-----END PGP SIGNATURE-----

From barry at python.org  Thu Jun 26 23:10:41 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Jun 2008 17:10:41 -0400
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
Message-ID: <9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 26, 2008, at 12:54 PM, glyph at divmod.com wrote:

> On 04:42 pm, schmir at gmail.com wrote:
>> On Thu, Jun 26, 2008 at 5:33 PM, Guido van Rossum  
>> <guido at python.org> wrote:
>>> an explanation about *why* Django cannot even be imported than a
>>> blanket complaint that this is a disgrace. So why is it?
>
>> and already discussed:
>> http://mail.python.org/pipermail/python-dev/2008-April/078421.html
>
> Following that trail of breadcrumbs, I ended up here:
>
>   http://bugs.python.org/issue2235
>
> with a message from some guy named "Barry Warsaw" (anyone know him?)  
> that says:
>
> """
> Guido, can you comment on Amaury's latest patch?  I'm going to bump  
> this
> back to critical so as not to hold up 2.6 alpha, but it should be  
> marked
> as a release blocker for the first beta.
> """
>
> I don't know if this "Barry" guy has the appropriate permissions on  
> the bugtracker to increase priorities, so I've taken the liberty of  
> upgrading it as a release blocker for the _second_ beta ... ;-).   
> So, at least there's been one productive consequence of this  
> discussion.

Glyph, you did the right thing.  I wish there was a way of marking a  
bug "not-release-blocker-for-now" and have it automatically update  
back to blocker at a certain time or after a certain event.

I think it's valid for this issue to block beta2.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGQF0XEjvBPtnXfVAQJwiQP/T9f33lNtAQ9/eooKEyVCDms7NXJE7mIf
QPOQtXuTZdsfUl51OxkxewVj6ERZasHPwIB2c13HYuHRrMrmrE9EYStdbmMxh0BI
7EhsO4SfDI3ZnYFJvAEMrTvgY8Vz4v817LhzWNZ7RWxq/yOHG8C/ZgubPJIa8mnd
EGWUVoLg77E=
=OJHh
-----END PGP SIGNATURE-----

From guido at python.org  Thu Jun 26 23:17:02 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 14:17:02 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>
	<4863CE78.7010108@v.loewis.de>
	<20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com>
Message-ID: <ca471dc20806261417t7c562083u4cae90b94f7e1381@mail.gmail.com>

On Thu, Jun 26, 2008 at 12:35 PM,  <glyph at divmod.com> wrote:
>
> On 05:14 pm, martin at v.loewis.de wrote:
>>>
>>> I don't know.  JP is already addressing the issues affecting Twisted in
>>> another thread (incompatible changes in the private-but-necessary-to-
>>> get-any-testing-done API of the warnings module).  But I really think
>>> that whoever made the change which broke it should be the one
>>> investigating it, not me.
>>
>> How could they have known that they broke it?
>
> Because the relevant community buildbot turned red with that revision of
> trunk.  Keep in mind I'm not talking about every piece of Python code in the
> universe; just the ones selected for the community buildbots.  It would be
> nice if there were a dozen or so projects on there, but paying attention to
> the two builders that do actually run right now would be a good start.

Sorry, this is an unacceptable burden on the core developers. The core
developers aren't going to look at the community buildbots (unless
voluntarily) and they aren't going to roll back changes just because
some community buildbot goes red.

In general someone outside the core developer group needs to bring the
issue to the core developers' attention and then a fix will be created
if appropriate. Rollbacks are generally reserved for accidental
checkins or checkins against the process rules (e.g. during a code
freeze). Heck, we don't even roll back if one of our own buildbots
goes red.

I'm fine with requesting that the core developers pay serious
attention to reports about 3rd party code being broken. The community
buildbots are a fine tool to find out about this. But any policy that
requires an automatic rollback because a buildbot (community or core)
goes red is unacceptable.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Thu Jun 26 23:20:49 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 26 Jun 2008 23:20:49 +0200
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
	<9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org>
Message-ID: <g41117$v1r$1@ger.gmane.org>

Barry Warsaw schrieb:

>> I don't know if this "Barry" guy has the appropriate permissions on  
>> the bugtracker to increase priorities, so I've taken the liberty of  
>> upgrading it as a release blocker for the _second_ beta ... ;-).   
>> So, at least there's been one productive consequence of this  
>> discussion.
> 
> Glyph, you did the right thing.  I wish there was a way of marking a  
> bug "not-release-blocker-for-now" and have it automatically update  
> back to blocker at a certain time or after a certain event.
> 
> I think it's valid for this issue to block beta2.

I just added a "deferred blocker" priority -- that implements one half
of your wish. Mass issue updating must be done by someone who knows
Roundup better than me, I'm afraid.

Georg


From guido at python.org  Thu Jun 26 23:24:52 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 14:24:52 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<g40rcc$bd9$1@ger.gmane.org>
	<20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com>
Message-ID: <ca471dc20806261424tf47a9bcya4a06fcf88aeb6f1@mail.gmail.com>

On Thu, Jun 26, 2008 at 1:06 PM,  <glyph at divmod.com> wrote:
> On 07:44 pm, g.brandl at gmx.net wrote:
>>
>> At no time will a policy "the community buildbots must be green" be
>> useful: the tests that run on these buildbots are not under our control,
>> so if the tests test things we deem non-public we can't do anything
>> about it. (And we may have a hard time convincing project authors to
>> change the tests if we promised to make them run anyway.)
>
> That's not what I'm suggesting.
>
> If there is a legitimate disagreement between Python developers and
> developers of a project about whether an API should continue to be
> supported, then clearly, the Python developers get to win.  Welcome to open
> source.
>
> However, I believe that the core team is not paying attention to other
> projects breaking.  I'm not saying that you want to make breaking changes,
> or that bug reports are not dealt with, but the problem is that dealing with
> these problems after the fact makes it _much_ more painful. When you get to
> the release, and you have 30 bug reports due to other projects breaking,
> they get triaged, some get left in, and some features of lots of different
> projects are left broken.  And many projects do not bother to test with the
> beta.

Well, sorry, that's life. We're not going to deal with breakage in 3rd
party code on a "drop all other work" basis.

You seem to have a different idea about how to do development than the
core developers. Tough luck. You can't tell us how to do our work.
We're doing the best we can, and we're happy to listen to suggestions
you're making, but the set of suggestions you're making in this thread
are an unacceptable burden, and it's not going to happen.

> If developers were simply presented with the results of their changes
> immediately ("you broke twisted, django doesn't import, zope segfaults and
> mercurial destroys your repository") then perhaps they would weigh the
> benefits of compatibility differently, and do the trivial, obvious fix
> immediately, rather than waiting 6 months and having to figure out what the
> heck change caused the bug.  I accept the fact that python core development
> is done differently than i.e. Java development, and some backward
> compatibility may simply be broken.
>
> Case in point: changes to the warnings module being discussed in a separate
> thread break a significant number of Twisted's tests, because they remove
> functionality which is not public but is required to test code that uses the
> warnings module.  Would Brett have taken this into account if he knew about
> it immediately when revision 62303 was committed?  Maybe not, but now that
> the code is written and done, there's significantly less motivation to go
> back and fix it.

I disagree. It's broken and should be fixed. Beta 1 just came out so
this is the perfect time to file a bug.

> At the time it might have only been a few minutes' work to
> continue supporting the use-case which Twisted needs.  Brett wouldn't even
> have necessarily needed to do the work: it would have been easier to
> convince a Twisted developer to do the work it to keep the community
> buildbot green rather than to make it a bit less red.

Why? That sounds like it's a problem with the psychology of the
Twisted developers.

> Now, though, it's
> much easier to say "oh well, that's private, you lose".  I don't ascribe
> this to malice - it really *would* be much harder to fix it now, for us as
> well as for him.

I don't believe this.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Thu Jun 26 23:26:42 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 26 Jun 2008 22:26:42 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806261355j71041a0dvca7672605c66f74d@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<ca471dc20806261328j79c7473en935f88d4d1e92f5@mail.gmail.com>
	<5c6f2a5d0806261355j71041a0dvca7672605c66f74d@mail.gmail.com>
Message-ID: <5c6f2a5d0806261426x71cd35ffm8a07574d8f09be00@mail.gmail.com>

On Thu, Jun 26, 2008 at 9:55 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
>
> It's disadvantage from Python's point of view is that some features are IEEE 754

Aargh!  I can't believe I wrote that.  Its.  Its.  Its.  Anyway;  some
more detail:

Both C99 and Java 1.5/1.6 support hex floating-point literals;  both
in exactly the
same format, as far as I can tell.  Here are the relevant productions
from the Java
grammar:

HexDigit: one of
        0 1 2 3 4 5 6 7 8 9 a b c d e f A B C D E F

HexNumeral:
        0 x HexDigits
        0 X HexDigits

HexDigits:
        HexDigit
        HexDigit HexDigits

HexadecimalFloatingPointLiteral:
        HexSignificand BinaryExponent FloatTypeSuffix_opt

HexSignificand:
        HexNumeral
        HexNumeral .
        0x HexDigits_opt . HexDigits
        0X HexDigits_opt . HexDigits

BinaryExponent:
        BinaryExponentIndicator SignedInteger

BinaryExponentIndicator:one of
        p P

Java's 'Double' class has a 'toHexString' method that
outputs a valid hex floating point string, and the Double()
constructor also accepts such strings.

C99 also appears to have full support for input/output
of hex floats; e.g. using strtod and printf('%a', ...).

Not sure how helpful this is.

Mark

From guido at python.org  Thu Jun 26 23:28:56 2008
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Jun 2008 14:28:56 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
Message-ID: <ca471dc20806261428mcdbbe8h4aae5f18eca69a5f@mail.gmail.com>

On Thu, Jun 26, 2008 at 1:46 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> I'd be delighted with '%a' support.

Remind me what %a does?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Thu Jun 26 23:30:43 2008
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 26 Jun 2008 14:30:43 -0700
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
References: <20080620041816.4D5E81E4002@bag.python.org><FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1><4863AA71.5020901@gmail.com><328D5F1E-E082-4002-9A65-38A31625C161@python.org><4863B400.4090401@gmail.com><ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com><DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1><4863F43C.2080904@v.loewis.de><5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com><4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
Message-ID: <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>

[MvL]
>> Then I'd argue that the feature should be symmetric: 
>> If there is support for printing floating point numbers 
>> as hex, there should also be support for hex floating 
>> point literals.

[Mark]
> I agree with this.  Or at least support for hex floating point
> strings, if not literals.

ISTM, that the currently proposed output format gives
us this benefit for free (no changes to the parser).  
The format is already close to the C99 notation 
but replaces the 'p' with '* 2.0 **' which I find to
be both readable and self-explanatory.


Raymond

From martin at v.loewis.de  Thu Jun 26 23:35:44 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 26 Jun 2008 23:35:44 +0200
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>	<4863AA71.5020901@gmail.com>	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>	<4863B400.4090401@gmail.com>	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>	<4863F43C.2080904@v.loewis.de>	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
Message-ID: <48640BB0.9080304@v.loewis.de>

> I think the above it still a bit easier to understand than
> if one has to figure out where the sign/exponent and
> exponent/fraction bit boundaries are, unbias the exponent,
> and add the extra hidden '1' bit into the mantissa.  That's
> a lot of mental work.

Sure. However, I'd argue that most people are unable to even
remotely guess the number in decimal when presented with the
number in hexadecimal - at a minimum, they'll fail to recognize
that the exponent is not to the power of 10, or when they
realize that, might guess that it is to the power of 16
(I find it fairly confusing, but consequential, that it is
to the power of 2, even when the digits are hex - and then,
the exponent is decimal :-).

So I'd like to dismiss any objective of "the output must be
human-understandable" as unrealistic. That an unambiguous
representation is desired, I can understand - but only if there
also is a way to enter the same representation elsewhere.

In addition, I fail to see the point in binary representation.
For unambiguous representation, it's sufficient to use hex.
I can't accept claims that people will be actually able to
understand what number is represented when given the bit string.
For educational purposes, decoding mantissa and biased exponent
directly out of the IEEE representation is better than having
binary output builtin.

>     Also, to follow C's tradition, it would be better if that was
>     *not* integrated into the hex function (or a hex method), but
>     if there was support for %a in string formatting.
> 
> 
> I'd be delighted with '%a' support.

I personally find that much less problematic than extending the
hex, and wouldn't object to a patch providing such formatting
(assuming it does the same thing that C99 printf does).

Regards,
Martin

From barry at python.org  Thu Jun 26 23:43:55 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Jun 2008 17:43:55 -0400
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <g41117$v1r$1@ger.gmane.org>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
	<9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org>
	<g41117$v1r$1@ger.gmane.org>
Message-ID: <F95DC65B-9148-4570-BC96-534D306EF088@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 26, 2008, at 5:20 PM, Georg Brandl wrote:

> Barry Warsaw schrieb:
>
>>> I don't know if this "Barry" guy has the appropriate permissions  
>>> on  the bugtracker to increase priorities, so I've taken the  
>>> liberty of  upgrading it as a release blocker for the _second_  
>>> beta ... ;-).   So, at least there's been one productive  
>>> consequence of this  discussion.
>> Glyph, you did the right thing.  I wish there was a way of marking  
>> a  bug "not-release-blocker-for-now" and have it automatically  
>> update  back to blocker at a certain time or after a certain event.
>> I think it's valid for this issue to block beta2.
>
> I just added a "deferred blocker" priority -- that implements one half
> of your wish. Mass issue updating must be done by someone who knows
> Roundup better than me, I'm afraid.

Cool, thanks!  I should have thought of that before.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGQNm3EjvBPtnXfVAQIy1wP/QPFwklmnay8VPPovHA6H4XA6tW+ziWPV
hbeyAZX/MMxEYv/98Z4nOQU7M4AczlXHZks80UvDdKLwPe2wTwfOIgmCTeb+vciI
20GpxrHpWQNHy/XKeawEBxyLHJZ4qNGIAFmHwoiUusM6erTeVb9kWa0czG31En6r
Z4MV0YKfu+A=
=Zxh4
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Thu Jun 26 23:47:31 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jun 2008 23:47:31 +0200
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <ca471dc20806261428mcdbbe8h4aae5f18eca69a5f@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<4863AA71.5020901@gmail.com>	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>	<4863B400.4090401@gmail.com>	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>	<4863F43C.2080904@v.loewis.de>	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>	<4863FC7B.6070903@v.loewis.de>	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<ca471dc20806261428mcdbbe8h4aae5f18eca69a5f@mail.gmail.com>
Message-ID: <48640E73.5070109@v.loewis.de>

Guido van Rossum wrote:
> On Thu, Jun 26, 2008 at 1:46 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
>> I'd be delighted with '%a' support.
> 
> Remind me what %a does?

It's a C99 feature. From the spec (7.19.6.1p8)

       a,A     A  double  argument  representing  a  floating-point
               number is converted  in  the  style  [-]0xh.hhhhp?d,
               where  there  is  one  hexadecimal  digit  (which is
               nonzero if the argument is  a  normalized  floating-
               point  number  and  is otherwise unspecified) before
               the decimal-point character235) and  the  number  of
               hexadecimal   digits   after  it  is  equal  to  the
               precision; if the precision is missing and FLT_RADIX
               is  a  power  of 2, then the precision is sufficient
               for an exact representation of  the  value;  if  the
               precision is missing and FLT_RADIX is not a power of
               2,   then   the   precision   is    sufficient    to
               distinguish236) values of type double,  except  that
               trailing  zeros  may be omitted; if the precision is
               zero and the # flag is not  specified,  no  decimal-
               point  character  appears.   The  letters abcdef are
               used for a conversion and the letters ABCDEF  for  A
               conversion.   The  A conversion specifier produces a
               number with X  and  P  instead  of  x  and  p.   The
               exponent  always  contains  at  least one digit, and
               only as many more digits as necessary  to  represent
               the  decimal  exponent  of 2.  If the value is zero,
               the exponent is zero.

               A double argument representing an infinity or NaN is
               converted  in  the  style  of  an  f or F conversion
               specifier.

Footnotes
     235) Binary implementations can choose the  hexadecimal  digit
          to  the  left  of  the  decimal-point  character  so that
          subsequent digits align to nibble (4-bit) boundaries.

     236) The  precision  p  is sufficient to distinguish values of
          the source type if 16p-1>bn where b is FLT_RADIX and n is
          the  number  of  base-b  digits in the significand of the
          source type.  A smaller p might suffice depending on  the
          implementation's  scheme for determining the digit to the
          left of the decimal-point character.

This is symmetric with C99's hexadecimal floating point literals:

 hexadecimal-floating-constant:
   hexadecimal-prefix hexadecimal-fractional-constant
                     binary-exponent-part floating-suffix-opt
   hexadecimal-prefix hexadecimal-digit-sequence
                     binary-exponent-part floating-suffix-opt

 hexadecimal-fractional-constant:
   hexadecimal-digit-sequence-opt . hexadecimal-digit-sequence
   hexadecimal-digit-sequence .

 binary-exponent-part:
   p sign-opt digit-sequence
   P sign-opt digit-sequence

 hexadecimal-digit-sequence:
   hexadecimal-digit
   hexadecimal-digit-sequence hexadecimal-digit

scanf and strtod support the same format.

Regards,
Martin



From dickinsm at gmail.com  Thu Jun 26 23:50:10 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 26 Jun 2008 22:50:10 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <ca471dc20806261428mcdbbe8h4aae5f18eca69a5f@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<ca471dc20806261428mcdbbe8h4aae5f18eca69a5f@mail.gmail.com>
Message-ID: <5c6f2a5d0806261450v711dcc52m6346b42d060707ba@mail.gmail.com>

On Thu, Jun 26, 2008 at 10:28 PM, Guido van Rossum <guido at python.org> wrote:
> Remind me what %a does?

From the C99 standard (section 7.19.6.1):

A double argument representing a ?oating-point number is converted in the
style [?]0xh.hhhhp?d, [...]

From dickinsm at gmail.com  Thu Jun 26 23:53:18 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 26 Jun 2008 22:53:18 +0100
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
In-Reply-To: <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
Message-ID: <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>

On Thu, Jun 26, 2008 at 10:30 PM, Raymond Hettinger <python at rcn.com> wrote:
> ISTM, that the currently proposed output format gives
> us this benefit for free (no changes to the parser).  The format is already
> close to the C99 notation but replaces the 'p' with '* 2.0 **' which I find
> to
> be both readable and self-explanatory.

There's one other major difference between the C99 notation and the
current patch:  the C99 notation includes a (hexa)decimal point.  The
advantages of this include:

 - the exponent gives a rough idea of the magnitude of the number, and
 - the exponent doesn't vary with changes to the least significant bits
   of the float.

The disadvantage is the loss of evalability.  (Is that a word?)

Mark

From martin at v.loewis.de  Thu Jun 26 23:54:17 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jun 2008 23:54:17 +0200
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <g41117$v1r$1@ger.gmane.org>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>	<9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org>
	<g41117$v1r$1@ger.gmane.org>
Message-ID: <48641009.4080105@v.loewis.de>

> I just added a "deferred blocker" priority -- that implements one half
> of your wish. Mass issue updating must be done by someone who knows
> Roundup better than me, I'm afraid.

I doubt true mass update will be necessary. If you remind me that I
should convert all "deferred blocker" issues to some other priority,
I'll do that manually in a matter of minutes (using a query to find
all issues with that priority).

Regards,
Martin

From g.brandl at gmx.net  Fri Jun 27 00:00:13 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 27 Jun 2008 00:00:13 +0200
Subject: [Python-Dev] [Python-checkins] r64424 - in
 python/trunk:Include/object.h Lib/test/test_sys.py
 Misc/NEWSObjects/intobject.c Objects/longobject.c
 Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806261450v711dcc52m6346b42d060707ba@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>	<4863B400.4090401@gmail.com>	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>	<4863F43C.2080904@v.loewis.de>	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>	<4863FC7B.6070903@v.loewis.de>	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>	<ca471dc20806261428mcdbbe8h4aae5f18eca69a5f@mail.gmail.com>
	<5c6f2a5d0806261450v711dcc52m6346b42d060707ba@mail.gmail.com>
Message-ID: <g413b3$7t2$1@ger.gmane.org>

Mark Dickinson schrieb:
> On Thu, Jun 26, 2008 at 10:28 PM, Guido van Rossum <guido at python.org> wrote:
>> Remind me what %a does?
> 
> From the C99 standard (section 7.19.6.1):
> 
> A double argument representing a ?oating-point number is converted in the
> style [?]0xh.hhhhp?d, [...]

Let me remind you that %a currently means "call ascii()" in 3.0.

Georg


From martin at v.loewis.de  Thu Jun 26 23:57:32 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jun 2008 23:57:32 +0200
Subject: [Python-Dev] [Python-checkins] r64424
 -	inpython/trunk:Include/object.h
 Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>	<4863B400.4090401@gmail.com>	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>	<4863F43C.2080904@v.loewis.de>	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>	<4863FC7B.6070903@v.loewis.de>	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
	<5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>
Message-ID: <486410CC.2030906@v.loewis.de>

> The disadvantage is the loss of evalability.  (Is that a word?)

Until the parser has support for it, having a float class method,
or even the float callable itself for conversion seems reasonable.

If repr() didn't produce it, eval() doesn't need to understand it.

Regards,
Martin

From tjreedy at udel.edu  Fri Jun 27 00:00:23 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 26 Jun 2008 18:00:23 -0400
Subject: [Python-Dev] [Python-checkins] r64424 - in
 python/trunk:Include/object.h Lib/test/test_sys.py
 Misc/NEWSObjects/intobject.c Objects/longobject.c
 Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>	<4863AA71.5020901@gmail.com>	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>	<4863B400.4090401@gmail.com>	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
Message-ID: <g413hm$8t7$1@ger.gmane.org>



Raymond Hettinger wrote:
> From: "Guido van Rossum" <guido at python.org>
>> Let's step back and discuss the API some more.
>>
>> - Do we need all three?
> 
> I think so -- see the the reasons below.

I would prefer 1, see below.

>  Of course, my first choice was 
> not on your list.  To me, the one obvious way to convert a number to a 
> eval-able string in a different base is to use bin(), oct(), or hex().  
> But that appears to be off the table for reasons that I've read but 
> don't make any sense to me.

Let me try.  I am one of those who prefer smaller to bigger for the core 
  language to make it easier to learn and teach.  But, to me, there 
deeper consideration that applies here.  A Python interpreter, human or 
mechanical, must do exact integer arithmetic.  But a Python interpreter 
does not have to convert float literals to fixed size binary and does 
*not* have to do float arithmetic with binary presentations that are 
usually approximations.  (Indeed, human interpreters do neither, which 
is why they are often surprised at CPython's float output, and which is 
why this function will be useful.)  If built-in functions are part of 
the language definition, as Guido just clarified, their definition and 
justification should not depend on the float implementation.

> It seems simple enough, extendable enough, and clean enough
> for bin/oct/hex to use __index__ if present and __float__ if not.

To me, a binary representation, in whatever base, of a Decimal is 
senseless.  The point of this issue is to reveal the exact binary bit 
pattern of float instances.

>> - If so, why not .tobase(N)? (Even if N is restricted to 2, 8 and 16.)
> 
> I don't think it's user-friendly to have the float-to-bin API
> fail to parallel the int-to-bin API.  IMO, it should be done
> the same way in both places.

I would like to turn this around.  I think that 3 nearly identical 
built-ins is 2 too many. I am going to propose on the Py3 list that bin, 
oct, and hex be condensed to one function, bin(integer, base=2,8,or16), 
for 3.1 if not 3.0.  Base 8 and 16 are, to me, compressed binary.

Three methods is definitely too many for a somewhat subsidiary function. 
  So, I would like to see float.bin([base=2])

> I don't find it attractive in appearance.  Any use case I can
> imagine involves multiple calls using the same base and I would likely 
> end-up using functools.partial or somesuch
> to factor-out the repeated use of the same variable. 

Make the base that naive users want to see the default.  I believe this 
to be 2.  Numerical analysts who want base 16 can deal with partial if 
they really have scattered calls (as opposes to a few within loops) and 
cannot deal with typing '16' over and over.


>>>> bin(.6)
> '0b10011001100110011001100110011001100110011001100110011 * 2.0**-53'
...
> Both of those bits of analysis become awkward with the tobase() method:
>>>> (.6).tobase(2)

Eliminate the unneeded parentheses and default value, and this is
 >>> .6.bin()
which is just one extra char.

>> - What should the output format be? I know you originally favored
>> 0b10101.010101 etc. Now that it's not overloaded on the bin/oct/hex
>> builtins, the constraint that it needs to be an eval() able expression
>> may be dropped (unless you see a use case for that too).
> 
> The other guys convinced me that round tripping was important
> and that there is a good use case for being able to read/write
> precisely specified floats in a platform independent manner.

Definitely.  The paper I referenced in the issue discussion,
http://bugs.python.org/issue3008   mentioned a few times here, is
http://hal.archives-ouvertes.fr/docs/00/28/14/29/PDF/floating-point-article.pdf

> Also, my original idea didn't scale well without exponential
> notation -- i.e.  bin(125E-100) would have a heckofa lot
> of leading zeroes.   Terry and Mark also pointed-out that
> the hex with exponential notation was the normal notation
> used in papers on floating point arithmetic.  Lastly, once I
> changed over to the new way, it dramatically simplified the
> implementation.

I originally thought I preferred the 'hexponential' notation that uses P 
for power instead of E for exponential.  But with multiple bases, the 
redundancy of repeating the bases is ok, and being able to eval() 
without changing the parser is a plus.  But I would prefer losing the 
spaces around the ** operator.

Terry Jan Reedy


From dickinsm at gmail.com  Fri Jun 27 00:01:47 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 26 Jun 2008 23:01:47 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <g413b3$7t2$1@ger.gmane.org>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<ca471dc20806261428mcdbbe8h4aae5f18eca69a5f@mail.gmail.com>
	<5c6f2a5d0806261450v711dcc52m6346b42d060707ba@mail.gmail.com>
	<g413b3$7t2$1@ger.gmane.org>
Message-ID: <5c6f2a5d0806261501j4ff3c5eeqcf5d4197ebe26a44@mail.gmail.com>

On Thu, Jun 26, 2008 at 11:00 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Let me remind you that %a currently means "call ascii()" in 3.0.

Oh well.  That's out then.  I'll rephrase to "I'd be delighted with something
similar in spirit to '%a' support."  :-)

Mark

From brett at python.org  Fri Jun 27 00:11:25 2008
From: brett at python.org (Brett Cannon)
Date: Thu, 26 Jun 2008 15:11:25 -0700
Subject: [Python-Dev] Py3k DeprecationWarning in stdlib
In-Reply-To: <20080626160118.4714.38891548.divmod.quotient.13435@ohm>
References: <4863BB32.3040101@gmail.com>
	<20080626160118.4714.38891548.divmod.quotient.13435@ohm>
Message-ID: <bbaeab100806261511n4def8fe2yc0b05e32edbe629e@mail.gmail.com>

On Thu, Jun 26, 2008 at 9:01 AM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> On Fri, 27 Jun 2008 01:52:18 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> Jean-Paul Calderone wrote:
>>>
>>> I don't particularly care about the details, I just want some public
>>> API for this.  Making warn_explicit public seems better to me, since
>>> it was already there in previous versions of Python, and it lets you
>>> completely ignore both the filters list and the global registry, but
>>> if others would rather make the filters and global registry a part of
>>> the public API, that's fine by me as well.
>>
>> Why do you say warn_explicit isn't public? It's in both the 2.5 and 2.6
>> API docs for the warnings module.
>
> Brett told me it was private (on this list several weeks or a month or so
> ago).  It's also no longer called in 2.6 by the C implementation of the
> warning system.
>

I don't remember saying it is private, just that you can't replace the
function in the module and expect everything to continue to work
properly like you can with showwarning().

-Brett

From dickinsm at gmail.com  Fri Jun 27 00:11:35 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 26 Jun 2008 23:11:35 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <g413hm$8t7$1@ger.gmane.org>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
	<g413hm$8t7$1@ger.gmane.org>
Message-ID: <5c6f2a5d0806261511x11c1398cp4d3067950139430e@mail.gmail.com>

On Thu, Jun 26, 2008 at 11:00 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> Definitely.  The paper I referenced in the issue discussion,
> http://bugs.python.org/issue3008   mentioned a few times here, is
> http://hal.archives-ouvertes.fr/docs/00/28/14/29/PDF/floating-point-article.pdf

Perhaps it's worth reproducing the most relevant paragraph of that paper
(the end of section 2.1) here:

"""Conversion to and from decimal representation is delicate; special
care must be taken in order not to introduce inaccuracies or discrepan-
cies. [Steele and White, 1990, Clinger, 1990]. Because of this, C99 introduces
hexadecimal floating-point literals in source code. [ISO, 1999, ?6.4.4.2] Their
syntax is as follows: 0xmmmmmm.mmmmp?ee where mmmmmm.mmmm is a mantissa in
hexadecimal, possibly containing a point, and ee is an exponent,
written in deci-
mal, possibly preceded by a sign. They are interpreted as [mmmmmm.mmmm]16?2ee .
Hexadecimal floating-point representations are especially important when val-
ues must be represented exactly, for reproducible results ? for instance, for
testing "borderline cases" in algorithms. For this reason, we shall use them in
this paper wherever it is important to specify exact values. See also
Section 4.4
for more information on inputting and outputting floating-point values."""

From schmir at gmail.com  Fri Jun 27 00:22:41 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Fri, 27 Jun 2008 00:22:41 +0200
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <ca471dc20806261145m164374bnb4dcdf770c2be0fa@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
	<932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com>
	<ca471dc20806261055g5219d34fi23b84f9462af8962@mail.gmail.com>
	<932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com>
	<ca471dc20806261145m164374bnb4dcdf770c2be0fa@mail.gmail.com>
Message-ID: <932f8baf0806261522m7b5eb75aiac986ea029cdd6ff@mail.gmail.com>

On Thu, Jun 26, 2008 at 8:45 PM, Guido van Rossum <guido at python.org> wrote:
>
> So you're saying py.test needs to be fixed? Fine with me, but then I
> don't understand why you bothered bringing it up here instead of
> fixing it (or reporting to the py.test maintainers, I don't know if
> you're one or not).
>

no, I'm not. Just wanted to point out, that this ticket/patch does not
solve the django issue.
(this is what I first thought it would do).

- Ralf

From eric+python-dev at trueblade.com  Fri Jun 27 00:37:01 2008
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Thu, 26 Jun 2008 18:37:01 -0400
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806261501j4ff3c5eeqcf5d4197ebe26a44@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>	<4863F43C.2080904@v.loewis.de>	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>	<4863FC7B.6070903@v.loewis.de>	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>	<ca471dc20806261428mcdbbe8h4aae5f18eca69a5f@mail.gmail.com>	<5c6f2a5d0806261450v711dcc52m6346b42d060707ba@mail.gmail.com>	<g413b3$7t2$1@ger.gmane.org>
	<5c6f2a5d0806261501j4ff3c5eeqcf5d4197ebe26a44@mail.gmail.com>
Message-ID: <48641A0D.6020203@trueblade.com>

Mark Dickinson wrote:
> On Thu, Jun 26, 2008 at 11:00 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Let me remind you that %a currently means "call ascii()" in 3.0.
> 
> Oh well.  That's out then.  I'll rephrase to "I'd be delighted with something
> similar in spirit to '%a' support."  :-)

It could be added to str.format().  Well, actually float.__format__(). 
Not that I'm advocating it, but it's a place it would fit, and since 
it's restricted to the float format specifier, it would at least be well 
contained.  And other types are free to implement it, or not.


From barry at python.org  Fri Jun 27 00:34:20 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Jun 2008 18:34:20 -0400
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <48641009.4080105@v.loewis.de>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>	<9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org>
	<g41117$v1r$1@ger.gmane.org> <48641009.4080105@v.loewis.de>
Message-ID: <E784723C-D8D6-498B-9425-3F0F4FBDBFAD@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jun 26, 2008, at 5:54 PM, Martin v. L?wis wrote:

>> I just added a "deferred blocker" priority -- that implements one  
>> half
>> of your wish. Mass issue updating must be done by someone who knows
>> Roundup better than me, I'm afraid.
>
> I doubt true mass update will be necessary. If you remind me that I
> should convert all "deferred blocker" issues to some other priority,
> I'll do that manually in a matter of minutes (using a query to find
> all issues with that priority).

Martin, I think that will work great.  The idea being that after each  
release, we always bump deferred blockers back to release blocker.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSGQZbHEjvBPtnXfVAQIwWQQAqWoIoqV0S0O8ZpkXpWd3LO4Fo7MBfjRT
LL5QgIPBWdQdZ4RR68LSfdAhiHGnZCZorpNksGaeIxP55qgS1z31fy3JF39UUbVR
3ojymtF6Is0qCB5lmkJIx9Na/2RUEOUWTQoQP3dG851oRRuSVTXb4uRaBlNlQErY
oN9JxG5xQR0=
=z9nN
-----END PGP SIGNATURE-----

From steve at pearwood.info  Fri Jun 27 01:34:57 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 27 Jun 2008 09:34:57 +1000
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
In-Reply-To: <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
Message-ID: <200806270934.57783.steve@pearwood.info>

On Fri, 27 Jun 2008 07:30:43 am Raymond Hettinger wrote:
> The format is already close to the C99 notation
> but replaces the 'p' with '* 2.0 **' which I find to
> be both readable and self-explanatory.

Since we're talking about what's "readable and self-explanatory", I find 
that jarring, unexpected, unintuitive, and mathematically bizarre (even 
if it is the convention in some areas). It's like writing '123 * A.0 
** -2' for 1.23.

And putting spaces around the operators is ugly.

I'd like to mention that what bin() et al is actually doing is not so 
much returning a binary number string but returning a hybrid 
binary/decimal arithmetic expression. So bin() returns a binary number 
string for int arguments, and an expression for float arguments: these 
are conceptually different kinds of things, even if they're both 
strings.

Frankly, I'd be much happier if the API (whatever it is) returned a 
tuple of (binary string, base int, exponent int), and let users write 
their own helper function to format it any way they like. Or failing 
that, the p notation used by Java and C99. (And yes, mixing decimal 
exponents with binary mantissas upsets me too, but somehow it's less 
upsetting.)


-- 
Steven

From greg.ewing at canterbury.ac.nz  Fri Jun 27 02:11:00 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Jun 2008 12:11:00 +1200
Subject: [Python-Dev] C API for gc.enable() and gc.disable()
In-Reply-To: <1bc395c10806251408w700f0791y3474fcb1f639ee6e@mail.gmail.com>
References: <acd65fa20805312111mc03a857h58c0b2edab8c9c5f@mail.gmail.com>
	<1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de>
	<1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com>
	<485F1ADA.6090107@v.loewis.de>
	<loom.20080623T102812-837@post.gmane.org>
	<486010CD.2000904@v.loewis.de>
	<loom.20080624T093235-422@post.gmane.org>
	<1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com>
	<4862B0C4.2080901@v.loewis.de>
	<1bc395c10806251408w700f0791y3474fcb1f639ee6e@mail.gmail.com>
Message-ID: <48643014.5020008@canterbury.ac.nz>

Jeff Hall wrote:
> I mistakenly thought that was 
> because they were assumed to be small. It sounds like they're ignored 
> because they're automatically collected and so they SHOULD be ignored 
> for object garbage collection.

Strings aren't tracked by the cyclic garbage collector
because they don't contain object references and therefore
can't form part of a cycle.

However, unless I'm mistaken, allocations and deallocations
of them are still counted for the purpose of determining
when to perform a cyclic GC pass. So if you allocate lots
of strings and they aren't getting deallocated, a cyclic
GC pass will eventually occur, in case the strings are
being referenced from a cycle that can be cleaned up.

I don't know whether/how re uses string objects internally
while it's matching, so I can't say what its garbage
collection characteristics might be when matching against
a huge string.

The behaviour you observed might have been due to the
nature of the re being matched -- some res can have
quadratic or exponential behaviour all by themselves.

-- 
Greg

From glyph at divmod.com  Fri Jun 27 02:33:56 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 27 Jun 2008 00:33:56 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <ca471dc20806261417t7c562083u4cae90b94f7e1381@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>
	<4863CE78.7010108@v.loewis.de>
	<20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com>
	<ca471dc20806261417t7c562083u4cae90b94f7e1381@mail.gmail.com>
Message-ID: <20080627003356.25821.685560775.divmod.xquotient.10844@joule.divmod.com>


On 09:17 pm, guido at python.org wrote:
>On Thu, Jun 26, 2008 at 12:35 PM,  <glyph at divmod.com> wrote:

>>Because the relevant community buildbot turned red with that revision 
>>of
>>trunk.  Keep in mind I'm not talking about every piece of Python code 
>>in the
>>universe; just the ones selected for the community buildbots.  It 
>>would be
>>nice if there were a dozen or so projects on there, but paying 
>>attention to
>>the two builders that do actually run right now would be a good start.
>
>Sorry, this is an unacceptable burden on the core developers. The core
>developers aren't going to look at the community buildbots (unless
>voluntarily) and they aren't going to roll back changes just because
>some community buildbot goes red.

OK, fair enough.  Taking a step back, I was pushing this really hard 
because to *me*, it seems like dealing with the influx of bug reports 
after the fact is an unreasonable amount of additional work, whereas 
immediate reverts are just an occasional, temporary inconvenience. 
Based on my experience, "We'll deal with the reports as they come in" 
sounds like "no".
>In general someone outside the core developer group needs to bring the
>issue to the core developers' attention and then a fix will be created
>if appropriate. Rollbacks are generally reserved for accidental
>checkins or checkins against the process rules (e.g. during a code
>freeze). Heck, we don't even roll back if one of our own buildbots
>goes red.

I think since the last time we had a similar conversation, other Twisted 
developers have been fairly diligent in reporting bugs.  (In the time 
they've been reporting Python bugs, I've mostly been planning a wedding. 
Others who have survived it tell me that eventually, this process 
ends... and then I should be participating more directly.)  I'll try to 
step up that feeback loop and get other projects involved.  I hope I am 
wrong about generating an unreasonable amount of work.
>I'm fine with requesting that the core developers pay serious
>attention to reports about 3rd party code being broken. The community
>buildbots are a fine tool to find out about this. But any policy that
>requires an automatic rollback because a buildbot (community or core)
>goes red is unacceptable.

Thanks.  I appreciate it; an increased emphasis on 3rd party code being 
broken is really what I was looking for.  This is fine with me.  I mean, 
whatever your decision is, I'm going to have to live with it :), but in 
either case, we have to be looking for bugs and helping to investigate 
them.  On my end of things I guess it's not going to make much 
difference.

From rhamph at gmail.com  Fri Jun 27 02:53:15 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 26 Jun 2008 18:53:15 -0600
Subject: [Python-Dev] GC Proposal
Message-ID: <aac2c7cb0806261753xe16ca0dt7624c9a2c458edd@mail.gmail.com>

We need two counters: one is the total number of traceable objects
(those we would inspect if we did a full collection) and a number of
"pending" trace operations.  Every time an object is moved into the
last generation, we increase "pending" by two - once for itself and
once for an older object.  Once pending equals the total number of
traceable objects we do a full collection (and reset "pending" to 0).

This behaves well under both extremes.  First, if allocating in a
burst, it waits until the total number of objects has doubled before
doing a collection, which works out to about 2 passes per object.

Second, a long running program may allocate an object, get it into the
oldest generation (adding to "pending"), but then delete it through
normal means.  The pending trace is effectively borrowed by the
remaining older objects, which ensures they will get scanned again
eventually.  However, we already payed for it when the young object
was allocated, so the net cost is still a constant factor.

Various refinements are possible, such as only adding 1.5 to
"pending", or taking off some of the 2 if a young object is deleted
without getting traced.  We could also tweak when in the object
life-cycle "pending" is increased.

The first two generations would remain as they are today.

-- 
Adam Olsen, aka Rhamphoryncus

From greg.ewing at canterbury.ac.nz  Fri Jun 27 03:11:27 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Jun 2008 13:11:27 +1200
Subject: [Python-Dev] [Python-checkins] r64424 - in
 python/trunk:Include/object.h Lib/test/test_sys.py
 Misc/NEWSObjects/intobject.c Objects/longobject.c
 Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806251349l4f83908bk7080359273f96881@mail.gmail.com>
	<2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1>
	<ca471dc20806260652o4f0d47b2hca753f3a03b96e9@mail.gmail.com>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
Message-ID: <48643E3F.3010500@canterbury.ac.nz>

Raymond Hettinger wrote:
> To me, the one obvious way to convert a number to a 
> eval-able string in a different base is to use bin(), oct(), or hex().  

What use cases are there for an eval-able representation
of a float in those bases, as opposed to a human-readable
one?

-- 
Greg

From glyph at divmod.com  Fri Jun 27 03:13:15 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 27 Jun 2008 01:13:15 -0000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <ca471dc20806261424tf47a9bcya4a06fcf88aeb6f1@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<g40rcc$bd9$1@ger.gmane.org>
	<20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com>
	<ca471dc20806261424tf47a9bcya4a06fcf88aeb6f1@mail.gmail.com>
Message-ID: <20080627011315.25821.242932439.divmod.xquotient.10918@joule.divmod.com>

On 26 Jun, 09:24 pm, guido at python.org wrote:
>On Thu, Jun 26, 2008 at 1:06 PM,  <glyph at divmod.com> wrote:
>>On 07:44 pm, g.brandl at gmx.net wrote:

>Well, sorry, that's life. We're not going to deal with breakage in 3rd
>party code on a "drop all other work" basis.

For the record, "automatic revert" does not mean "drop all other work". 
The changeset's still there.  It's still in the revision history.  It 
can easily be applied to anybody's working copy.  It can easily be put 
into a branch.  It can be automatically re-merged later.  I do all of 
these things all the time, and I was *not* intending to suggest that any 
3rd-party breakage should cause a code freeze or anything even remotely 
like that.
>>Case in point: changes to the warnings module

>I disagree. It's broken and should be fixed. Beta 1 just came out so
>this is the perfect time to file a bug.

I'll go back over the recent conversation and work out the specifics of 
the bug (if JP doesn't, or hasn't already beaten me to it).
>>(...) it would have been easier to
>>convince a Twisted developer to do the work it to keep the community
>>buildbot green rather than to make it a bit less red.
>
>Why? That sounds like it's a problem with the psychology of the
>Twisted developers.

I hardly think it's unique to us.  TDD test runners typically only know 
2 colors and 2 states: "passed" and "fails".  Once you're in the "fail" 
state, you tend to accumulate more failures; there's a much bigger bang 
for your buck.  Better tools with nicer interfaces would let you easily 
mark individual tests as "usually intermittent" or something and make it 
a "slightly dirty green" or something, but we don't have those.  The 
move from "red" to "green" is much more psychologically significant to 
just about anyone I know than the move from "red but 14 failures" to 
"red but 12 failures".

Despite the idea's origins in a now-highly-controversial book on 
criminology, this is often referred to in XP discussion circles (wikis 
and the like) as the "fix broken windows" collaboration pattern.  I 
notice this in lots of other areas of my life; a little pile of papers 
tends to become a big pile of papers, the first dent in a car makes the 
driver a little less careful, and so on.

From bender at physik.rwth-aachen.de  Fri Jun 27 12:14:03 2008
From: bender at physik.rwth-aachen.de (Walter Bender)
Date: Fri, 27 Jun 2008 12:14:03 +0200
Subject: [Python-Dev] deepcopy
Message-ID: <4864BD6B.1030203@physik.rwth-aachen.de>

<mailto:python-dev at python.org>Hello togehter,

I get an error while doing a deep copy of an "event". An event is an 
object, with has a lot of __dict__["attr"]  = Event("SAMPLE") OTHER 
Events added to the object (no self refence) and store additional 
__dict__["attr2"] =  2 informations.
 
deep copy is failing with no proper error message.


    nevent = copy.deepcopy(event);
  File 
"/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", 
line 174, in deepcopy
    y = copier(x, memo)
  File 
"/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", 
line 305, in _deepcopy_inst
    state = deepcopy(state, memo)
  File 
"/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", 
line 174, in deepcopy
    y = copier(x, memo)
  File 
"/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", 
line 268, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File 
"/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", 
line 174, in deepcopy
    y = copier(x, memo)
  File 
"/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", 
line 241, in _deepcopy_list
    y.append(deepcopy(a, memo))
  File 
"/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", 
line 174, in deepcopy
    y = copier(x, memo)
  File 
"/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", 
line 241, in _deepcopy_list
    y.append(deepcopy(a, memo))
  File 
"/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", 
line 185, in deepcopy
    y = copier(x, memo)
TypeError: __deepcopy__() takes no arguments (1 given

cheers,
Walter

-- 
Walter Bender
bender at physik.rwth-aachen.de
RWTH Aachen
III. Physikalisches Institut A
Tel. +49-241-8027286


From ncoghlan at gmail.com  Fri Jun 27 13:00:35 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 21:00:35 +1000
Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k
 DeprecationWarning in stdlib)
In-Reply-To: <1afaf6160806261309o3d346901jfc38fbda223ee9cb@mail.gmail.com>
References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com>	
	<ca471dc20806251400w11957e0cj85df44aae6a4c891@mail.gmail.com>	
	<1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com>	
	<4863A2C2.8000202@gmail.com>	
	<ca471dc20806260819h4293b840v1149806cf81c376c@mail.gmail.com>	
	<4863B678.5040205@gmail.com>	
	<ca471dc20806260834v18a6fe25k5daa51c4551e7f85@mail.gmail.com>
	<1afaf6160806261309o3d346901jfc38fbda223ee9cb@mail.gmail.com>
Message-ID: <4864C853.9040305@gmail.com>

Benjamin Peterson wrote:
> On Thu, Jun 26, 2008 at 10:34 AM, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Jun 26, 2008 at 8:32 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> I'm personally fine with that approach, but some of the new items in there
>>> for 2.6 could probably use a bit of cleaning up to improve the APIs and/or
>>> the way they work.
>> So get crackin'!
> 
> I'll add to my list. :)
> 
> Anyway, if we are going to make test.support public for real why don't
> we take Nick's suggestion to rename the test package to _test, but
> move test.support to a toplevel position like testutils or such.

Leaving the actual test modules undocumented (which is the situation 
now) is fine in my opinion. The _test idea was just for if we wanted to 
document test_support for our own benefit without promising to keep it 
stable for later releases.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Fri Jun 27 13:18:42 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jun 2008 21:18:42 +1000
Subject: [Python-Dev] Community buildbots and Python
	release	quality	metrics
In-Reply-To: <20080626193210.25821.328171947.divmod.xquotient.10633@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>	<20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>	<ca471dc20806261003y140f1b29sba2cdd11ef657835@mail.gmail.com>
	<20080626193210.25821.328171947.divmod.xquotient.10633@joule.divmod.com>
Message-ID: <4864CC92.2000405@gmail.com>

glyph at divmod.com wrote:
> 
> I do tend to ramble on, so here's an executive summary of my response:
> 
> I want python developers to pay attention to the community buildbots and 
> to treat breakages of existing projects as a serious issue.

Counter-proposal:

* Interested developers or users of the major third party projects 
tested by the community buildbots should monitor the community 
buildbots, and start filing bug reports with the appropriate party as 
soon as the upcoming Python release hits 2.Xa1 (i.e. first alpha).

* If the failure is due to an incompatibility between Python 2.X and 
2.X-1, then the bug report should be filed against Python. While these 
issues may or may not be addressed before the first beta, they *must* be 
addressed before the first release candidate.

* If the failure is due to an incompatibility between Python 2.X and 
2.X-2 that was properly deprecated in 2.X-1, then the bug report should 
be filed against the third party project. Prioritising these is a 
question for the developers of that project.

* Before filing a bug report against Python for a community buildbot 
failure, check if the relevant regression is also causing failures of 
the core buildbots. If it is, skip the bug report until the core 
buildbots are passing again.

It's currently a fact of life that we do NOT keep the trunk in an 
always-releasable state. We just don't. It might be nice if we did, 
there are lots of reasons why that's a good way to run a project, but at 
this point in time it isn't the case with Python. Reacting every time a 
community buildbot goes red would be a serious waste of effort.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From amauryfa at gmail.com  Fri Jun 27 13:56:54 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 27 Jun 2008 13:56:54 +0200
Subject: [Python-Dev] deepcopy
In-Reply-To: <4864BD6B.1030203@physik.rwth-aachen.de>
References: <4864BD6B.1030203@physik.rwth-aachen.de>
Message-ID: <e27efe130806270456h2ba4017cr3b7489c00e5dcfa@mail.gmail.com>

Walter Bender wrote:
Hello togehter,
>
> I get an error while doing a deep copy of an "event". An event is an object,
> with has a lot of __dict__["attr"]  = Event("SAMPLE") OTHER Events added to
> the object (no self refence) and store additional __dict__["attr2"] =  2
> informations.
>
> deep copy is failing with no proper error message.
>
>
>   nevent = copy.deepcopy(event);
>  File
> "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py",
> line 174, in deepcopy
>   y = copier(x, memo)
>  File
> "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py",
> line 305, in _deepcopy_inst
>   state = deepcopy(state, memo)
>  File
> "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py",
> line 174, in deepcopy
>   y = copier(x, memo)
>  File
> "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py",
> line 268, in _deepcopy_dict
>   y[deepcopy(key, memo)] = deepcopy(value, memo)
>  File
> "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py",
> line 174, in deepcopy
>   y = copier(x, memo)
>  File
> "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py",
> line 241, in _deepcopy_list
>   y.append(deepcopy(a, memo))
>  File
> "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py",
> line 174, in deepcopy
>   y = copier(x, memo)
>  File
> "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py",
> line 241, in _deepcopy_list
>   y.append(deepcopy(a, memo))
>  File
> "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py",
> line 185, in deepcopy
>   y = copier(x, memo)
> TypeError: __deepcopy__() takes no arguments (1 given

It's possible that "event" contains an object that cannot be copied.
>From the traceback, one can see that your object has an attribute that
is a list of lists, and the failing object is there.
Googling for the error message suggests that the object could be an
array.array.
The corresponding problem was corrected with python 2.4.4.

-- 
Amaury Forgeot d'Arc

From theller at ctypes.org  Fri Jun 27 14:45:31 2008
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 27 Jun 2008 14:45:31 +0200
Subject: [Python-Dev] Vacation
Message-ID: <g42ncl$ive$2@ger.gmane.org>

FYI: I'm going to a vacation for the next two or three weeks.
I will shutdown my buildbots, and restart them when I'm back:

x86 osx.5, x86 XP-3, amd64 XP.

Sorry for the inconvenience,
Thomas


From chris at atlee.ca  Fri Jun 27 15:42:35 2008
From: chris at atlee.ca (Chris AtLee)
Date: Fri, 27 Jun 2008 09:42:35 -0400
Subject: [Python-Dev] urllib, multipart/form-data encoding and file uploads
Message-ID: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>

Hello,

I notice that there is some work being done on urllib / urllib2 for
python 2.6/3.0.  One thing I've always missed in urllib/urllib2 is the
facility to encode POST data as multipart/form-data.  I think it would
also be useful to be able to stream a POST request to the remote
server rather than having requiring the user to create the entire POST
body in memory before starting the request.  This would be extremely
useful when writing any kind of code that does file uploads.

I didn't see any recent discussion about this so I thought I'd ask
here: do you think this would make a good addition to the new urllib
package?

Cheers,
Chris

From janssen at parc.com  Fri Jun 27 17:40:40 2008
From: janssen at parc.com (Bill Janssen)
Date: Fri, 27 Jun 2008 08:40:40 PDT
Subject: [Python-Dev] urllib,
	multipart/form-data encoding and file uploads
In-Reply-To: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> 
References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>
Message-ID: <08Jun27.084042pdt."58698"@synergy1.parc.xerox.com>

> I notice that there is some work being done on urllib / urllib2 for
> python 2.6/3.0.  One thing I've always missed in urllib/urllib2 is the
> facility to encode POST data as multipart/form-data.  I think it would
> also be useful to be able to stream a POST request to the remote
> server rather than having requiring the user to create the entire POST
> body in memory before starting the request.  This would be extremely
> useful when writing any kind of code that does file uploads.
> 
> I didn't see any recent discussion about this so I thought I'd ask
> here: do you think this would make a good addition to the new urllib
> package?

I think it would be very helpful.  I'd separate the two things,
though; you want to be able to format a set of values as
"multipart/form-data", and do various things with that resulting
"document", and you want to be able to stream a POST (or PUT) request.

Bill

From status at bugs.python.org  Fri Jun 27 18:06:27 2008
From: status at bugs.python.org (Python tracker)
Date: Fri, 27 Jun 2008 18:06:27 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20080627160627.863E678116@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (06/20/08 - 06/27/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1933 open (+40) / 13106 closed (+25) / 15039 total (+65)

Open issues with patches:   600

Average duration of open issues: 707 days.
Median duration of open issues: 1532 days.

Open Issues Breakdown
   open  1908 (+39)
pending    25 ( +1)

Issues Created Or Reopened (69)
_______________________________

Move Demo/classes/Rat.py to Lib/fractions.py and fix it up.      06/25/08
       http://bugs.python.org/issue1682    reopened marketdickinson           
       patch                                                                   

pickling of large recursive structures crashes cPickle           06/25/08
       http://bugs.python.org/issue2702    reopened facundobatista            
       patch                                                                   

Let bin/oct/hex show floats                                      06/25/08
       http://bugs.python.org/issue3008    reopened rhettinger                
       patch                                                                   

ast module docs document private attribute                       06/20/08
CLOSED http://bugs.python.org/issue3152    created  brett.cannon              
       easy, 26backport                                                        

sqlite leaks on error                                            06/20/08
       http://bugs.python.org/issue3153    created  Rhamphoryncus             
                                                                               

"Quick search" box renders too long on FireFox 3                 06/20/08
       http://bugs.python.org/issue3154    created  tebeka                    
                                                                               

Python should expose a pthread_cond_timedwait API for threading  06/20/08
       http://bugs.python.org/issue3155    created  gregory.p.smith           
                                                                               

bytes type has inconsistent methods (append, insert)             06/20/08
       http://bugs.python.org/issue3156    created  akuchling                 
       patch                                                                   

sqlite3 minor documentation issues                               06/21/08
CLOSED http://bugs.python.org/issue3157    created  slewis                    
                                                                               

Doctest fails to find doctests in extension modules              06/21/08
       http://bugs.python.org/issue3158    created  fer_perez                 
                                                                               

glob.py improvements                                             06/21/08
       http://bugs.python.org/issue3159    created  Indy                      
       patch                                                                   

Building a Win32 binary installer crashes                        06/21/08
       http://bugs.python.org/issue3160    created  complex                   
                                                                               

Missing import for sys in _abcoll                                06/21/08
CLOSED http://bugs.python.org/issue3161    created  hodgestar                 
       patch                                                                   

ssl.SSLSocket implements methods that are overriden by socket.so 06/21/08
       http://bugs.python.org/issue3162    created  hodgestar                 
                                                                               

module struct support for ssize_t and size_t                     06/21/08
       http://bugs.python.org/issue3163    created  MrJean1                   
       patch                                                                   

cPickle calls to save_string and save_unicode with unicode objec 06/21/08
CLOSED http://bugs.python.org/issue3164    created  cuerty                    
       patch                                                                   

cPickle recursion problem                                        06/21/08
CLOSED http://bugs.python.org/issue3165    created  cuerty                    
       patch                                                                   

Make conversions from long to float correctly rounded.           06/21/08
       http://bugs.python.org/issue3166    created  marketdickinson           
       patch                                                                   

math test fails on Solaris 10                                    06/21/08
       http://bugs.python.org/issue3167    created  MrJean1                   
                                                                               

cmath test fails on Solaris 10                                   06/21/08
       http://bugs.python.org/issue3168    created  MrJean1                   
                                                                               

email/header.py doesn't handle Base64 headers that have been ins 06/21/08
       http://bugs.python.org/issue3169    created  jasonjwwilliams           
                                                                               

test_pydoc has no way to regenerate pristine data                06/22/08
       http://bugs.python.org/issue3170    created  brett.cannon              
                                                                               

operator.*slice() should be deprecated in 2.6                    06/22/08
       http://bugs.python.org/issue3171    created  scoder                    
                                                                               

test macos skipped "unexpected" on linux2                        06/22/08
CLOSED http://bugs.python.org/issue3172    created  MrJean1                   
                                                                               

external strftime for Python?                                    06/23/08
       http://bugs.python.org/issue3173    created  skip.montanaro            
       patch, patch                                                            

3.0b1 doesn't seem to install on macs                            06/23/08
       http://bugs.python.org/issue3174    created  erickt                    
                                                                               

multiprocessing build fails on Fedora 8 and Xubuntu 8 + solution 06/23/08
CLOSED http://bugs.python.org/issue3175    created  mark                      
                                                                               

result error for 2/5 and 4/5                                     06/23/08
CLOSED http://bugs.python.org/issue3176    created  hiweed                    
                                                                               

implement os.startfile on posix and MacOSX                       06/23/08
       http://bugs.python.org/issue3177    created  ganadist                  
       patch                                                                   

__radd__(self,other) isn't called if self and other are of the s 06/23/08
CLOSED http://bugs.python.org/issue3178    created  Frosburn                  
                                                                               

cPickle is seriously broken                                      06/23/08
CLOSED http://bugs.python.org/issue3179    created  schmir                    
       patch                                                                   

Interrupts are lost during readline PyOS_InputHook processing    06/23/08
       http://bugs.python.org/issue3180    created  Araneidae                 
       patch                                                                   

ConfigParsers are classic classes                                06/23/08
       http://bugs.python.org/issue3181    created  hauva                     
       patch                                                                   

2to3 Slight Patch                                                06/23/08
       http://bugs.python.org/issue3182    created  nedds                     
                                                                               

sha modules & Modules/Setup.dist                                 06/23/08
       http://bugs.python.org/issue3183    created  rwgk                      
                                                                               

The elementtree.ElementTree Module doc is missing                06/23/08
CLOSED http://bugs.python.org/issue3184    created  gladstein                 
                                                                               

Documentation of time.time() differs from source documentation   06/24/08
CLOSED http://bugs.python.org/issue3185    created  cgrohmann                 
                                                                               

bin(long) doesn't have a trailing 'L'                            06/24/08
CLOSED http://bugs.python.org/issue3186    created  marketdickinson           
                                                                               

os.listdir can return byte strings                               06/24/08
       http://bugs.python.org/issue3187    reopened amaury.forgeotdarc        
       patch                                                                   

float('infinity') should be valid                                06/24/08
       http://bugs.python.org/issue3188    created  marketdickinson           
       patch                                                                   

Py3k DeprecationWarning in difflib                               06/24/08
CLOSED http://bugs.python.org/issue3189    created  vdupras                   
       patch                                                                   

Pydoc should ignore __package__ attributes                       06/24/08
       http://bugs.python.org/issue3190    created  ncoghlan                  
                                                                               

round docstring is inaccurate                                    06/24/08
       http://bugs.python.org/issue3191    created  marketdickinson           
                                                                               

exec(open(filename)) doesn't work                                06/24/08
CLOSED http://bugs.python.org/issue3192    created  ms                        
                                                                               

Python 2.5.2 - Python                                            06/24/08
CLOSED http://bugs.python.org/issue3193    created  peterdilley               
                                                                               

Demo/loop.c passing "char *" instead of "wchar_t *"              06/25/08
       http://bugs.python.org/issue3194    created  zanella                   
       patch                                                                   

invalid conversion xml.etree.ElementTree.Element object to boole 06/25/08
CLOSED http://bugs.python.org/issue3195    created  xaka                      
                                                                               

Option in pydoc to show docs from private methods                06/25/08
       http://bugs.python.org/issue3196    created  mortenlj                  
       patch                                                                   

Documentation for fractions module needs work                    06/25/08
       http://bugs.python.org/issue3197    created  marketdickinson           
       patch                                                                   

strings don't seem to roundtrip with repr()                      06/25/08
CLOSED http://bugs.python.org/issue3198    created  mark                      
                                                                               

2.6b1 Build Fails On OSX 10.5                                    06/25/08
CLOSED http://bugs.python.org/issue3199    created  johna                     
                                                                               

Python's crash in heavy multithreading IO operations             06/25/08
CLOSED http://bugs.python.org/issue3200    created  sadit                     
                                                                               

struct.Struct size attribute undocumented                        06/25/08
CLOSED http://bugs.python.org/issue3201    created  ericvw                    
                                                                               

Wish: disable tests in unittest                                  06/25/08
       http://bugs.python.org/issue3202    created  jmp                       
       patch                                                                   

sphinx - table of contents doesn't render correctly (html)       06/25/08
       http://bugs.python.org/issue3203    created  grflanagan                
                                                                               

operator module docs are not updated to 3.0                      06/26/08
CLOSED http://bugs.python.org/issue3204    created  mishok13                  
       patch                                                                   

bz2 iterator fails silently on MemoryError                       06/26/08
       http://bugs.python.org/issue3205    created  mdehoon                   
       patch                                                                   

Multiprocessing Array and sharedctypes.Array error in docs/imple 06/26/08
       http://bugs.python.org/issue3206    created  mishok13                  
       patch                                                                   

file.write() after file.readline() in mode "r+"                  06/26/08
CLOSED http://bugs.python.org/issue3207    created  peterdemin                
                                                                               

function annotation for builtin and C function                   06/26/08
       http://bugs.python.org/issue3208    created  bhy                       
                                                                               

Grammar error in UserDict module docs                            06/26/08
CLOSED http://bugs.python.org/issue3209    created  technel                   
                                                                               

subprocess.Popen does not release process handles if process can 06/26/08
       http://bugs.python.org/issue3210    created  gjb1002                   
                                                                               

warnings.warn_explicit raises SystemError                        06/26/08
CLOSED http://bugs.python.org/issue3211    created  exarkun                   
                                                                               

ssl module - should test for a wrong cert                        06/26/08
       http://bugs.python.org/issue3212    created  jonas.wagner              
       patch                                                                   

"pydoc -p" should listen to [::] if IPv6 is supported            06/27/08
       http://bugs.python.org/issue3213    created  ndim                      
       patch                                                                   

Suggest change to glossary explanation:  "Duck Typing"           06/27/08
       http://bugs.python.org/issue3214    created  paddy3118                 
                                                                               

Can't import sqlite3 in Python 2.6b1                             06/27/08
       http://bugs.python.org/issue3215    created  craigneuro                
                                                                               

errors in msilib documentation                                   06/27/08
       http://bugs.python.org/issue3216    created  sjoerd                    
                                                                               

IndexError: Add bad index to msg                                 06/23/08
       http://bugs.python.org/issue1534607 reopened benjamin.peterson         
                                                                               



Issues Now Closed (52)
______________________

simple patch, improving unreachable bytecode removing             232 days
       http://bugs.python.org/issue1394    gvanrossum                
       patch                                                                   

os.system() fails for commands with multiple quoted file names    208 days
       http://bugs.python.org/issue1524    d4rk1                     
                                                                               

Failure when calling __str__ for MIMEBase(message, rfc822) objec  199 days
       http://bugs.python.org/issue1556    facundobatista            
                                                                               

module-cgi: handling GET and POST together                        161 days
       http://bugs.python.org/issue1817    facundobatista            
       patch, easy                                                             

shutil: NameError (WindowsError) when moving from ext3 to fat32    80 days
       http://bugs.python.org/issue2549    draghuram                 
       easy                                                                    

default_scheme in urlparse.urlparse() useless                      75 days
       http://bugs.python.org/issue2569    facundobatista            
                                                                               

csv.reader accepts string instead of file object (duck typing go   56 days
       http://bugs.python.org/issue2701    facundobatista            
                                                                               

os.getcwd fails for long path names on linux                       54 days
       http://bugs.python.org/issue2722    facundobatista            
       patch, easy                                                             

curses.textpad loses characters at the end of lines                52 days
       http://bugs.python.org/issue2732    akuchling                 
                                                                               

pprint produces different output in 2.6 and 3.0                    36 days
       http://bugs.python.org/issue2888    facundobatista            
       patch                                                                   

Verify doc updates for the decimal module                          21 days
       http://bugs.python.org/issue3017    facundobatista            
                                                                               

Recursion bug in deepcopy                                          16 days
       http://bugs.python.org/issue3043    facundobatista            
                                                                               

heapq change breaking compatibility                                13 days
       http://bugs.python.org/issue3051    rhettinger                
       patch                                                                   

Let set.union and set.intersection accept multiple arguments       13 days
       http://bugs.python.org/issue3069    rhettinger                
       patch                                                                   

chapter 17.1.3.5 'Replacing os.popen*' in the Python library ref   11 days
       http://bugs.python.org/issue3085    benjamin.peterson         
       patch                                                                   

str.format("{0:n}") poss. bug with setlocale()                      5 days
       http://bugs.python.org/issue3140    eric.smith                
                                                                               

urllib docs don't match the new modules                             4 days
       http://bugs.python.org/issue3142    georg.brandl              
                                                                               

Sphinx/LaTeX fails on Python 3.0b1 documentation                    3 days
       http://bugs.python.org/issue3146    georg.brandl              
                                                                               

ast module docs document private attribute                          0 days
       http://bugs.python.org/issue3152    loewis                    
       easy, 26backport                                                        

sqlite3 minor documentation issues                                  2 days
       http://bugs.python.org/issue3157    georg.brandl              
                                                                               

Missing import for sys in _abcoll                                   1 days
       http://bugs.python.org/issue3161    rhettinger                
       patch                                                                   

cPickle calls to save_string and save_unicode with unicode objec    1 days
       http://bugs.python.org/issue3164    facundobatista            
       patch                                                                   

cPickle recursion problem                                           4 days
       http://bugs.python.org/issue3165    facundobatista            
       patch                                                                   

test macos skipped "unexpected" on linux2                           0 days
       http://bugs.python.org/issue3172    benjamin.peterson         
                                                                               

multiprocessing build fails on Fedora 8 and Xubuntu 8 + solution    0 days
       http://bugs.python.org/issue3175    benjamin.peterson         
                                                                               

result error for 2/5 and 4/5                                        0 days
       http://bugs.python.org/issue3176    amaury.forgeotdarc        
                                                                               

__radd__(self,other) isn't called if self and other are of the s    0 days
       http://bugs.python.org/issue3178    amaury.forgeotdarc        
                                                                               

cPickle is seriously broken                                         3 days
       http://bugs.python.org/issue3179    jcea                      
       patch                                                                   

The elementtree.ElementTree Module doc is missing                   0 days
       http://bugs.python.org/issue3184    gladstein                 
                                                                               

Documentation of time.time() differs from source documentation      0 days
       http://bugs.python.org/issue3185    loewis                    
                                                                               

bin(long) doesn't have a trailing 'L'                               0 days
       http://bugs.python.org/issue3186    marketdickinson           
                                                                               

Py3k DeprecationWarning in difflib                                  0 days
       http://bugs.python.org/issue3189    rhettinger                
       patch                                                                   

exec(open(filename)) doesn't work                                   2 days
       http://bugs.python.org/issue3192    georg.brandl              
                                                                               

Python 2.5.2 - Python                                               0 days
       http://bugs.python.org/issue3193    peterdilley               
                                                                               

invalid conversion xml.etree.ElementTree.Element object to boole    0 days
       http://bugs.python.org/issue3195    benjamin.peterson         
                                                                               

strings don't seem to roundtrip with repr()                         0 days
       http://bugs.python.org/issue3198    amaury.forgeotdarc        
                                                                               

2.6b1 Build Fails On OSX 10.5                                       0 days
       http://bugs.python.org/issue3199    marketdickinson           
                                                                               

Python's crash in heavy multithreading IO operations                0 days
       http://bugs.python.org/issue3200    amaury.forgeotdarc        
                                                                               

struct.Struct size attribute undocumented                           0 days
       http://bugs.python.org/issue3201    benjamin.peterson         
                                                                               

operator module docs are not updated to 3.0                         0 days
       http://bugs.python.org/issue3204    georg.brandl              
       patch                                                                   

file.write() after file.readline() in mode "r+"                     0 days
       http://bugs.python.org/issue3207    amaury.forgeotdarc        
                                                                               

Grammar error in UserDict module docs                               0 days
       http://bugs.python.org/issue3209    georg.brandl              
                                                                               

warnings.warn_explicit raises SystemError                           0 days
       http://bugs.python.org/issue3211    brett.cannon              
                                                                               

POP3 lib timeout                                                 1602 days
       http://bugs.python.org/issue888830  facundobatista            
                                                                               

xrange overflows                                                 1417 days
       http://bugs.python.org/issue1003935 rhettinger                
                                                                               

docs for Py_UNICODE are wrong                                    1349 days
       http://bugs.python.org/issue1044479 facundobatista            
       easy                                                                    

functions replaced by subprocess should point to its docs        1287 days
       http://bugs.python.org/issue1083895 facundobatista            
                                                                               

BaseHTTPServer uses deprecated mimetools.Message                 1169 days
       http://bugs.python.org/issue1180470 benjamin.peterson         
                                                                               

installation problem with python 2.4.1 on Win2k system           1137 days
       http://bugs.python.org/issue1199808 loewis                    
                                                                               

poplib.py list interface                                          633 days
       http://bugs.python.org/issue1567948 facundobatista            
                                                                               

File Read/Write Flushing Patch                                    527 days
       http://bugs.python.org/issue1636874 amaury.forgeotdarc        
       patch                                                                   

Add Tkinter.Checkbutton get() and set(value)                      317 days
       http://bugs.python.org/issue1774370 gpolo                     
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 12 math test fails on Solaris 10                                      6 days
open    http://bugs.python.org/issue3167   

 12 implement PEP 3134 exception reporting                            13 days
open    http://bugs.python.org/issue3112   

 11 sys.getsizeof() gives an AttributeError for _sre objects.         11 days
open    http://bugs.python.org/issue3122   

  9 pickling of large recursive structures crashes cPickle             2 days
open    http://bugs.python.org/issue2702   

  8 Let bin/oct/hex show floats                                        3 days
open    http://bugs.python.org/issue3008   

  8 No float formatting in PyString_FromFormat                        48 days
open    http://bugs.python.org/issue2813   

  8 email.MIMEText.MIMEText.as_string incorrectly folding long subj  149 days
open    http://bugs.python.org/issue1974   

  7 exec(open(filename)) doesn't work                                  2 days
closed  http://bugs.python.org/issue3192   

  7 cPickle is seriously broken                                        3 days
closed  http://bugs.python.org/issue3179   

  6 rlcompleter add "(" to callables feature                        2515 days
open    http://bugs.python.org/issue449227 




From guido at python.org  Fri Jun 27 20:10:11 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Jun 2008 11:10:11 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <932f8baf0806261522m7b5eb75aiac986ea029cdd6ff@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>
	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>
	<932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com>
	<ca471dc20806261055g5219d34fi23b84f9462af8962@mail.gmail.com>
	<932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com>
	<ca471dc20806261145m164374bnb4dcdf770c2be0fa@mail.gmail.com>
	<932f8baf0806261522m7b5eb75aiac986ea029cdd6ff@mail.gmail.com>
Message-ID: <ca471dc20806271110y1d2eb549uceb2ee6965a7a964@mail.gmail.com>

On Thu, Jun 26, 2008 at 3:22 PM, Ralf Schmitt <schmir at gmail.com> wrote:
> On Thu, Jun 26, 2008 at 8:45 PM, Guido van Rossum <guido at python.org> wrote:
>>
>> So you're saying py.test needs to be fixed? Fine with me, but then I
>> don't understand why you bothered bringing it up here instead of
>> fixing it (or reporting to the py.test maintainers, I don't know if
>> you're one or not).
>>
>
> no, I'm not. Just wanted to point out, that this ticket/patch does not
> solve the django issue.
> (this is what I first thought it would do).

I'm still missing details.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun 27 20:42:47 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Jun 2008 11:42:47 -0700
Subject: [Python-Dev] Community buildbots and Python release quality
	metrics
In-Reply-To: <20080627003356.25821.685560775.divmod.xquotient.10844@joule.divmod.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>
	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>
	<20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com>
	<4863CE78.7010108@v.loewis.de>
	<20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com>
	<ca471dc20806261417t7c562083u4cae90b94f7e1381@mail.gmail.com>
	<20080627003356.25821.685560775.divmod.xquotient.10844@joule.divmod.com>
Message-ID: <ca471dc20806271142v4f079c3codb6f09f11eb318f2@mail.gmail.com>

On Thu, Jun 26, 2008 at 5:33 PM,  <glyph at divmod.com> wrote:
> OK, fair enough.  Taking a step back, I was pushing this really hard because
> to *me*, it seems like dealing with the influx of bug reports after the fact
> is an unreasonable amount of additional work, whereas immediate reverts are
> just an occasional, temporary inconvenience. Based on my experience, "We'll
> deal with the reports as they come in" sounds like "no".

No, that's not how it was intended. I'd rather hear you tell us about
your pain than telling us how to fix it.

> I think since the last time we had a similar conversation, other Twisted
> developers have been fairly diligent in reporting bugs.  (In the time
> they've been reporting Python bugs, I've mostly been planning a wedding.
> Others who have survived it tell me that eventually, this process ends...
> and then I should be participating more directly.)  I'll try to step up that
> feeback loop and get other projects involved.  I hope I am wrong about
> generating an unreasonable amount of work.

Again, I'd much rather see you generate a huge pile of work for us in
the form of specific bug reports than trying to tell us how to change
our process.

> Thanks.  I appreciate it; an increased emphasis on 3rd party code being
> broken is really what I was looking for.  This is fine with me.  I mean,
> whatever your decision is, I'm going to have to live with it :), but in
> either case, we have to be looking for bugs and helping to investigate them.
>  On my end of things I guess it's not going to make much difference.

Right. The beta period is the time to pay attention to 3rd party
breakage. While we won't make any specific promises, we're taking 3rd
party code breakage very seriously and, depending on the specifics, it
can hold up a release. It just won't be automatic; many 3rd party
breakages are most easily fixed by making small adjustments to the 3rd
party code involved. (Imagine a 3rd party package still using string
exceptions.)

.
.
.

On Thu, Jun 26, 2008 at 6:13 PM,  <glyph at divmod.com> wrote:
> For the record, "automatic revert" does not mean "drop all other work". The
> changeset's still there.  It's still in the revision history.  It can easily
> be applied to anybody's working copy.  It can easily be put into a branch.
>  It can be automatically re-merged later.  I do all of these things all the
> time, and I was *not* intending to suggest that any 3rd-party breakage
> should cause a code freeze or anything even remotely like that.

Still, a revert often holds up other work that depends on the reverted
change. I expect that in most cases, finding the problem and applying
a fix is much less of a burden for the core developer team as a whole
than rolling back, working on a fix, and re-applying. I also expect
that the policy of semi-automated merges from 2.6 into 3.0 would make
a revert even more work, and more likely to cause second-order problem
in the 3.0 branch.

>>> (...) it would have been easier to
>>> convince a Twisted developer to do the work it to keep the community
>>> buildbot green rather than to make it a bit less red.
>>
>> Why? That sounds like it's a problem with the psychology of the
>> Twisted developers.
>
> I hardly think it's unique to us.  TDD test runners typically only know 2
> colors and 2 states: "passed" and "fails".  Once you're in the "fail" state,
> you tend to accumulate more failures; there's a much bigger bang for your
> buck.  Better tools with nicer interfaces would let you easily mark
> individual tests as "usually intermittent" or something and make it a
> "slightly dirty green" or something, but we don't have those.  The move from
> "red" to "green" is much more psychologically significant to just about
> anyone I know than the move from "red but 14 failures" to "red but 12
> failures".

Still sounds like perhaps you're too much focused on the "green bar".
I know this is encouraged by XP, but I don't want to have a religious
debates about whether XP is the right development model for Python.
(The time would be better spent on improving the buildbot reporting!)

> Despite the idea's origins in a now-highly-controversial book on
> criminology, this is often referred to in XP discussion circles (wikis and
> the like) as the "fix broken windows" collaboration pattern.  I notice this
> in lots of other areas of my life; a little pile of papers tends to become a
> big pile of papers, the first dent in a car makes the driver a little less
> careful, and so on.

For me, with that first dent comes the realization that it's really
not worth it to obsess over scratches, and that makes me a more
relaxed and hence *better* (because less stressed) driver. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun 27 21:02:32 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Jun 2008 12:02:32 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <48643E3F.3010500@canterbury.ac.nz>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
	<48643E3F.3010500@canterbury.ac.nz>
Message-ID: <ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>

Now that I've learned about the hex float format supported by C++ and
Java, I wonder if it wouldn't be better to support conversion to and
from that format and nothing else.

E.g.

>>> math.tohex(3.14)
'0x1.91eb851eb851fp+1'
>>> math.fromhex('0x1.91eb851eb851fp+1')
3.1400000000000001
>>>

BTW I am still hoping to be able the output of the second command to
just "3.14" but the tracker issue for that
(http://bugs.python.org/issue1580) is stuck on trying to decide
whether it's okay to have repr(<float>) occasionally return a string
that doesn't convert to the exact same value on another platform. (My
preferred approach would ensure that it does convert to the same value
on the same platform, because that's how I'd compute it.) Perhaps the
existance of an unambiguous hex format that is also interchangeable
with Java and C (and presumably C++) would alleviate that concern.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Jun 27 21:04:34 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Jun 2008 12:04:34 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
	<48643E3F.3010500@canterbury.ac.nz>
	<ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
Message-ID: <ca471dc20806271204u6e194947u42968fbcd1f6410a@mail.gmail.com>

PS. I can't get excited about having support for this in %-style
format strings (and even less so now %a already means "call ascii()").
It would be easy enough to add support for it to float.__format__()
though.

On Fri, Jun 27, 2008 at 12:02 PM, Guido van Rossum <guido at python.org> wrote:
> Now that I've learned about the hex float format supported by C++ and
> Java, I wonder if it wouldn't be better to support conversion to and
> from that format and nothing else.
>
> E.g.
>
>>>> math.tohex(3.14)
> '0x1.91eb851eb851fp+1'
>>>> math.fromhex('0x1.91eb851eb851fp+1')
> 3.1400000000000001
>>>>
>
> BTW I am still hoping to be able the output of the second command to
> just "3.14" but the tracker issue for that
> (http://bugs.python.org/issue1580) is stuck on trying to decide
> whether it's okay to have repr(<float>) occasionally return a string
> that doesn't convert to the exact same value on another platform. (My
> preferred approach would ensure that it does convert to the same value
> on the same platform, because that's how I'd compute it.) Perhaps the
> existance of an unambiguous hex format that is also interchangeable
> with Java and C (and presumably C++) would alleviate that concern.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tomerfiliba at gmail.com  Fri Jun 27 21:07:30 2008
From: tomerfiliba at gmail.com (tomer filiba)
Date: Fri, 27 Jun 2008 21:07:30 +0200
Subject: [Python-Dev] repeated keyword arguments
Message-ID: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>

the following code works on python 2.5:

>>> def f(**kwargs):
...     print kwargs
...
>>> f(a=5,b=7,a=8)
{'a': 8, 'b': 7}
>>>

but fails on python2.4, saying that "a" is given twice.
is this a bug or a feature?


-tomer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080627/900bc8cc/attachment.htm>

From guido at python.org  Fri Jun 27 21:11:07 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Jun 2008 12:11:07 -0700
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
Message-ID: <ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>

Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due
to the switch to the  new AST-based compiler. Can you file a bug? I
think we should leave 2.5 alone (too much risk of breaking code) but
fix it in 2.6 and 3.0 if we can.

--Guido

On Fri, Jun 27, 2008 at 12:07 PM, tomer filiba <tomerfiliba at gmail.com> wrote:
> the following code works on python 2.5:
>
>>>> def f(**kwargs):
> ...     print kwargs
> ...
>>>> f(a=5,b=7,a=8)
> {'a': 8, 'b': 7}
>>>>
>
> but fails on python2.4, saying that "a" is given twice.
> is this a bug or a feature?
>
>
> -tomer
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From chris at atlee.ca  Fri Jun 27 22:20:59 2008
From: chris at atlee.ca (Chris AtLee)
Date: Fri, 27 Jun 2008 16:20:59 -0400
Subject: [Python-Dev] urllib,
	multipart/form-data encoding and file uploads
In-Reply-To: <4610679334594632865@unknownmsgid>
References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>
	<4610679334594632865@unknownmsgid>
Message-ID: <7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com>

On Fri, Jun 27, 2008 at 11:40 AM, Bill Janssen <janssen at parc.com> wrote:
>> I notice that there is some work being done on urllib / urllib2 for
>> python 2.6/3.0.  One thing I've always missed in urllib/urllib2 is the
>> facility to encode POST data as multipart/form-data.  I think it would
>> also be useful to be able to stream a POST request to the remote
>> server rather than having requiring the user to create the entire POST
>> body in memory before starting the request.  This would be extremely
>> useful when writing any kind of code that does file uploads.
>>
>> I didn't see any recent discussion about this so I thought I'd ask
>> here: do you think this would make a good addition to the new urllib
>> package?
>
> I think it would be very helpful.  I'd separate the two things,
> though; you want to be able to format a set of values as
> "multipart/form-data", and do various things with that resulting
> "document", and you want to be able to stream a POST (or PUT) request.

How about if the function that encoded the values as "multipart/form-data"
was able to stream data to a POST (or PUT) request via an iterator that
yielded chunks of data?

def multipart_encode(params, boundary=None):
    """Encode ``params`` as multipart/form-data.

    ``params`` should be a dictionary where the keys represent parameter names,
    and the values are either parameter values, or file-like objects to
    use as the parameter value.  The file-like object must support the .read(),
    .seek(), and .tell() methods.

    If ``boundary`` is set, then it as used as the MIME boundary.  Otherwise
    a randomly generated boundary will be used.  In either case, if the
    boundary string appears in the parameter values a ValueError will be
    raised.

    Returns an iterable object that will yield blocks of data representing
    the encoded parameters."""

The file objects need to support .seek() and .tell() so we can determine
how large they are before including them in the output.  I've been trying
to come up with a good way to specify the size separately so you could use
unseekable objects, but no good ideas have come to mind.  Maybe it could
look for a 'size' attribute or callable on the object?  That seems a bit
hacky...

A couple helper functions would be necessary as well, one to generate
random boundary strings that are guaranteed not to collide with file data,
and another function to calculate the total size of the encoding to be used
in the 'Content-Length' header in the main HTTP request.

Then we'd need to change either urllib or httplib to support iterable
objects in addition to the regular strings that it currently uses.

Cheers,
Chris

From steve at holdenweb.com  Fri Jun 27 23:18:42 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 27 Jun 2008 17:18:42 -0400
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
Message-ID: <48655932.2010203@holdenweb.com>

So we wait until they port their code to 2.6 to break it?

regards
  Steve

Guido van Rossum wrote:
> Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due
> to the switch to the  new AST-based compiler. Can you file a bug? I
> think we should leave 2.5 alone (too much risk of breaking code) but
> fix it in 2.6 and 3.0 if we can.
> 
> --Guido
> 
> On Fri, Jun 27, 2008 at 12:07 PM, tomer filiba <tomerfiliba at gmail.com> wrote:
>> the following code works on python 2.5:
>>
>>>>> def f(**kwargs):
>> ...     print kwargs
>> ...
>>>>> f(a=5,b=7,a=8)
>> {'a': 8, 'b': 7}
>> but fails on python2.4, saying that "a" is given twice.
>> is this a bug or a feature?
>>
>>
>> -tomer
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
> 
> 
> 


-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Fri Jun 27 23:18:42 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 27 Jun 2008 17:18:42 -0400
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
Message-ID: <48655932.2010203@holdenweb.com>

So we wait until they port their code to 2.6 to break it?

regards
  Steve

Guido van Rossum wrote:
> Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due
> to the switch to the  new AST-based compiler. Can you file a bug? I
> think we should leave 2.5 alone (too much risk of breaking code) but
> fix it in 2.6 and 3.0 if we can.
> 
> --Guido
> 
> On Fri, Jun 27, 2008 at 12:07 PM, tomer filiba <tomerfiliba at gmail.com> wrote:
>> the following code works on python 2.5:
>>
>>>>> def f(**kwargs):
>> ...     print kwargs
>> ...
>>>>> f(a=5,b=7,a=8)
>> {'a': 8, 'b': 7}
>> but fails on python2.4, saying that "a" is given twice.
>> is this a bug or a feature?
>>
>>
>> -tomer
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
> 
> 
> 


-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From musiccomposition at gmail.com  Fri Jun 27 23:23:30 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 27 Jun 2008 16:23:30 -0500
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
Message-ID: <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>

On Fri, Jun 27, 2008 at 2:11 PM, Guido van Rossum <guido at python.org> wrote:
> Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due
> to the switch to the  new AST-based compiler. Can you file a bug? I
> think we should leave 2.5 alone (too much risk of breaking code) but
> fix it in 2.6 and 3.0 if we can.

I think code that uses this is probably already quite broken in some
fundamental way and putting the fix in 2.5 isn't much of a risk.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From wolever at cs.toronto.edu  Fri Jun 27 23:30:20 2008
From: wolever at cs.toronto.edu (David Wolever)
Date: Fri, 27 Jun 2008 18:30:20 -0300
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
Message-ID: <7AA4A53F-3A92-49D6-ABDE-EEEFC684F2B3@cs.toronto.edu>

On 27-Jun-08, at 6:23 PM, Benjamin Peterson wrote:
> On Fri, Jun 27, 2008 at 2:11 PM, Guido van Rossum  
> <guido at python.org> wrote:
>> Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably  
>> due
>> to the switch to the  new AST-based compiler. Can you file a bug? I
>> think we should leave 2.5 alone (too much risk of breaking code) but
>> fix it in 2.6 and 3.0 if we can.
>
> I think code that uses this is probably already quite broken in some
> fundamental way and putting the fix in 2.5 isn't much of a risk.

I don't have 2.4 handy to test it, but it is more likely that a  
keyword and dictionary are passed, both containing the same item:
 >>> f(a=3, **{'a': 4})

Would that be a potential risk?

From scott+python-dev at scottdial.com  Fri Jun 27 23:40:21 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Fri, 27 Jun 2008 17:40:21 -0400
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <7AA4A53F-3A92-49D6-ABDE-EEEFC684F2B3@cs.toronto.edu>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
	<7AA4A53F-3A92-49D6-ABDE-EEEFC684F2B3@cs.toronto.edu>
Message-ID: <48655E45.50503@scottdial.com>

David Wolever wrote:
> I don't have 2.4 handy to test it, but it is more likely that a keyword 
> and dictionary are passed, both containing the same item:
>  >>> f(a=3, **{'a': 4})
> 
> Would that be a potential risk?

Python 2.4.3
 >>> f(a=3, **{'a': 4})
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
TypeError: f() got multiple values for keyword argument 'a'

Python 2.5
 >>> f(a=3, **{'a': 4})
TypeError: f() got multiple values for keyword argument 'a'

The regression is purely in the way an argument list is reduced to a 
dictionary.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From fdrake at acm.org  Fri Jun 27 23:54:46 2008
From: fdrake at acm.org (Fred Drake)
Date: Fri, 27 Jun 2008 17:54:46 -0400
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
Message-ID: <F14738BD-9AC7-4D25-B376-D81D5AC524E2@acm.org>

On Jun 27, 2008, at 5:23 PM, Benjamin Peterson wrote:
> I think code that uses this is probably already quite broken in some
> fundamental way and putting the fix in 2.5 isn't much of a risk.

I suspect the risk has more to do with breaking something else in  
Python than in breaking 3rd-party code in this case.

I think it should be fixed for 2.5 as well, myself.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From dickinsm at gmail.com  Fri Jun 27 23:54:52 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 27 Jun 2008 22:54:52 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
	<48643E3F.3010500@canterbury.ac.nz>
	<ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
Message-ID: <5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com>

On Fri, Jun 27, 2008 at 8:02 PM, Guido van Rossum <guido at python.org> wrote:
> Now that I've learned about the hex float format supported by C++ and
> Java, I wonder if it wouldn't be better to support conversion to and
> from that format and nothing else.
>
> E.g.
>
>>>> math.tohex(3.14)
> '0x1.91eb851eb851fp+1'
>>>> math.fromhex('0x1.91eb851eb851fp+1')
> 3.1400000000000001

This would certainly be enough for me, though I think there's still
some educational value in having binary output available.  But
that's just a matter of substituting a four-bit binary string for
each hexadecimal digit (or learning to read hexadecimal as
though it were binary).

In fromhex, what would be done with a string that gives more
hex digits than the machine precision can support?  An obvious
answer is just to round to the nearest float, but since part of the
point of hex floats is having a way to specify a given value
*exactly*, it might make more sense to raise an exception
rather than changing the value by rounding it.

Mark

From scott+python-dev at scottdial.com  Sat Jun 28 00:00:23 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Fri, 27 Jun 2008 18:00:23 -0400
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <48655E45.50503@scottdial.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>	<7AA4A53F-3A92-49D6-ABDE-EEEFC684F2B3@cs.toronto.edu>
	<48655E45.50503@scottdial.com>
Message-ID: <486562F7.80608@scottdial.com>

Scott Dial wrote:
> The regression is purely in the way an argument list is reduced to a 
> dictionary.

To further elaborate:

Python 2.4:
 >>> import dis
 >>> dis.dis(compile('f(a=3, b=1, a=4)', '', 'eval'))
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
SyntaxError: duplicate keyword argument

Python 2.5:
 >>> import dis
 >>> dis.dis(compile('f(a=3, b=1, a=4)', '', 'eval'))
   1           0 LOAD_NAME                0 (f)
               3 LOAD_CONST               0 ('a')
               6 LOAD_CONST               1 (3)
               9 LOAD_CONST               2 ('b')
              12 LOAD_CONST               3 (1)
              15 LOAD_CONST               0 ('a')
              18 LOAD_CONST               4 (4)
              21 CALL_FUNCTION          768
              24 RETURN_VALUE

The old compiler checked for this, but the AST-based compiler just 
blindly compiles the entire keyword argument sequence.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From musiccomposition at gmail.com  Sat Jun 28 00:03:36 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 27 Jun 2008 17:03:36 -0500
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <486562F7.80608@scottdial.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
	<7AA4A53F-3A92-49D6-ABDE-EEEFC684F2B3@cs.toronto.edu>
	<48655E45.50503@scottdial.com> <486562F7.80608@scottdial.com>
Message-ID: <1afaf6160806271503p7157c91x1aeb82a5224c7589@mail.gmail.com>

On Fri, Jun 27, 2008 at 5:00 PM, Scott Dial
<scott+python-dev at scottdial.com> wrote:
>
> The old compiler checked for this, but the AST-based compiler just blindly
> compiles the entire keyword argument sequence.

The new compiler will check for it when my patch on the issue 3219 is applied.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

From dickinsm at gmail.com  Sat Jun 28 00:50:35 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 27 Jun 2008 23:50:35 +0100
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4863AA71.5020901@gmail.com>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
	<48643E3F.3010500@canterbury.ac.nz>
	<ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
Message-ID: <5c6f2a5d0806271550yc0623cbxe0c6b2bc677b57d4@mail.gmail.com>

On Fri, Jun 27, 2008 at 8:02 PM, Guido van Rossum <guido at python.org> wrote:
> Now that I've learned about the hex float format supported by C++ and
> Java, I wonder if it wouldn't be better to support conversion to and
> from that format and nothing else.

By the way, this particular format is also recommended by the draft versions
of IEEE 754r that I've seen:  section 7.12.2 of draft version 1.2.5
(this is publicly
available---there's a link from the wikipedia 754r page) says:

"""Implementations supporting binary formats shall provide conversions
between all supported internal binary
formats and external hexadecimal character sequences. External
hexadecimal character sequences for finite
numbers are of the form specified by C99 subclauses:
6.4.4.2 floating constants,
20.1.3 strtod,
7.19.6.2 fscanf (a, e, f, g), and
7.19.6.1 fprintf (a, A)."""

More recent 754r drafts spell the grammar out explicitly instead of
referring to C99,
and weaken the 'shall' (i.e., 'is required to') to a 'should' ('is
recommended to').

Mark

From guido at python.org  Sat Jun 28 00:56:18 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Jun 2008 15:56:18 -0700
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
Message-ID: <ca471dc20806271556r6aa2874pa765d3e9396984f0@mail.gmail.com>

On Fri, Jun 27, 2008 at 2:23 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Fri, Jun 27, 2008 at 2:11 PM, Guido van Rossum <guido at python.org> wrote:
>> Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due
>> to the switch to the  new AST-based compiler. Can you file a bug? I
>> think we should leave 2.5 alone (too much risk of breaking code) but
>> fix it in 2.6 and 3.0 if we can.
>
> I think code that uses this is probably already quite broken in some
> fundamental way and putting the fix in 2.5 isn't much of a risk.

No, it could just be a harmless typo in a long argument list. I really
don't want to break code that is apparently working as part of an
upgrade from 2.5.2 to 2.5.3. I want people to be completely fearless
when it comes to such an upgrade: they should be able to just install
it without having to think about testing anything, just like most
people routinely install a new minor Linux upgrade pushed by their
distribution.

2.6 is a different story, everyone knows they have to do testing
before deciding it's safe to upgrade from 2.x to 2.(x+1).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Jun 28 00:56:46 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Jun 2008 15:56:46 -0700
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <48655932.2010203@holdenweb.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
	<48655932.2010203@holdenweb.com>
Message-ID: <ca471dc20806271556n49df5dc7kada1b06fe13980f5@mail.gmail.com>

Yes.

On Fri, Jun 27, 2008 at 2:18 PM, Steve Holden <steve at holdenweb.com> wrote:
> So we wait until they port their code to 2.6 to break it?
>
> regards
>  Steve
>
> Guido van Rossum wrote:
>>
>> Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due
>> to the switch to the  new AST-based compiler. Can you file a bug? I
>> think we should leave 2.5 alone (too much risk of breaking code) but
>> fix it in 2.6 and 3.0 if we can.
>>
>> --Guido
>>
>> On Fri, Jun 27, 2008 at 12:07 PM, tomer filiba <tomerfiliba at gmail.com>
>> wrote:
>>>
>>> the following code works on python 2.5:
>>>
>>>>>> def f(**kwargs):
>>>
>>> ...     print kwargs
>>> ...
>>>>>>
>>>>>> f(a=5,b=7,a=8)
>>>
>>> {'a': 8, 'b': 7}
>>> but fails on python2.4, saying that "a" is given twice.
>>> is this a bug or a feature?
>>>
>>>
>>> -tomer
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>>
>>
>>
>>
>
>
> --
> Steve Holden        +1 571 484 6266   +1 800 494 3119
> Holden Web LLC              http://www.holdenweb.com/
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Sat Jun 28 00:59:08 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 27 Jun 2008 18:59:08 -0400
Subject: [Python-Dev] [Python-checkins] r64424 - in
 python/trunk:Include/object.h Lib/test/test_sys.py
 Misc/NEWSObjects/intobject.c Objects/longobject.c
 Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<4863AA71.5020901@gmail.com>	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>	<4863B400.4090401@gmail.com>	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>	<48643E3F.3010500@canterbury.ac.nz>	<ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
	<5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com>
Message-ID: <g43rbo$tf2$1@ger.gmane.org>



Mark Dickinson wrote:

>>
>>>>> math.tohex(3.14)
>> '0x1.91eb851eb851fp+1'
>>>>> math.fromhex('0x1.91eb851eb851fp+1')
>> 3.1400000000000001

How about just one self-inverse method .hex?
.hex(float/hexstring) returns corresponding hexstring/float


From guido at python.org  Sat Jun 28 01:03:45 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Jun 2008 16:03:45 -0700
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <F14738BD-9AC7-4D25-B376-D81D5AC524E2@acm.org>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
	<F14738BD-9AC7-4D25-B376-D81D5AC524E2@acm.org>
Message-ID: <ca471dc20806271603k66463d18y3ae8b131d105adf4@mail.gmail.com>

On Fri, Jun 27, 2008 at 2:54 PM, Fred Drake <fdrake at acm.org> wrote:
> On Jun 27, 2008, at 5:23 PM, Benjamin Peterson wrote:
>>
>> I think code that uses this is probably already quite broken in some
>> fundamental way and putting the fix in 2.5 isn't much of a risk.
>
> I suspect the risk has more to do with breaking something else in Python
> than in breaking 3rd-party code in this case.
>
> I think it should be fixed for 2.5 as well, myself.

Let me clarify why I want to be so careful with this.

If there is code that was expected to work but due to a bug in our
code raises an exception, it's generally safe to fix this: people who
ran into the issue found that their code didn't work, used a
work-around, and that's the end of the story.

But if there is code that was expected to *break* but due to a bug in
our code *doesn't* raise an exception, people can very well have
harmless occurrences of such code, and never noticed. Maybe their code
is "broken" in the sense that it doesn't produce the correct result,
but it may well be in a "don't care" way -- but if an upgrade suddenly
starts raising the exception, they are likely to get unhandled
exceptions where before they had none. This is particularly annoying
when the author of the program that breaks is not the user of the
program, to whose machine the upgrade was applied.

In such cases I think it's better not to introduce new exceptions in
point-point releases.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Jun 28 01:04:44 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Jun 2008 16:04:44 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
	<48643E3F.3010500@canterbury.ac.nz>
	<ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
	<5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com>
Message-ID: <ca471dc20806271604r3127aee4r797f737330567070@mail.gmail.com>

On Fri, Jun 27, 2008 at 2:54 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Fri, Jun 27, 2008 at 8:02 PM, Guido van Rossum <guido at python.org> wrote:
>> Now that I've learned about the hex float format supported by C++ and
>> Java, I wonder if it wouldn't be better to support conversion to and
>> from that format and nothing else.
>>
>> E.g.
>>
>>>>> math.tohex(3.14)
>> '0x1.91eb851eb851fp+1'
>>>>> math.fromhex('0x1.91eb851eb851fp+1')
>> 3.1400000000000001
>
> This would certainly be enough for me, though I think there's still
> some educational value in having binary output available.  But
> that's just a matter of substituting a four-bit binary string for
> each hexadecimal digit (or learning to read hexadecimal as
> though it were binary).

If it's educational it can be left as an exercise for the reader. :-)

> In fromhex, what would be done with a string that gives more
> hex digits than the machine precision can support?  An obvious
> answer is just to round to the nearest float, but since part of the
> point of hex floats is having a way to specify a given value
> *exactly*, it might make more sense to raise an exception
> rather than changing the value by rounding it.

Whatever Java and C99 do.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Jun 28 01:06:01 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Jun 2008 16:06:01 -0700
Subject: [Python-Dev] [Python-checkins] r64424 - in
	python/trunk:Include/object.h Lib/test/test_sys.py
	Misc/NEWSObjects/intobject.c Objects/longobject.c
	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <g43rbo$tf2$1@ger.gmane.org>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>
	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>
	<48643E3F.3010500@canterbury.ac.nz>
	<ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
	<5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com>
	<g43rbo$tf2$1@ger.gmane.org>
Message-ID: <ca471dc20806271606n1d34ed55qed1afcfd9137408d@mail.gmail.com>

On Fri, Jun 27, 2008 at 3:59 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>
>
> Mark Dickinson wrote:
>
>>>
>>>>>> math.tohex(3.14)
>>>
>>> '0x1.91eb851eb851fp+1'
>>>>>>
>>>>>> math.fromhex('0x1.91eb851eb851fp+1')
>>>
>>> 3.1400000000000001
>
> How about just one self-inverse method .hex?
> .hex(float/hexstring) returns corresponding hexstring/float

That seems to be a misplaced attempt at economy, obscuring the intent
from the reader.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Sat Jun 28 01:08:13 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jun 2008 01:08:13 +0200
Subject: [Python-Dev] [Python-checkins] r64424 -
 in	python/trunk:Include/object.h
 Lib/test/test_sys.py	Misc/NEWSObjects/intobject.c
 Objects/longobject.c	Objects/typeobject.cPython/bltinmodule.c
In-Reply-To: <ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>	<FB295AFC531D4E9EA4CC470E1861A57E@RaymondLaptop1>	<4863AA71.5020901@gmail.com>	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>	<4863B400.4090401@gmail.com>	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>	<ca471dc20806261150p50e2d9abod1901e91c147bb9f@mail.gmail.com>	<6F8B0560F400452A934A174137F0F227@RaymondLaptop1>	<48643E3F.3010500@canterbury.ac.nz>
	<ca471dc20806271202s3e06ccc4leb7299a2ee1345a9@mail.gmail.com>
Message-ID: <486572DD.9080609@v.loewis.de>

> Now that I've learned about the hex float format supported by C++ and
> Java, I wonder if it wouldn't be better to support conversion to and
> from that format and nothing else.

I would be fine with that, and prefer it over the original change.

Regards,
Martin

From janssen at parc.com  Sat Jun 28 01:21:03 2008
From: janssen at parc.com (Bill Janssen)
Date: Fri, 27 Jun 2008 16:21:03 PDT
Subject: [Python-Dev] urllib,
	multipart/form-data encoding and file uploads
In-Reply-To: <7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com> 
References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>
	<4610679334594632865@unknownmsgid>
	<7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com>
Message-ID: <08Jun27.162111pdt."58698"@synergy1.parc.xerox.com>

All sounds reasonable to me.

Bill

> On Fri, Jun 27, 2008 at 11:40 AM, Bill Janssen <janssen at parc.com> wrote:
> >> I notice that there is some work being done on urllib / urllib2 for
> >> python 2.6/3.0.  One thing I've always missed in urllib/urllib2 is the
> >> facility to encode POST data as multipart/form-data.  I think it would
> >> also be useful to be able to stream a POST request to the remote
> >> server rather than having requiring the user to create the entire POST
> >> body in memory before starting the request.  This would be extremely
> >> useful when writing any kind of code that does file uploads.
> >>
> >> I didn't see any recent discussion about this so I thought I'd ask
> >> here: do you think this would make a good addition to the new urllib
> >> package?
> >
> > I think it would be very helpful.  I'd separate the two things,
> > though; you want to be able to format a set of values as
> > "multipart/form-data", and do various things with that resulting
> > "document", and you want to be able to stream a POST (or PUT) request.
> 
> How about if the function that encoded the values as "multipart/form-data"
> was able to stream data to a POST (or PUT) request via an iterator that
> yielded chunks of data?
> 
> def multipart_encode(params, boundary=None):
>     """Encode ``params`` as multipart/form-data.
> 
>     ``params`` should be a dictionary where the keys represent parameter names,
>     and the values are either parameter values, or file-like objects to
>     use as the parameter value.  The file-like object must support the .read(),
>     .seek(), and .tell() methods.
> 
>     If ``boundary`` is set, then it as used as the MIME boundary.  Otherwise
>     a randomly generated boundary will be used.  In either case, if the
>     boundary string appears in the parameter values a ValueError will be
>     raised.
> 
>     Returns an iterable object that will yield blocks of data representing
>     the encoded parameters."""
> 
> The file objects need to support .seek() and .tell() so we can determine
> how large they are before including them in the output.  I've been trying
> to come up with a good way to specify the size separately so you could use
> unseekable objects, but no good ideas have come to mind.  Maybe it could
> look for a 'size' attribute or callable on the object?  That seems a bit
> hacky...
> 
> A couple helper functions would be necessary as well, one to generate
> random boundary strings that are guaranteed not to collide with file data,
> and another function to calculate the total size of the encoding to be used
> in the 'Content-Length' header in the main HTTP request.
> 
> Then we'd need to change either urllib or httplib to support iterable
> objects in addition to the regular strings that it currently uses.
> 
> Cheers,
> Chris
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com


From jml at mumak.net  Sat Jun 28 02:19:25 2008
From: jml at mumak.net (Jonathan Lange)
Date: Sat, 28 Jun 2008 10:19:25 +1000
Subject: [Python-Dev] Disable tests in unittest (issue3202)
In-Reply-To: <20080626185727.GA29900@mit.edu>
References: <20080625211349.GA471@mit.edu>
	<d06a5cd30806251751ie8bc983w9aabb292ab903845@mail.gmail.com>
	<20080626185727.GA29900@mit.edu>
Message-ID: <d06a5cd30806271719x73c6819bnef767011adeb5731@mail.gmail.com>

On Fri, Jun 27, 2008 at 4:57 AM, Justin Mazzola Paluska <jmp at mit.edu> wrote:
> I wasn't aware of bzrlib's extentions to unittest ? many of them,
> including KnownFailure, TestSkipped,
> bzrlib.test.ExtendedTestResult.expectFailure, and the sundry list of
> bzrlib.test.TestCase.assert* look useful enough to belong in the
> standard library.
>

I agree 100%. Twisted came up with some of the same things
independently, and for a lot of things, we've stolen bzrlib's ideas.

I think I've filed a couple of tickets with patches along these lines.
I'll try to get my act in gear and finish the rest of the patches to
unittest and get them landed.

jml

From hall.jeff at gmail.com  Sat Jun 28 02:25:19 2008
From: hall.jeff at gmail.com (Jeff Hall)
Date: Fri, 27 Jun 2008 20:25:19 -0400
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <ca471dc20806271603k66463d18y3ae8b131d105adf4@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
	<F14738BD-9AC7-4D25-B376-D81D5AC524E2@acm.org>
	<ca471dc20806271603k66463d18y3ae8b131d105adf4@mail.gmail.com>
Message-ID: <1bc395c10806271725q6c766eabjeba27dcdc1660100@mail.gmail.com>

That's all fine and good but in this case there may be "stealth errors". If
the user/programmer is expecting the first value to hold but instead

On Fri, Jun 27, 2008 at 7:03 PM, Guido van Rossum <guido at python.org> wrote:

> On Fri, Jun 27, 2008 at 2:54 PM, Fred Drake <fdrake at acm.org> wrote:
> > On Jun 27, 2008, at 5:23 PM, Benjamin Peterson wrote:
> >>
> >> I think code that uses this is probably already quite broken in some
> >> fundamental way and putting the fix in 2.5 isn't much of a risk.
> >
> > I suspect the risk has more to do with breaking something else in Python
> > than in breaking 3rd-party code in this case.
> >
> > I think it should be fixed for 2.5 as well, myself.
>
> Let me clarify why I want to be so careful with this.
>
> If there is code that was expected to work but due to a bug in our
> code raises an exception, it's generally safe to fix this: people who
> ran into the issue found that their code didn't work, used a
> work-around, and that's the end of the story.
>
> But if there is code that was expected to *break* but due to a bug in
> our code *doesn't* raise an exception, people can very well have
> harmless occurrences of such code, and never noticed. Maybe their code
> is "broken" in the sense that it doesn't produce the correct result,
> but it may well be in a "don't care" way -- but if an upgrade suddenly
> starts raising the exception, they are likely to get unhandled
> exceptions where before they had none. This is particularly annoying
> when the author of the program that breaks is not the user of the
> program, to whose machine the upgrade was applied.
>
> In such cases I think it's better not to introduce new exceptions in
> point-point releases.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/>
> )
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/hall.jeff%40gmail.com
>



-- 
Haikus are easy
Most make very little sense
Refrigerator
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080627/8fec0a21/attachment.htm>

From hall.jeff at gmail.com  Sat Jun 28 02:26:52 2008
From: hall.jeff at gmail.com (Jeff Hall)
Date: Fri, 27 Jun 2008 20:26:52 -0400
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <1bc395c10806271725q6c766eabjeba27dcdc1660100@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
	<F14738BD-9AC7-4D25-B376-D81D5AC524E2@acm.org>
	<ca471dc20806271603k66463d18y3ae8b131d105adf4@mail.gmail.com>
	<1bc395c10806271725q6c766eabjeba27dcdc1660100@mail.gmail.com>
Message-ID: <1bc395c10806271726t7fda334l4e2285c1d32a8b7b@mail.gmail.com>

oops... baby jumped in my lap... i pretty much said it all though... I think
the error of the software functioning incorrectly may necessitate the
patch... I certainly understand Guido's concern, however.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20080627/7f073d8d/attachment.htm>

From ncoghlan at gmail.com  Sat Jun 28 03:04:57 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Jun 2008 11:04:57 +1000
Subject: [Python-Dev] Community buildbots and Python release
	quality	metrics
In-Reply-To: <ca471dc20806271110y1d2eb549uceb2ee6965a7a964@mail.gmail.com>
References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com>	<ca471dc20806260833td271366h4db6750e0528844c@mail.gmail.com>	<932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com>	<20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com>	<932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com>	<ca471dc20806261055g5219d34fi23b84f9462af8962@mail.gmail.com>	<932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com>	<ca471dc20806261145m164374bnb4dcdf770c2be0fa@mail.gmail.com>	<932f8baf0806261522m7b5eb75aiac986ea029cdd6ff@mail.gmail.com>
	<ca471dc20806271110y1d2eb549uceb2ee6965a7a964@mail.gmail.com>
Message-ID: <48658E39.9040700@gmail.com>

Guido van Rossum wrote:
> On Thu, Jun 26, 2008 at 3:22 PM, Ralf Schmitt <schmir at gmail.com> wrote:
>> On Thu, Jun 26, 2008 at 8:45 PM, Guido van Rossum <guido at python.org> wrote:
>>> So you're saying py.test needs to be fixed? Fine with me, but then I
>>> don't understand why you bothered bringing it up here instead of
>>> fixing it (or reporting to the py.test maintainers, I don't know if
>>> you're one or not).
>>>
>> no, I'm not. Just wanted to point out, that this ticket/patch does not
>> solve the django issue.
>> (this is what I first thought it would do).
> 
> I'm still missing details.
> 

I'm going to spend some time looking into this this weekend (issue 
assigned appropriately. Getting it working again shouldn't be too bad 
(just revert the relevants bits to their 2.5 equivalents) - the tricky 
part is going to be keeping the warning of the associated change to the 
semantics in 3.0.

(I'll also take a look at 3.0 to make sure the relevant test case from 
the tracker issue works properly there)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sat Jun 28 03:06:45 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Jun 2008 11:06:45 +1000
Subject: [Python-Dev] urllib,
	multipart/form-data encoding and file uploads
In-Reply-To: <7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com>
References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>	<4610679334594632865@unknownmsgid>
	<7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com>
Message-ID: <48658EA5.4000803@gmail.com>

Chris AtLee wrote:
> Then we'd need to change either urllib or httplib to support iterable
> objects in addition to the regular strings that it currently uses.

Chris,

To avoid losing these ideas, could you add them to the issue tracker as 
feature requests? It's too late to get them into 2.6/3.0 but they may 
make good additions for the next release cycle.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From greg.ewing at canterbury.ac.nz  Sat Jun 28 03:17:10 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 28 Jun 2008 13:17:10 +1200
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
Message-ID: <48659116.10302@canterbury.ac.nz>

tomer filiba wrote:

>  >>> def f(**kwargs):
> ...     print kwargs
> ...
>  >>> f(a=5,b=7,a=8)
> {'a': 8, 'b': 7}
>  >>>

I can't think of any reason why one would need to be
able to write such code, or even want to.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Jun 28 03:36:05 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 28 Jun 2008 13:36:05 +1200
Subject: [Python-Dev] urllib,
	multipart/form-data encoding and file uploads
In-Reply-To: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>
References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>
Message-ID: <48659585.8030100@canterbury.ac.nz>

Chris AtLee wrote:
> One thing I've always missed in urllib/urllib2 is the
> facility to encode POST data as multipart/form-data.

I second that, having had to reinvent it a couple of
times recently. It seems like an obvious thing to want
to do, and it's surprising to find it's not supported.

-- 
Greg

From steve at pearwood.info  Sat Jun 28 03:58:41 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 28 Jun 2008 11:58:41 +1000
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <48659116.10302@canterbury.ac.nz>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<48659116.10302@canterbury.ac.nz>
Message-ID: <200806281158.41558.steve@pearwood.info>

On Sat, 28 Jun 2008 11:17:10 am Greg Ewing wrote:
> tomer filiba wrote:
> >  >>> def f(**kwargs):
> >
> > ...     print kwargs
> > ...
> >
> >  >>> f(a=5,b=7,a=8)
> >
> > {'a': 8, 'b': 7}
>
> I can't think of any reason why one would need to be
> able to write such code, or even want to.

It would be nice to be able to do this:

defaults = dict(a=5, b=7)
f(**defaults, a=8)  # override the value of a in defaults

but unfortunately that gives a syntax error. Reversing the order would 
override the wrong value. So as Python exists now, no, it's not 
terribly useful. But it's not inherently a stupid idea.


-- 
Steven

From scott+python-dev at scottdial.com  Sat Jun 28 04:11:03 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Fri, 27 Jun 2008 22:11:03 -0400
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <200806281158.41558.steve@pearwood.info>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>	<48659116.10302@canterbury.ac.nz>
	<200806281158.41558.steve@pearwood.info>
Message-ID: <48659DB7.4090008@scottdial.com>

Steven D'Aprano wrote:
> It would be nice to be able to do this:
> 
> defaults = dict(a=5, b=7)
> f(**defaults, a=8)  # override the value of a in defaults

I can't help but think that would be difficult coding convention to use. 
However, I'm considerably less bothered by:

def f_with_defaults(**kw):
     defaults = dict(a=5, b=7)
     defaults.update(kw)
     return f(**defaults)
f_with_default(a=8)

The only way f(**defaults, a=8) would be useful is if it happens a lot, 
and in that case, it's just as good of a candidate for being made into a 
function itself. I see this pattern all the time, and given that 
"f_with_defaults" probably has some semantic meaning, it probably would 
be nice to give it it's own name ("g").

Just my 2 cents.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From martin at v.loewis.de  Sat Jun 28 09:47:16 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jun 2008 09:47:16 +0200
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <1bc395c10806271725q6c766eabjeba27dcdc1660100@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>	<F14738BD-9AC7-4D25-B376-D81D5AC524E2@acm.org>	<ca471dc20806271603k66463d18y3ae8b131d105adf4@mail.gmail.com>
	<1bc395c10806271725q6c766eabjeba27dcdc1660100@mail.gmail.com>
Message-ID: <4865EC84.5050706@v.loewis.de>

Jeff Hall wrote:
> That's all fine and good but in this case there may be "stealth errors".

That is fully understood, in all of its consequences.

Regards,
Martin

From martin at v.loewis.de  Sat Jun 28 10:14:29 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jun 2008 10:14:29 +0200
Subject: [Python-Dev] urllib,
	multipart/form-data encoding and file uploads
In-Reply-To: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>
References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>
Message-ID: <4865F2E5.4000506@v.loewis.de>

> I didn't see any recent discussion about this so I thought I'd ask
> here: do you think this would make a good addition to the new urllib
> package?

Just in case that isn't clear: any such change must be delayed for
2.7/3.1. That is not to say that you couldn't start implementing it
now, of course.

Regards,
Martin

From tomerfiliba at gmail.com  Sat Jun 28 10:30:38 2008
From: tomerfiliba at gmail.com (tomer filiba)
Date: Sat, 28 Jun 2008 01:30:38 -0700 (PDT)
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <ca471dc20806271556r6aa2874pa765d3e9396984f0@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> 
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com> 
	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> 
	<ca471dc20806271556r6aa2874pa765d3e9396984f0@mail.gmail.com>
Message-ID: <a5096c49-a302-44f7-8a3d-4172b0541e32@x41g2000hsb.googlegroups.com>

On Jun 28, 12:56?am, "Guido van Rossum" <gu... at python.org> wrote:
> No, it could just be a harmless typo in a long argument list.

to elaborate on this point a little, i came across this error when i
ported my code to 2.4. i used the optparse class which takes 10's of
kwargs, and it turned out i'd given the same parameter twice (the
infamous copy-paste problem).

so on the one hand, it was a harmless typo (because the latest
instance was taken). on the other hand, it's a good thing i tested it
on 2.4, or i'd never notice the repeated argument, which may have led
to weird runtime errors (if the order of the params was changed).

i'd be in favor of fixing this in 2.5, just to eliminate possibly hard-
to-debug runtime errors. since it's a syntax error, it would be early-
noticed when the code is first run/imported, and it wouldn't require
the original author of the code to fix.


-tomer

From ncoghlan at gmail.com  Sat Jun 28 10:32:09 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Jun 2008 18:32:09 +1000
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <ca471dc20806271603k66463d18y3ae8b131d105adf4@mail.gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>	<F14738BD-9AC7-4D25-B376-D81D5AC524E2@acm.org>
	<ca471dc20806271603k66463d18y3ae8b131d105adf4@mail.gmail.com>
Message-ID: <4865F709.4010901@gmail.com>

Guido van Rossum wrote:
> In such cases I think it's better not to introduce new exceptions in
> point-point releases.

Perhaps they should be backported to the maintenance as warnings? Then 
users can decide on a case-by-case basis if they want to make that 
particularly warning trigger an exception.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From martin at v.loewis.de  Sat Jun 28 11:23:28 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jun 2008 11:23:28 +0200
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <a5096c49-a302-44f7-8a3d-4172b0541e32@x41g2000hsb.googlegroups.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
	<ca471dc20806271556r6aa2874pa765d3e9396984f0@mail.gmail.com>
	<a5096c49-a302-44f7-8a3d-4172b0541e32@x41g2000hsb.googlegroups.com>
Message-ID: <48660310.1080307@v.loewis.de>

> i'd be in favor of fixing this in 2.5, just to eliminate possibly hard-
> to-debug runtime errors. since it's a syntax error, it would be early-
> noticed when the code is first run/imported, and it wouldn't require
> the original author of the code to fix.

As release manager for Python 2.5, I'd like to support Guido's position:
the risk of breaking existing code is just not worth it. Developers who
made such a mistake will find out when they port the code to 2.6; there
is no value whatsoever in end-users finding out minor bugs in software
they didn't even know was written in Python.

Regards,
Martin

From martin at v.loewis.de  Sat Jun 28 11:24:43 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jun 2008 11:24:43 +0200
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <4865F709.4010901@gmail.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>	<F14738BD-9AC7-4D25-B376-D81D5AC524E2@acm.org>	<ca471dc20806271603k66463d18y3ae8b131d105adf4@mail.gmail.com>
	<4865F709.4010901@gmail.com>
Message-ID: <4866035B.4030706@v.loewis.de>

> Perhaps they should be backported to the maintenance as warnings? Then
> users can decide on a case-by-case basis if they want to make that
> particularly warning trigger an exception.

No. There will likely be one more 2.5 release. This entire issue never
came up in the lifetime of 2.5, so it can't be so serious to annoy
end-users with a warning they don't know how to deal with.

Regards,
Martin

From Roger.Wenham at gmx.net  Sat Jun 28 14:59:50 2008
From: Roger.Wenham at gmx.net (Roger wenham)
Date: Sat, 28 Jun 2008 14:59:50 +0200
Subject: [Python-Dev] Python 2.6 SSL
Message-ID: <20080628125950.189860@gmx.net>

HI,

I have used the SSL lib to wrap sockets in existing client server code, but I find that
the max send size is 16K. In other words I send 20K, the write call returns 20K
but the receiving end only gets 16K.

I remove the wraper and everything worksagain.

I even modified the unit test to send >16K, but got the same result.

Is therea patch for this ?

I hope this the correct place to report this...

Regards

Roger


-- 
Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! 
Ideal f?r Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

From eyal.lotem at gmail.com  Sat Jun 28 15:40:04 2008
From: eyal.lotem at gmail.com (Eyal Lotem)
Date: Sat, 28 Jun 2008 16:40:04 +0300
Subject: [Python-Dev] Cycle collection enhancement idea
Message-ID: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>

I see why a cycle that has multiple objects with a __del__ method is a problem.

Once you call __del__ on one of the objects, its no longer usable by
the others, and its not clear which order is correct.

My question regards the case where a cycle of objects only has 1
object which has a __del__.

I think a correct strategy to collect the entire cycle is the same one
used on a single object.  On a single object Python uses:
1. Temporarily revive object
2. Call __del__
3. Unrevive object (if(--refcount == 0) then we're done), otherwise it
was resurrected).

We can apply this to the whole cycle:
1. Temporarily revive entire cycle (each of its objects)
2. Call __del__
3. Unrevive the objects of the entire cycle (each of its objects).

Step 1 will allow __del__ to run safely. Since there is only one
__del__ in the cycle, it is not dangerous that its references will
disappear from "under its feet".
(Some code restructuring will probably be necessary, because of
assumptions that are hard-coded into slot_tp_del and subtype_dealloc).

I believe this enhancement is important, because:
A. When using existing code -- you do not control whether its objects
have a __del__. In my experience, a majority of these cases only have
a single __del__-containing object in their cycles.
B. Python's exit cleanup calls __del__ in the wrong order, and
Python's runtime is full of cycles (Each global is a cycle, including
the class objects themselves: class->dict->function->func_globals)).
These cycles very often have only 1 __del__ method.

Some examples of the problem posed by B:
http://www.google.com/search?q=ignored+%22%27NoneType%27+object+has+no+attribute%22+%22__del__+of%22&btnG=Search
Ugly workarounds exist even in the standard library [subprocess]: "def
__del__(self, sys=sys):").

Example:

import os
class RunningFile(object):
    filename = '/tmp/running'
    def __init__(self):
        open(self.filename, 'wb')
    def __del__(self):
        os.unlink(self.filename)
running_file = RunningFile()

The deller object is in a cycle as described above [as well as the
Deller class itself].  When Python exits, it could call
deller.__del__() and then collect the cycle. But Python does the wrong
thing here, and gets rid of the globals before calling __del__:
Exception exceptions.AttributeError: "'NoneType' object has no
attribute 'unlink'" in <bound method RunningFile.__del__ of
<__main__.RunningFile object at 0x7f9655eb92d0>> ignored

I believe applying the above enhancement would solve these problems.

From ncoghlan at gmail.com  Sat Jun 28 17:21:19 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 29 Jun 2008 01:21:19 +1000
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
Message-ID: <486656EF.1080209@gmail.com>

Eyal Lotem wrote:
> Example:
> 
> import os
> class RunningFile(object):
>     filename = '/tmp/running'
>     def __init__(self):
>         open(self.filename, 'wb')
>     def __del__(self):
>         os.unlink(self.filename)
> running_file = RunningFile()
> 
> The deller object is in a cycle as described above [as well as the
> Deller class itself].  When Python exits, it could call
> deller.__del__() and then collect the cycle. But Python does the wrong
> thing here, and gets rid of the globals before calling __del__:
> Exception exceptions.AttributeError: "'NoneType' object has no
> attribute 'unlink'" in <bound method RunningFile.__del__ of
> <__main__.RunningFile object at 0x7f9655eb92d0>> ignored

I don't know what you're trying to get at with this example. There isn't 
any cyclic GC involved at all, just referencing counting. And before the 
module globals are cleared, running_file is still referenced, so calling 
its __del__ method early would be an outright error in the interpreter 
(as far as I know, getting __del__ methods to run is one of the 
*reasons* for clearing the module globals).

It's a fact of Python development: __del__ methods cannot safely 
reference module globals, because those globals may be gone by the time 
that method is invoked.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Sat Jun 28 17:31:08 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Jun 2008 08:31:08 -0700
Subject: [Python-Dev] Python 2.6 SSL
In-Reply-To: <20080628125950.189860@gmx.net>
References: <20080628125950.189860@gmx.net>
Message-ID: <ca471dc20806280831u10b8867we52ae8b8b06dcf1a@mail.gmail.com>

Roger, can you show us the relevant code?

On Sat, Jun 28, 2008 at 5:59 AM, Roger wenham <Roger.Wenham at gmx.net> wrote:
> HI,
>
> I have used the SSL lib to wrap sockets in existing client server code, but I find that
> the max send size is 16K. In other words I send 20K, the write call returns 20K
> but the receiving end only gets 16K.
>
> I remove the wraper and everything worksagain.
>
> I even modified the unit test to send >16K, but got the same result.
>
> Is therea patch for this ?
>
> I hope this the correct place to report this...
>
> Regards
>
> Roger
>
>
> --
> Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
> Ideal f?r Modem und ISDN: http://www.gmx.net/de/go/smartsurfer
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Sat Jun 28 17:32:08 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jun 2008 17:32:08 +0200
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
Message-ID: <48665978.3020605@v.loewis.de>

> Example:
> 
> import os
> class RunningFile(object):
>     filename = '/tmp/running'
>     def __init__(self):
>         open(self.filename, 'wb')
>     def __del__(self):
>         os.unlink(self.filename)
> running_file = RunningFile()
> 
> The deller object is in a cycle as described above [as well as the
> Deller class itself].

I think you are mistaken here. The RunningFile instance in above code
is *not* part of a cycle. It doesn't have any instance variables (i.e.
its __dict__ is empty), and it only refers to its class, which (AFAICT)
doesn't refer back to the instance.

> When Python exits, it could call
> deller.__del__() and then collect the cycle. But Python does the wrong
> thing here, and gets rid of the globals before calling __del__:
> Exception exceptions.AttributeError: "'NoneType' object has no
> attribute 'unlink'" in <bound method RunningFile.__del__ of
> <__main__.RunningFile object at 0x7f9655eb92d0>> ignored

This is a different issue. For shutdown, Python doesn't rely on
cyclic garbage collection (only). Instead, all modules get forcefully
cleared, causing this problem.

> I believe applying the above enhancement would solve these problems.

No, they wouldn't.

To work around the real problem in your case, put everything that the
destructor uses into an instance or class attribute:

class RunningFile(object):
    filename = '/tmp/running'
    _unlink = os.unlink
    def __init__(self):
        open(self.filename, 'wb')
    def __del__(self):
        self._unlink(self.filename)

Regards,
Martin

From guido at python.org  Sat Jun 28 17:38:41 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Jun 2008 08:38:41 -0700
Subject: [Python-Dev] repeated keyword arguments
In-Reply-To: <a5096c49-a302-44f7-8a3d-4172b0541e32@x41g2000hsb.googlegroups.com>
References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com>
	<ca471dc20806271211w500e217bsc0e1b2d58d9d56a0@mail.gmail.com>
	<1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com>
	<ca471dc20806271556r6aa2874pa765d3e9396984f0@mail.gmail.com>
	<a5096c49-a302-44f7-8a3d-4172b0541e32@x41g2000hsb.googlegroups.com>
Message-ID: <ca471dc20806280838i63af6427lc03c7d013cce8b14@mail.gmail.com>

On Sat, Jun 28, 2008 at 1:30 AM, tomer filiba <tomerfiliba at gmail.com> wrote:
> On Jun 28, 12:56 am, "Guido van Rossum" <gu... at python.org> wrote:
>> No, it could just be a harmless typo in a long argument list.
>
> to elaborate on this point a little, i came across this error when i
> ported my code to 2.4. i used the optparse class which takes 10's of
> kwargs, and it turned out i'd given the same parameter twice (the
> infamous copy-paste problem).
>
> so on the one hand, it was a harmless typo (because the latest
> instance was taken). on the other hand, it's a good thing i tested it
> on 2.4, or i'd never notice the repeated argument, which may have led
> to weird runtime errors (if the order of the params was changed).
>
> i'd be in favor of fixing this in 2.5, just to eliminate possibly hard-
> to-debug runtime errors. since it's a syntax error, it would be early-
> noticed when the code is first run/imported, and it wouldn't require
> the original author of the code to fix.

But your anecdote is exactly why I don't want it fixed in 2.5. Your
code was working, the typo was harmless. I don't want upgrades from
2.5.2 to 2.5.3 (or any x.y.z to x.y.(z+1)) to break code that was
working before the upgrade! This is a principle we've adopted for such
point-point upgrades for a long time. Also note that it took a long
time before this was first reported, so it's not exactly like it's an
important or frequent problem.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Sat Jun 28 18:57:04 2008
From: janssen at parc.com (Bill Janssen)
Date: Sat, 28 Jun 2008 09:57:04 PDT
Subject: [Python-Dev] Python 2.6 SSL
In-Reply-To: <20080628125950.189860@gmx.net> 
References: <20080628125950.189860@gmx.net>
Message-ID: <08Jun28.095712pdt."58698"@synergy1.parc.xerox.com>

> I hope this the correct place to report this...

Hi, Roger.

Please file a bug report at http://bugs.python.org/, and assign it to
me.  Please attach a patch for the change you made to the unit test
suite to send >16K.

Thanks!

Bill

From solipsis at pitrou.net  Sat Jun 28 20:47:54 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 28 Jun 2008 18:47:54 +0000 (UTC)
Subject: [Python-Dev] GC Proposal
References: <aac2c7cb0806261753xe16ca0dt7624c9a2c458edd@mail.gmail.com>
Message-ID: <loom.20080628T184253-31@post.gmane.org>

Adam Olsen <rhamph <at> gmail.com> writes:
> 
> We need two counters: one is the total number of traceable objects
> (those we would inspect if we did a full collection) and a number of
> "pending" trace operations.  Every time an object is moved into the
> last generation, we increase "pending" by two - once for itself and
> once for an older object.  Once pending equals the total number of
> traceable objects we do a full collection (and reset "pending" to 0).

It sounds rather similar to Martin's proposal, except with different
coefficients and slightly different definitions (but the "total number
of traceable objects" should be roughly equal to the number of objects
in the oldest generation, and the "number of pending trace operations"
roughly equal to the number of survivor objects after a collection of
the middle generation).

Am I missing something?



From martin at v.loewis.de  Sat Jun 28 21:50:20 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jun 2008 21:50:20 +0200
Subject: [Python-Dev] GC Proposal
In-Reply-To: <loom.20080628T184253-31@post.gmane.org>
References: <aac2c7cb0806261753xe16ca0dt7624c9a2c458edd@mail.gmail.com>
	<loom.20080628T184253-31@post.gmane.org>
Message-ID: <486695FC.2090407@v.loewis.de>

> It sounds rather similar to Martin's proposal, except with different
> coefficients and slightly different definitions (but the "total number
> of traceable objects" should be roughly equal to the number of objects
> in the oldest generation, and the "number of pending trace operations"
> roughly equal to the number of survivor objects after a collection of
> the middle generation).
> 
> Am I missing something?

I think that's an accurate description. I think the major difference
is the factor, and I think making it 100% (in my terminology) might
hold up collections for too long, in some cases. Whether or not 10%
is the "best" factor, I don't know (it most likely is not).

Regards,
Martin

From rhamph at gmail.com  Sat Jun 28 21:59:17 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 28 Jun 2008 13:59:17 -0600
Subject: [Python-Dev] GC Proposal
In-Reply-To: <loom.20080628T184253-31@post.gmane.org>
References: <aac2c7cb0806261753xe16ca0dt7624c9a2c458edd@mail.gmail.com>
	<loom.20080628T184253-31@post.gmane.org>
Message-ID: <aac2c7cb0806281259q5e4e60a1pb338b81a290ac18f@mail.gmail.com>

On Sat, Jun 28, 2008 at 12:47 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Adam Olsen <rhamph <at> gmail.com> writes:
>>
>> We need two counters: one is the total number of traceable objects
>> (those we would inspect if we did a full collection) and a number of
>> "pending" trace operations.  Every time an object is moved into the
>> last generation, we increase "pending" by two - once for itself and
>> once for an older object.  Once pending equals the total number of
>> traceable objects we do a full collection (and reset "pending" to 0).
>
> It sounds rather similar to Martin's proposal, except with different
> coefficients and slightly different definitions (but the "total number
> of traceable objects" should be roughly equal to the number of objects
> in the oldest generation, and the "number of pending trace operations"
> roughly equal to the number of survivor objects after a collection of
> the middle generation).

The effect is similar for the "batch allocation" case, but opposite
for the "long-running program" case.  Which is preferred is
debatable.. If we had an incremental GC mine wouldn't have any bad
cases, just the constant overhead.  However, lacking an incremental
GC, and since refcounting GC is sufficient for most cases, we might
prefer to save overhead and avoid the pauses than to handle the
"long-running program" case.

My proposal can be made equivalent to Martin's proposal by removing
all of its pending traces when an untraced object is deleted.  We
could even change this at runtime, by adding a counter for pending
objects.

Come to think of it, I think Martin's proposal needs to be implemented
as mine.  He wants the middle generation to be 10% larger than the
oldest generation, but to find out the size you need to either iterate
it (reintroducing the original problem), or keep some counters.  With
counters, his middle generation size is my "pending traces".


> Am I missing something?

Actually, I was.  I lost track of Martin's thread when preparing my idea.  Doh!


-- 
Adam Olsen, aka Rhamphoryncus

From martin at v.loewis.de  Sat Jun 28 22:42:11 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jun 2008 22:42:11 +0200
Subject: [Python-Dev] GC Proposal
In-Reply-To: <aac2c7cb0806281259q5e4e60a1pb338b81a290ac18f@mail.gmail.com>
References: <aac2c7cb0806261753xe16ca0dt7624c9a2c458edd@mail.gmail.com>	<loom.20080628T184253-31@post.gmane.org>
	<aac2c7cb0806281259q5e4e60a1pb338b81a290ac18f@mail.gmail.com>
Message-ID: <4866A223.8040902@v.loewis.de>

> The effect is similar for the "batch allocation" case, but opposite
> for the "long-running program" case.

I don't understand. Where is the difference?

> My proposal can be made equivalent to Martin's proposal by removing
> all of its pending traces when an untraced object is deleted.  We
> could even change this at runtime, by adding a counter for pending
> objects.

What is a "pending trace"?

> Come to think of it, I think Martin's proposal needs to be implemented
> as mine.  He wants the middle generation to be 10% larger than the
> oldest generation

Not exactly: 10% of the oldest generation, not 10% larger. So if the
oldest generation has 1,000,000 objects, I want to collect when the
survivors from the middle generation reach 100,000, not when they reach
1,100,000.

> but to find out the size you need to either iterate
> it (reintroducing the original problem), or keep some counters.  With
> counters, his middle generation size is my "pending traces".

Yes, I have two counters: one for the number of objects in the oldest
generation (established at the last collection, and unchanged
afterwards), and one for the number of survivors from the middle
collection (increased every time objects move to the oldest
generation).

So it seems there are minor difference (such as whether a counter
for the total number of traceable objects is maintained, which you
seem to be suggesting), but otherwise, I still think the proposals
are essentially the same.

Regards,
Martin


From python at rcn.com  Sun Jun 29 01:46:46 2008
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 28 Jun 2008 16:46:46 -0700
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
References: <20080620041816.4D5E81E4002@bag.python.org>
	<328D5F1E-E082-4002-9A65-38A31625C161@python.org>
	<4863B400.4090401@gmail.com>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
	<5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>
Message-ID: <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1>

From: "Mark Dickinson" <dickinsm at gmail.com>
> There's one other major difference between the C99 notation and the
> current patch:  the C99 notation includes a (hexa)decimal point.  The
> advantages of this include:
> 
> - the exponent gives a rough idea of the magnitude of the number, and
> - the exponent doesn't vary with changes to the least significant bits
>   of the float.

Is everyone agreed on a tohex/fromhex pair using the C99 notation as recommended in 754R?

Are you thinking of math module functions or as a method and classmethod on floats?


Raymond

From greg.ewing at canterbury.ac.nz  Sun Jun 29 02:39:11 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 29 Jun 2008 12:39:11 +1200
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <486656EF.1080209@gmail.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com>
Message-ID: <4866D9AF.1050508@canterbury.ac.nz>

Nick Coghlan wrote:

> It's a fact of Python development: __del__ methods cannot safely 
> reference module globals, because those globals may be gone by the time 
> that method is invoked.

Speaking of this, has there been any more thought given
to the idea of dropping the module clearing and just
relying on cyclic GC?

-- 
Greg

From aleaxit at gmail.com  Sun Jun 29 04:12:49 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Sat, 28 Jun 2008 19:12:49 -0700
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
In-Reply-To: <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
	<5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>
	<58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1>
Message-ID: <e8a0972d0806281912w4328c55tfc55c5a298c3333c@mail.gmail.com>

On Sat, Jun 28, 2008 at 4:46 PM, Raymond Hettinger <python at rcn.com> wrote:
> From: "Mark Dickinson" <dickinsm at gmail.com>
>>
>> There's one other major difference between the C99 notation and the
>> current patch:  the C99 notation includes a (hexa)decimal point.  The
>> advantages of this include:
>>
>> - the exponent gives a rough idea of the magnitude of the number, and
>> - the exponent doesn't vary with changes to the least significant bits
>>  of the float.
>
> Is everyone agreed on a tohex/fromhex pair using the C99 notation as
> recommended in 754R?

Dunno about everyone, but I'm +1 on that.


> Are you thinking of math module functions or as a method and classmethod on
> floats?

I'd prefer math modules functions.


Alex

From rhamph at gmail.com  Sun Jun 29 04:31:37 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 28 Jun 2008 20:31:37 -0600
Subject: [Python-Dev] GC Proposal
In-Reply-To: <4866A223.8040902@v.loewis.de>
References: <aac2c7cb0806261753xe16ca0dt7624c9a2c458edd@mail.gmail.com>
	<loom.20080628T184253-31@post.gmane.org>
	<aac2c7cb0806281259q5e4e60a1pb338b81a290ac18f@mail.gmail.com>
	<4866A223.8040902@v.loewis.de>
Message-ID: <aac2c7cb0806281931h52138888pcbaf3774332cd786@mail.gmail.com>

On Sat, Jun 28, 2008 at 2:42 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> The effect is similar for the "batch allocation" case, but opposite
>> for the "long-running program" case.
>
> I don't understand. Where is the difference?
>
>> My proposal can be made equivalent to Martin's proposal by removing
>> all of its pending traces when an untraced object is deleted.  We
>> could even change this at runtime, by adding a counter for pending
>> objects.
>
> What is a "pending trace"?
>
>> Come to think of it, I think Martin's proposal needs to be implemented
>> as mine.  He wants the middle generation to be 10% larger than the
>> oldest generation
>
> Not exactly: 10% of the oldest generation, not 10% larger. So if the
> oldest generation has 1,000,000 objects, I want to collect when the
> survivors from the middle generation reach 100,000, not when they reach
> 1,100,000.
>
>> but to find out the size you need to either iterate
>> it (reintroducing the original problem), or keep some counters.  With
>> counters, his middle generation size is my "pending traces".
>
> Yes, I have two counters: one for the number of objects in the oldest
> generation (established at the last collection, and unchanged
> afterwards), and one for the number of survivors from the middle
> collection (increased every time objects move to the oldest
> generation).
>
> So it seems there are minor difference (such as whether a counter
> for the total number of traceable objects is maintained, which you
> seem to be suggesting), but otherwise, I still think the proposals
> are essentially the same.

They are definitely quite close to equivalent.  The terminology
doesn't quite match up, so let me rearrange things and compare:

old * 0.1 <= survivors    # Martin
old <= survivors * 2.0    # Adam

Looks about equivalent, but "survivors" may mean two different things
depending on if it removes deleted survivors or not.  Splitting that
up, we get this form:

old <= survivors * 2.0 + deleted * 1.0

The deleted count ensures stable memory loads will still eventually
cause full collections.  Since our GC isn't
incremental/concurrent/realtime, we'd probably don't want the full
collection pauses except from big bursts, which is trivially done by
making the deleted factor 0.0.  My original proposal was assuming a
non-zero deleted factor, while yours (and the existing codebase)
assumed it'd be zero - this is how our two proposals differed.

(My "pending traces" is merely a running total of survivors * 2.0 +
deleted * 1.0.  It looks much easier to keep separate counts though.)

-- 
Adam Olsen, aka Rhamphoryncus

From guido at python.org  Sun Jun 29 06:52:22 2008
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Jun 2008 21:52:22 -0700
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <4866D9AF.1050508@canterbury.ac.nz>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz>
Message-ID: <ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>

On Sat, Jun 28, 2008 at 5:39 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
>
>> It's a fact of Python development: __del__ methods cannot safely reference
>> module globals, because those globals may be gone by the time that method is
>> invoked.
>
> Speaking of this, has there been any more thought given
> to the idea of dropping the module clearing and just
> relying on cyclic GC?

No, but it is an intriguing thought nevertheless. The module clearing
causes nothing but trouble...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Sun Jun 29 08:59:32 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jun 2008 08:59:32 +0200
Subject: [Python-Dev] GC Proposal
In-Reply-To: <aac2c7cb0806281931h52138888pcbaf3774332cd786@mail.gmail.com>
References: <aac2c7cb0806261753xe16ca0dt7624c9a2c458edd@mail.gmail.com>	
	<loom.20080628T184253-31@post.gmane.org>	
	<aac2c7cb0806281259q5e4e60a1pb338b81a290ac18f@mail.gmail.com>	
	<4866A223.8040902@v.loewis.de>
	<aac2c7cb0806281931h52138888pcbaf3774332cd786@mail.gmail.com>
Message-ID: <486732D4.3020004@v.loewis.de>

> Looks about equivalent, but "survivors" may mean two different things
> depending on if it removes deleted survivors or not.  Splitting that
> up, we get this form:
> 
> old <= survivors * 2.0 + deleted * 1.0

What precisely would be the "deleted" count? If it counts deallocations,
is it relevant what generation the deallocated object was from?
If so, how do you determine the generation? If not, wouldn't

while 1:
  x=[]

trigger a full garbage collection fairly quickly?

Regards,
Martin

From eyal.lotem at gmail.com  Sun Jun 29 13:26:08 2008
From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com)
Date: Sun, 29 Jun 2008 04:26:08 -0700 (PDT)
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <486656EF.1080209@gmail.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com> 
	<486656EF.1080209@gmail.com>
Message-ID: <6a7bef08-bf66-4e19-959b-625ab1bcb013@z66g2000hsc.googlegroups.com>

On Jun 28, 6:21?pm, Nick Coghlan <ncogh... at gmail.com> wrote:
> Eyal Lotem wrote:
> > Example:
>
> > import os
> > class RunningFile(object):
> > ? ? filename = '/tmp/running'
> > ? ? def __init__(self):
> > ? ? ? ? open(self.filename, 'wb')
> > ? ? def __del__(self):
> > ? ? ? ? os.unlink(self.filename)
> > running_file = RunningFile()
>
> > The deller object is in a cycle as described above [as well as the
> > Deller class itself]. ?When Python exits, it could call
> > deller.__del__() and then collect the cycle. But Python does the wrong
> > thing here, and gets rid of the globals before calling __del__:
> > Exception exceptions.AttributeError: "'NoneType' object has no
> > attribute 'unlink'" in <bound method RunningFile.__del__ of
> > <__main__.RunningFile object at 0x7f9655eb92d0>> ignored
>
> I don't know what you're trying to get at with this example. There isn't
> any cyclic GC involved at all, just referencing counting. And before the
> module globals are cleared, running_file is still referenced, so calling
> its __del__ method early would be an outright error in the interpreter
> (as far as I know, getting __del__ methods to run is one of the
> *reasons* for clearing the module globals).
>
> It's a fact of Python development: __del__ methods cannot safely
> reference module globals, because those globals may be gone by the time
> that method is invoked.
That's because globals are being cleaned up in an ad-hoc manner,
instead of just using cyclic GC that supports cycles.

>
> Cheers,
> Nick.
>
> --
> Nick Coghlan ? | ? ncogh... at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
> ? ? ? ? ? ? ?http://www.boredomandlaziness.org
> _______________________________________________
> Python-Dev mailing list
> Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...

From eyal.lotem at gmail.com  Sun Jun 29 13:30:09 2008
From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com)
Date: Sun, 29 Jun 2008 04:30:09 -0700 (PDT)
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <48665978.3020605@v.loewis.de>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com> 
	<48665978.3020605@v.loewis.de>
Message-ID: <11765c26-7b07-4387-9a40-bf11aa7b89d2@a70g2000hsh.googlegroups.com>

On Jun 28, 6:32?pm, "Martin v. L?wis" <mar... at v.loewis.de> wrote:
> > Example:
>
> > import os
> > class RunningFile(object):
> > ? ? filename = '/tmp/running'
> > ? ? def __init__(self):
> > ? ? ? ? open(self.filename, 'wb')
> > ? ? def __del__(self):
> > ? ? ? ? os.unlink(self.filename)
> > running_file = RunningFile()
>
> > The deller object is in a cycle as described above [as well as the
> > Deller class itself].
>
> I think you are mistaken here. The RunningFile instance in above code
> is *not* part of a cycle. It doesn't have any instance variables (i.e.
> its __dict__ is empty), and it only refers to its class, which (AFAICT)
> doesn't refer back to the instance.

As I explained above, it *is* part of a cycle: """including
the class objects themselves: class->dict->function->func_globals""".

Note:
running_file.__class.__.__dict__['__init__'].func_globals['running_file']
is running_file.


>
> > When Python exits, it could call
> > deller.__del__() and then collect the cycle. But Python does the wrong
> > thing here, and gets rid of the globals before calling __del__:
> > Exception exceptions.AttributeError: "'NoneType' object has no
> > attribute 'unlink'" in <bound method RunningFile.__del__ of
> > <__main__.RunningFile object at 0x7f9655eb92d0>> ignored
>
> This is a different issue. For shutdown, Python doesn't rely on
> cyclic garbage collection (only). Instead, all modules get forcefully
> cleared, causing this problem.

I know. I assumed Python does not rely on cyclic garbage collectioin
for shutdown, because it wouldn't work, as _all_ globals that have
*any* instance method will be part of a cycle, and any of them which
have a __del__ will not be collected.

> > I believe applying the above enhancement would solve these problems.
>
> No, they wouldn't.
>
> To work around the real problem in your case, put everything that the
> destructor uses into an instance or class attribute:
>
> class RunningFile(object):
> ? ? filename = '/tmp/running'
> ? ? _unlink = os.unlink
> ? ? def __init__(self):
> ? ? ? ? open(self.filename, 'wb')
> ? ? def __del__(self):
> ? ? ? ? self._unlink(self.filename)
I *mentioned* this workaround. What I propose is not a workaround but
a solution. You wouldn't need to clean up module globals ad-hoc'ishly,
because the cyclic collection would collect your object, even with its
__del__.

> Regards,
> Martin
Please read my entire mail before replying to it. Thanks!
Eyal

> _______________________________________________
> Python-Dev mailing list
> Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...

From eyal.lotem at gmail.com  Sun Jun 29 13:33:28 2008
From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com)
Date: Sun, 29 Jun 2008 04:33:28 -0700 (PDT)
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com> 
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> 
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
Message-ID: <b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com>

On Jun 29, 7:52?am, "Guido van Rossum" <gu... at python.org> wrote:
> On Sat, Jun 28, 2008 at 5:39 PM, Greg Ewing <greg.ew... at canterbury.ac.nz> wrote:
> > Nick Coghlan wrote:
>
> >> It's a fact of Python development: __del__ methods cannot safely reference
> >> module globals, because those globals may be gone by the time that method is
> >> invoked.
>
> > Speaking of this, has there been any more thought given
> > to the idea of dropping the module clearing and just
> > relying on cyclic GC?
>
> No, but it is an intriguing thought nevertheless. The module clearing
> causes nothing but trouble...
This is exactly what my post tried to address.
I assumed it was clear that module clearing is the wrong solution, and
that it was also clear that due to the cycles I mentioned
(global.__class__.__dict__['any_method'].func_globals['global'] is
global), all globals that have a __del__ will not be collectible.
Therefore, I proposed a solution to cycles with a __del__ in them.
Only with this solution it is possible to replace module clearing with
normal garbage collection.

Eyal

>
> --
> --Guido van Rossum (home page:http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...

From solipsis at pitrou.net  Sun Jun 29 14:04:28 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 29 Jun 2008 12:04:28 +0000 (UTC)
Subject: [Python-Dev] Cycle collection enhancement idea
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz>
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com>
Message-ID: <loom.20080629T114854-767@post.gmane.org>

eyal.lotem+pyutils <at> gmail.com <eyal.lotem <at> gmail.com> writes:
> This is exactly what my post tried to address.
> I assumed it was clear that module clearing is the wrong solution, and
> that it was also clear that due to the cycles I mentioned
> (global.__class__.__dict__['any_method'].func_globals['global'] is
> global), all globals that have a __del__ will not be collectible.
> Therefore, I proposed a solution to cycles with a __del__ in them.
> Only with this solution it is possible to replace module clearing with
> normal garbage collection.

A more generic solution would be to pass a special flag to the GC when it is
called as part of interpreter shutdown. This flag would instruct it to not put
aside objects with finalizers, but first call all their __del__ methods and then
collect them all at once regardless of whether they are part of a cycle or not.




From eyal.lotem at gmail.com  Sun Jun 29 14:18:46 2008
From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com)
Date: Sun, 29 Jun 2008 05:18:46 -0700 (PDT)
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <loom.20080629T114854-767@post.gmane.org>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com> 
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> 
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com> 
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com> 
	<loom.20080629T114854-767@post.gmane.org>
Message-ID: <e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>

On Jun 29, 3:04?pm, Antoine Pitrou <solip... at pitrou.net> wrote:
> eyal.lotem+pyutils <at> gmail.com <eyal.lotem <at> gmail.com> writes:
>
> > This is exactly what my post tried to address.
> > I assumed it was clear that module clearing is the wrong solution, and
> > that it was also clear that due to the cycles I mentioned
> > (global.__class__.__dict__['any_method'].func_globals['global'] is
> > global), all globals that have a __del__ will not be collectible.
> > Therefore, I proposed a solution to cycles with a __del__ in them.
> > Only with this solution it is possible to replace module clearing with
> > normal garbage collection.
>
> A more generic solution would be to pass a special flag to the GC when it is
> called as part of interpreter shutdown. This flag would instruct it to not put
> aside objects with finalizers, but first call all their __del__ methods and then
> collect them all at once regardless of whether they are part of a cycle or not.

That would be no worse than what happens now - but its still not
perfect (__del__ ordering issues). Also, you would need to temporarily
revive the cycles as mentioned above (to avoid accessibility of
partially destructed objects).

>
> _______________________________________________
> Python-Dev mailing list
> Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...

From solipsis at pitrou.net  Sun Jun 29 14:36:46 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 29 Jun 2008 12:36:46 +0000 (UTC)
Subject: [Python-Dev] Cycle collection enhancement idea
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz>
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com>
	<loom.20080629T114854-767@post.gmane.org>
	<e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>
Message-ID: <loom.20080629T122913-407@post.gmane.org>

eyal.lotem+pyutils <at> gmail.com <eyal.lotem <at> gmail.com> writes:
> 
> That would be no worse than what happens now - but its still not
> perfect (__del__ ordering issues). Also, you would need to temporarily
> revive the cycles as mentioned above (to avoid accessibility of
> partially destructed objects).

The idea is to call all __del__'s *before* any object in the cycle is
deallocated (that is, call them manually rather than as part of deallocating
them). That way you shouldn't have the issues mentioned above.




From eyal.lotem at gmail.com  Sun Jun 29 16:12:40 2008
From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com)
Date: Sun, 29 Jun 2008 07:12:40 -0700 (PDT)
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <loom.20080629T122913-407@post.gmane.org>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com> 
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> 
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com> 
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com> 
	<loom.20080629T114854-767@post.gmane.org>
	<e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>
	<loom.20080629T122913-407@post.gmane.org>
Message-ID: <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com>

On Jun 29, 3:36?pm, Antoine Pitrou <solip... at pitrou.net> wrote:
> eyal.lotem+pyutils <at> gmail.com <eyal.lotem <at> gmail.com> writes:
>
>
>
> > That would be no worse than what happens now - but its still not
> > perfect (__del__ ordering issues). Also, you would need to temporarily
> > revive the cycles as mentioned above (to avoid accessibility of
> > partially destructed objects).
>
> The idea is to call all __del__'s *before* any object in the cycle is
> deallocated (that is, call them manually rather than as part of deallocating
> them). That way you shouldn't have the issues mentioned above.

Firstly, as I said above: you will still have __del__ ordering issues.
Secondly, the destructor itself currently calls __del__, so if you
call __del__ before any deallocation, it will get called again as part
of the deallocation.  Might be a technicality but it will still
probably require some code restructuring to work around (or making
that code even more hairy).

> _______________________________________________
> Python-Dev mailing list
> Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...

From eyal.lotem at gmail.com  Sun Jun 29 16:21:38 2008
From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com)
Date: Sun, 29 Jun 2008 07:21:38 -0700 (PDT)
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com> 
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> 
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com> 
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com> 
	<loom.20080629T114854-767@post.gmane.org>
	<e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>
	<loom.20080629T122913-407@post.gmane.org>
	<8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com>
Message-ID: <0d7ccb29-8e2e-44e8-8955-990517c6f8d8@x41g2000hsb.googlegroups.com>

On Jun 29, 5:12?pm, "eyal.lotem+pyut... at gmail.com"
<eyal.lo... at gmail.com> wrote:
> On Jun 29, 3:36?pm, Antoine Pitrou <solip... at pitrou.net> wrote:
>
> > eyal.lotem+pyutils <at> gmail.com <eyal.lotem <at> gmail.com> writes:
>
> > > That would be no worse than what happens now - but its still not
> > > perfect (__del__ ordering issues). Also, you would need to temporarily
> > > revive the cycles as mentioned above (to avoid accessibility of
> > > partially destructed objects).
>
> > The idea is to call all __del__'s *before* any object in the cycle is
> > deallocated (that is, call them manually rather than as part of deallocating
> > them). That way you shouldn't have the issues mentioned above.
>
> Firstly, as I said above: you will still have __del__ ordering issues.
> Secondly, the destructor itself currently calls __del__, so if you
> call __del__ before any deallocation, it will get called again as part
> of the deallocation. ?Might be a technicality but it will still
> probably require some code restructuring to work around (or making
> that code even more hairy).

Additionally, there is another problem: If the cycle is not
temporarily revived, and you call __del__ manually, it may break the
cycle by removing the references.
Thus, objects in the cycle will go down to refcount=0 during your
attempt to call __del__'s on the objects in the cycle.
The only sane thing is to temporarily revive the entire cycle so you
can safely call __del__'s on it.  Then, you might want to disable the
normal __del__ calling that occurs as part of the later destruction of
the cycle.

>
> > _______________________________________________
> > Python-Dev mailing list
> > Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...
>
> _______________________________________________
> Python-Dev mailing list
> Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...

From solipsis at pitrou.net  Sun Jun 29 16:40:54 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 29 Jun 2008 14:40:54 +0000 (UTC)
Subject: [Python-Dev] Cycle collection enhancement idea
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz>
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com>
	<loom.20080629T114854-767@post.gmane.org>
	<e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>
	<loom.20080629T122913-407@post.gmane.org>
	<8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com>
	<0d7ccb29-8e2e-44e8-8955-990517c6f8d8@x41g2000hsb.googlegroups.com>
Message-ID: <loom.20080629T143120-48@post.gmane.org>

eyal.lotem+pyutils <at> gmail.com <eyal.lotem <at> gmail.com> writes:
> 
> Additionally, there is another problem: If the cycle is not
> temporarily revived, and you call __del__ manually, it may break the
> cycle by removing the references.
> Thus, objects in the cycle will go down to refcount=0 during your
> attempt to call __del__'s on the objects in the cycle.

But if we use gcmodule's current linked list mechanisme, wouldn't this situation
 be correctly handled by _PyObject_GC_UNTRACK? That is, before the object is
destroyed, the gc module already automatically removes it from its current
"collection". Therefore, walking the collection (in this case, the list of
unreachable objects) still does the right thing.
(ISTM the gc relies heavily on this property)

> Then, you might want to disable the
> normal __del__ calling that occurs as part of the later destruction of
> the cycle.

I was thinking a flag in the PyObject header would do the trick but there aren't
any flags in the PyObject header... *gasp*.





From solipsis at pitrou.net  Sun Jun 29 16:53:45 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 29 Jun 2008 14:53:45 +0000 (UTC)
Subject: [Python-Dev] Cycle collection enhancement idea
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz>
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com>
	<loom.20080629T114854-767@post.gmane.org>
	<e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>
	<loom.20080629T122913-407@post.gmane.org>
	<8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com>
	<0d7ccb29-8e2e-44e8-8955-990517c6f8d8@x41g2000hsb.googlegroups.com>
	<loom.20080629T143120-48@post.gmane.org>
Message-ID: <loom.20080629T145018-261@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> I was thinking a flag in the PyObject header would do the trick but there
> aren't any flags in the PyObject header... *gasp*.

Actually, we only care about GC-tracked objects (the others are deallocated
simply when their refcount falls to zero), so this could be another special
value in the gc_refs field, together with a specific macro.




From rhamph at gmail.com  Sun Jun 29 19:12:14 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Sun, 29 Jun 2008 11:12:14 -0600
Subject: [Python-Dev] GC Proposal
In-Reply-To: <486732D4.3020004@v.loewis.de>
References: <aac2c7cb0806261753xe16ca0dt7624c9a2c458edd@mail.gmail.com>
	<loom.20080628T184253-31@post.gmane.org>
	<aac2c7cb0806281259q5e4e60a1pb338b81a290ac18f@mail.gmail.com>
	<4866A223.8040902@v.loewis.de>
	<aac2c7cb0806281931h52138888pcbaf3774332cd786@mail.gmail.com>
	<486732D4.3020004@v.loewis.de>
Message-ID: <aac2c7cb0806291012k68a480f3rb7559ef1e6682250@mail.gmail.com>

On Sun, Jun 29, 2008 at 12:59 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Looks about equivalent, but "survivors" may mean two different things
>> depending on if it removes deleted survivors or not.  Splitting that
>> up, we get this form:
>>
>> old <= survivors * 2.0 + deleted * 1.0
>
> What precisely would be the "deleted" count? If it counts deallocations,
> is it relevant what generation the deallocated object was from?
> If so, how do you determine the generation? If not, wouldn't
>
> while 1:
>  x=[]
>
> trigger a full garbage collection fairly quickly?

"deleted" means deallocated survivors.

I could imagine wanting to move survivors and deleted up to the first
generation, to give a stronger guarantee of how often it'd run, but
that'd require a much smaller deleted factor.


-- 
Adam Olsen, aka Rhamphoryncus

From rhamph at gmail.com  Sun Jun 29 19:22:05 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Sun, 29 Jun 2008 11:22:05 -0600
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz>
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
Message-ID: <aac2c7cb0806291022m7313130eh32f0afee660d522f@mail.gmail.com>

On Sat, Jun 28, 2008 at 10:52 PM, Guido van Rossum <guido at python.org> wrote:
> On Sat, Jun 28, 2008 at 5:39 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Nick Coghlan wrote:
>>
>>> It's a fact of Python development: __del__ methods cannot safely reference
>>> module globals, because those globals may be gone by the time that method is
>>> invoked.
>>
>> Speaking of this, has there been any more thought given
>> to the idea of dropping the module clearing and just
>> relying on cyclic GC?
>
> No, but it is an intriguing thought nevertheless. The module clearing
> causes nothing but trouble...

Already gone in python-safethread.  Then again, so is __del__. ;)
(Replaced with __finalize__)


-- 
Adam Olsen, aka Rhamphoryncus

From martin at v.loewis.de  Sun Jun 29 19:49:44 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jun 2008 19:49:44 +0200
Subject: [Python-Dev] GC Proposal
In-Reply-To: <aac2c7cb0806291012k68a480f3rb7559ef1e6682250@mail.gmail.com>
References: <aac2c7cb0806261753xe16ca0dt7624c9a2c458edd@mail.gmail.com>	
	<loom.20080628T184253-31@post.gmane.org>	
	<aac2c7cb0806281259q5e4e60a1pb338b81a290ac18f@mail.gmail.com>	
	<4866A223.8040902@v.loewis.de>	
	<aac2c7cb0806281931h52138888pcbaf3774332cd786@mail.gmail.com>	
	<486732D4.3020004@v.loewis.de>
	<aac2c7cb0806291012k68a480f3rb7559ef1e6682250@mail.gmail.com>
Message-ID: <4867CB38.70006@v.loewis.de>


>> What precisely would be the "deleted" count? If it counts deallocations,
>> is it relevant what generation the deallocated object was from?
>> If so, how do you determine the generation? If not, wouldn't
>>
>> while 1:
>>  x=[]
>>
>> trigger a full garbage collection fairly quickly?
> 
> "deleted" means deallocated survivors.

So when an object gets deallocated, how do you find out whether it
survived a middle collection?

> I could imagine wanting to move survivors and deleted up to the first
> generation, to give a stronger guarantee of how often it'd run, but
> that'd require a much smaller deleted factor.

This I don't understand. What is the first generation? Generation 0
(i.e. the youngest objects)? Why would you want to move objects
surviving a garbage collection back to the youngest generation? That
totally defeats the purpose of generational garbage collection.

Also, why would you want to move deleted objects anywhere? Their
storage is reclaimed, and they don't occur in any list of objects.

Regards,
Martin


From martin at v.loewis.de  Sun Jun 29 20:00:49 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jun 2008 20:00:49 +0200
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <11765c26-7b07-4387-9a40-bf11aa7b89d2@a70g2000hsh.googlegroups.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<48665978.3020605@v.loewis.de>
	<11765c26-7b07-4387-9a40-bf11aa7b89d2@a70g2000hsh.googlegroups.com>
Message-ID: <4867CDD1.1040006@v.loewis.de>

> As I explained above, it *is* part of a cycle: """including
> the class objects themselves: class->dict->function->func_globals""".

Ah, right. I must have missed that explanation.

> I know. I assumed Python does not rely on cyclic garbage collectioin
> for shutdown, because it wouldn't work, as _all_ globals that have
> *any* instance method will be part of a cycle, and any of them which
> have a __del__ will not be collected.

No. The mechanism for cleaning modules at shutdown time predates cyclic
GC, and was not removed because "it wouldn't work".

This specific issue certainly contributes to the fact that it doesn't
work, but there might be other problems as well (such as extension
modules holding onto objects participating in cycles).

> I *mentioned* this workaround. What I propose is not a workaround but
> a solution. You wouldn't need to clean up module globals ad-hoc'ishly,
> because the cyclic collection would collect your object, even with its
> __del__.

I don't think it solves the problem. You seem to be assuming that any
such cycle will contain only a single global object with an __del__.
However, as modules refer to each other, I very much doubt that is the
case.

> Please read my entire mail before replying to it. Thanks!

I really, really tried. I read it three times before replying.

However, I found it really, really difficult to follow your writing,
as it was mixing problem statement and solution, so that I couldn't
tell what paragraph was about what. English is not my native language,
complicating communication further. Please accept my apologies.

Regards,
Martin

From martin at v.loewis.de  Sun Jun 29 20:04:06 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jun 2008 20:04:06 +0200
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz>
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com>
	<loom.20080629T114854-767@post.gmane.org>
	<e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>
Message-ID: <4867CE96.5060707@v.loewis.de>

> That would be no worse than what happens now - but its still not
> perfect (__del__ ordering issues). Also, you would need to temporarily
> revive the cycles as mentioned above (to avoid accessibility of
> partially destructed objects).

I don't quite understand what you mean by "revive cycles". There is
not need to revive any object in a cycle, as all objects are still alive
(or else they wouldn't be garbage).

Regards,
Martin

From martin at v.loewis.de  Sun Jun 29 20:13:27 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jun 2008 20:13:27 +0200
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz>
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com>
	<loom.20080629T114854-767@post.gmane.org>	<e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>	<loom.20080629T122913-407@post.gmane.org>
	<8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com>
Message-ID: <4867D0C7.2060202@v.loewis.de>

> Firstly, as I said above: you will still have __del__ ordering issues.

Can you please elaborate? What would such __del__ ordering issues be?

> Secondly, the destructor itself currently calls __del__, so if you
> call __del__ before any deallocation, it will get called again as part
> of the deallocation.  Might be a technicality but it will still
> probably require some code restructuring to work around (or making
> that code even more hairy).

There could be a global barricade for calling __del__: you first call
all __del__s of existing objects, then set the barricade, and then
start breaking cycles.

This could even be done with the current approach to module clearing.

Regards,
Martin

From martin at v.loewis.de  Sun Jun 29 20:15:07 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jun 2008 20:15:07 +0200
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <0d7ccb29-8e2e-44e8-8955-990517c6f8d8@x41g2000hsb.googlegroups.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz>
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com>
	<loom.20080629T114854-767@post.gmane.org>	<e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>	<loom.20080629T122913-407@post.gmane.org>	<8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com>
	<0d7ccb29-8e2e-44e8-8955-990517c6f8d8@x41g2000hsb.googlegroups.com>
Message-ID: <4867D12B.6030506@v.loewis.de>

> Additionally, there is another problem: If the cycle is not
> temporarily revived, and you call __del__ manually, it may break the
> cycle by removing the references.
> Thus, objects in the cycle will go down to refcount=0 during your
> attempt to call __del__'s on the objects in the cycle.
> The only sane thing is to temporarily revive the entire cycle so you
> can safely call __del__'s on it.  Then, you might want to disable the
> normal __del__ calling that occurs as part of the later destruction of
> the cycle.

I still don't understand what "revive the cycle" means. You will need to
incref the object for which you call __del__, that's all.

Regards,
Martin

From eyal.lotem at gmail.com  Sun Jun 29 20:23:34 2008
From: eyal.lotem at gmail.com (Eyal Lotem)
Date: Sun, 29 Jun 2008 21:23:34 +0300
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <4867CDD1.1040006@v.loewis.de>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<48665978.3020605@v.loewis.de>
	<11765c26-7b07-4387-9a40-bf11aa7b89d2@a70g2000hsh.googlegroups.com>
	<4867CDD1.1040006@v.loewis.de>
Message-ID: <b64f365b0806291123g4ffef81aq14cee5fed9b6f65e@mail.gmail.com>

On Sun, Jun 29, 2008 at 9:00 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> As I explained above, it *is* part of a cycle: """including
>> the class objects themselves: class->dict->function->func_globals""".
>
> Ah, right. I must have missed that explanation.
>
>> I know. I assumed Python does not rely on cyclic garbage collectioin
>> for shutdown, because it wouldn't work, as _all_ globals that have
>> *any* instance method will be part of a cycle, and any of them which
>> have a __del__ will not be collected.
>
> No. The mechanism for cleaning modules at shutdown time predates cyclic
> GC, and was not removed because "it wouldn't work".
>
> This specific issue certainly contributes to the fact that it doesn't
> work, but there might be other problems as well (such as extension
> modules holding onto objects participating in cycles).
>
>> I *mentioned* this workaround. What I propose is not a workaround but
>> a solution. You wouldn't need to clean up module globals ad-hoc'ishly,
>> because the cyclic collection would collect your object, even with its
>> __del__.
>
> I don't think it solves the problem. You seem to be assuming that any
> such cycle will contain only a single global object with an __del__.
> However, as modules refer to each other, I very much doubt that is the
> case.
>
>> Please read my entire mail before replying to it. Thanks!
>
> I really, really tried. I read it three times before replying.
>
> However, I found it really, really difficult to follow your writing,
> as it was mixing problem statement and solution, so that I couldn't
> tell what paragraph was about what. English is not my native language,
> complicating communication further. Please accept my apologies.

I apologize for my tone as well, it just seemed frustrating that all
the replies seemed to completely ignore the core point I was trying to
make and claim that the problem I was trying to solve was not a
problem at all, but a behavior I was unaware of...

Mixing another quote from another mail:
> > That would be no worse than what happens now - but its still not
> > perfect (__del__ ordering issues). Also, you would need to temporarily
> > revive the cycles as mentioned above (to avoid accessibility of
> > partially destructed objects).
>
> I don't quite understand what you mean by "revive cycles". There is
> not need to revive any object in a cycle, as all objects are still alive
> (or else they wouldn't be garbage).
By "revive cycles", I mean make sure that they are referenced by an
independent referrer (one that won't go away as part of the __del__
calling process).  This is similar to how the tp_dealloc code
increases the refcount (actually sets it to 1, because it was
certainly 0 when entering the destructor) before calling the __del__
slot.  Without reviving the object before calling its __del__ in the
destructor, and without reviving the objects of a cycle before calling
its __del__'s, the __del__ Pythonic code may be exposed to "dead
objects" (refcount==0).

Consider the cycle:
a.x = b
b.x = a

Lets suppose the a object has a __del__. Lets assume each object in
the cycle has a refcount of 1 (and the cycle should die). Now lets say
this is a's __del__ code:
def __del__(self):
  self.x = None

Running it will set 'b's refcount to 0 and call its destructor, which
will set 'a's refcount to 0 and also call its destructor. But its
__del__ is currently running - so "self" must not have a refcount of
0.  If you only incref on 'a' before calling __del__, then you are
probably alright, as long as there is only one __del__.

> Can you please elaborate? What would such __del__ ordering issues be?

class A(object):
  def __del__(self):
    print self.x.attribute

class B(object):
  def __del__(self):
    print "B is going down!"
    del self.attribute

a = A()
b = B()
a.x = b
b.attribute = 1

If you call b's __del__ first then a's __del__ will fail. If you call
a's __del__ first, then all is well.  Ofcourse you can create true
cyclic dependencies that no order will work, and its pretty clear
there is no way to deduce the right order anyway.  This is what I mean
by "ordering issues".

> There could be a global barricade for calling __del__: you first call
> all __del__s of existing objects, then set the barricade, and then
> start breaking cycles.
> This could even be done with the current approach to module clearing.

Note that the __del__'s themselves may be breaking cycles and
refcounts will go to 0 - unless you temporarily revive (incref) the
entire cycle first.

> I still don't understand what "revive the cycle" means. You will need to
> incref the object for which you call __del__, that's all.

Unless there are multiple __del__'s in the cycle.

>
> Regards,
> Martin
>

From rhamph at gmail.com  Sun Jun 29 20:49:41 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Sun, 29 Jun 2008 12:49:41 -0600
Subject: [Python-Dev] GC Proposal
In-Reply-To: <4867CB38.70006@v.loewis.de>
References: <aac2c7cb0806261753xe16ca0dt7624c9a2c458edd@mail.gmail.com>
	<loom.20080628T184253-31@post.gmane.org>
	<aac2c7cb0806281259q5e4e60a1pb338b81a290ac18f@mail.gmail.com>
	<4866A223.8040902@v.loewis.de>
	<aac2c7cb0806281931h52138888pcbaf3774332cd786@mail.gmail.com>
	<486732D4.3020004@v.loewis.de>
	<aac2c7cb0806291012k68a480f3rb7559ef1e6682250@mail.gmail.com>
	<4867CB38.70006@v.loewis.de>
Message-ID: <aac2c7cb0806291149r6ed0858eu372c40039cf93f11@mail.gmail.com>

On Sun, Jun 29, 2008 at 11:49 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
>>> What precisely would be the "deleted" count? If it counts deallocations,
>>> is it relevant what generation the deallocated object was from?
>>> If so, how do you determine the generation? If not, wouldn't
>>>
>>> while 1:
>>>  x=[]
>>>
>>> trigger a full garbage collection fairly quickly?
>>
>> "deleted" means deallocated survivors.
>
> So when an object gets deallocated, how do you find out whether it
> survived a middle collection?

Reusing gc_refs for another flag.


>> I could imagine wanting to move survivors and deleted up to the first
>> generation, to give a stronger guarantee of how often it'd run, but
>> that'd require a much smaller deleted factor.
>
> This I don't understand. What is the first generation? Generation 0
> (i.e. the youngest objects)? Why would you want to move objects
> surviving a garbage collection back to the youngest generation? That
> totally defeats the purpose of generational garbage collection.
>
> Also, why would you want to move deleted objects anywhere? Their
> storage is reclaimed, and they don't occur in any list of objects.

Sorry, really poor wording on my part.  I meant to move the
*counters*, so that even the first generation qualifies as a
"survivor", and thus contributes to the "deleted" count.


-- 
Adam Olsen, aka Rhamphoryncus

From martin at v.loewis.de  Sun Jun 29 20:57:43 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jun 2008 20:57:43 +0200
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <b64f365b0806291123g4ffef81aq14cee5fed9b6f65e@mail.gmail.com>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>	
	<48665978.3020605@v.loewis.de>	
	<11765c26-7b07-4387-9a40-bf11aa7b89d2@a70g2000hsh.googlegroups.com>	
	<4867CDD1.1040006@v.loewis.de>
	<b64f365b0806291123g4ffef81aq14cee5fed9b6f65e@mail.gmail.com>
Message-ID: <4867DB27.3010302@v.loewis.de>

> By "revive cycles", I mean make sure that they are referenced by an
> independent referrer (one that won't go away as part of the __del__
> calling process).

I think this is a) unfortunate terminology (as the cycle is not dead,
so no need to revive), and b) unnecessary, as calling __del__ will
add a reference, anyway (in the implicit self parameter).

So the object won't go away as long as __del__ runs. It might go
away immediately after __del__ returns, which may or may not be a
problem.

> This is similar to how the tp_dealloc code
> increases the refcount (actually sets it to 1, because it was
> certainly 0 when entering the destructor) before calling the __del__
> slot.  Without reviving the object before calling its __del__ in the
> destructor, and without reviving the objects of a cycle before calling
> its __del__'s, the __del__ Pythonic code may be exposed to "dead
> objects" (refcount==0).

No, that can't happen, and, AFAICT, is *not* the reason why the
tp_dealloc resurrects the object. Instead, if it wouldn't
resurrect it, tp_dealloc might become recursive, deallocating the
object twice.

> Consider the cycle:
> a.x = b
> b.x = a
> 
> Lets suppose the a object has a __del__. Lets assume each object in
> the cycle has a refcount of 1 (and the cycle should die). Now lets say
> this is a's __del__ code:
> def __del__(self):
>   self.x = None
> 
> Running it will set 'b's refcount to 0 and call its destructor, which
> will set 'a's refcount to 0 and also call its destructor. But its
> __del__ is currently running - so "self" must not have a refcount of
> 0.

And it won't, because (say) PyObject_CallMethod (to call __del__) calls
PyObject_GetAttrString, which returns a bound method which refers to
im_self for the entire life of

> If you only incref on 'a' before calling __del__, then you are
> probably alright, as long as there is only one __del__.

Why would you think so? We explicitly call one __del__. Assume
that breaks the cycle, causing another object with __del__ to go
to refcount zero. Now, tp_dealloc is called, raises the refcount,
calls __del__ of the other object, and releases its storage.

>> Can you please elaborate? What would such __del__ ordering issues be?
> 
> If you call b's __del__ first then a's __del__ will fail. If you call
> a's __del__ first, then all is well.  Ofcourse you can create true
> cyclic dependencies that no order will work, and its pretty clear
> there is no way to deduce the right order anyway.  This is what I mean
> by "ordering issues".

I see. As we are in interpreter shutdown, any such exceptions should
be ignored (as exceptions in __del__ are, anyway). Programs involving
such cycles should be considered broken, and be rewritten to avoid them
(which I claim is always possible, and straight-forward)

> Note that the __del__'s themselves may be breaking cycles and
> refcounts will go to 0 - unless you temporarily revive (incref) the
> entire cycle first.

See above - you shouldn't need to.

>> I still don't understand what "revive the cycle" means. You will need to
>> incref the object for which you call __del__, that's all.
> 
> Unless there are multiple __del__'s in the cycle.

Not even then.

Regards,
Martin

From dickinsm at gmail.com  Mon Jun 30 02:26:11 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 30 Jun 2008 01:26:11 +0100
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
In-Reply-To: <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<ca471dc20806260828o40bf7af9u97442f1285c90e52@mail.gmail.com>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
	<5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>
	<58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1>
Message-ID: <5c6f2a5d0806291726o77bcd4ffvcf08c4ab2be539a4@mail.gmail.com>

On Sun, Jun 29, 2008 at 12:46 AM, Raymond Hettinger <python at rcn.com> wrote:
> Is everyone agreed on a tohex/fromhex pair using the C99 notation as
> recommended in 754R?

Sounds good to me.

I've attached a Python version of a possible implementation to the issue. See:

http://bugs.python.org/file10780/hex_float.py

It might be useful for testing.

Mark

From greg.ewing at canterbury.ac.nz  Mon Jun 30 02:43:28 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 30 Jun 2008 12:43:28 +1200
Subject: [Python-Dev] Cycle collection enhancement idea
In-Reply-To: <loom.20080629T122913-407@post.gmane.org>
References: <b64f365b0806280640s4354e3f9xea4754285ec1c33f@mail.gmail.com>
	<486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz>
	<ca471dc20806282152m2b125037r33656e52e8d6c66a@mail.gmail.com>
	<b4186cc3-8646-43ca-9072-3d2a06329aab@e39g2000hsf.googlegroups.com>
	<loom.20080629T114854-767@post.gmane.org>
	<e6519b3b-291c-485a-bec1-b107e877e82f@r66g2000hsg.googlegroups.com>
	<loom.20080629T122913-407@post.gmane.org>
Message-ID: <48682C30.3060106@canterbury.ac.nz>

Since it's already possible for __del__-containing cycles
to survive interpreter shutdown, I don't see that this
issue ought to be a showstopper for elimination of module
clearing.

Also, it seems to me that the kind of cycles module
clearing is designed to break, i.e. those between classes,
functions and the module dict, are unlikely to contain
objects with ___del__ methods, so there wouldn't be
much of a problem in practice.

-- 
Greg

From guido at python.org  Mon Jun 30 17:53:14 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Jun 2008 08:53:14 -0700
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
In-Reply-To: <5c6f2a5d0806291726o77bcd4ffvcf08c4ab2be539a4@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<DFB9DC3C394F437E891D6635A8BFC1C8@RaymondLaptop1>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
	<5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>
	<58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1>
	<5c6f2a5d0806291726o77bcd4ffvcf08c4ab2be539a4@mail.gmail.com>
Message-ID: <ca471dc20806300853m7d2f269bibaf753c18c6f48b3@mail.gmail.com>

FWIW, I'm fine with making these methods on float -- a class method
float.fromhex(...) echoes e.g. dict.fromkeys(...) and
datetime.fromordinal(...). The to-hex conversion could be x.hex() --
we don't tend to use ".toxyz()" as a naming convention much in Python.

On Sun, Jun 29, 2008 at 5:26 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> On Sun, Jun 29, 2008 at 12:46 AM, Raymond Hettinger <python at rcn.com> wrote:
>> Is everyone agreed on a tohex/fromhex pair using the C99 notation as
>> recommended in 754R?
>
> Sounds good to me.
>
> I've attached a Python version of a possible implementation to the issue. See:
>
> http://bugs.python.org/file10780/hex_float.py
>
> It might be useful for testing.
>
> Mark
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dickinsm at gmail.com  Mon Jun 30 18:31:06 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 30 Jun 2008 17:31:06 +0100
Subject: [Python-Dev] [Python-checkins] r64424 -
	inpython/trunk:Include/object.h
	Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c
	Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c
In-Reply-To: <ca471dc20806300853m7d2f269bibaf753c18c6f48b3@mail.gmail.com>
References: <20080620041816.4D5E81E4002@bag.python.org>
	<4863F43C.2080904@v.loewis.de>
	<5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com>
	<4863FC7B.6070903@v.loewis.de>
	<5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com>
	<04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1>
	<5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com>
	<58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1>
	<5c6f2a5d0806291726o77bcd4ffvcf08c4ab2be539a4@mail.gmail.com>
	<ca471dc20806300853m7d2f269bibaf753c18c6f48b3@mail.gmail.com>
Message-ID: <5c6f2a5d0806300931x7635cee5t597c09ff7b06fc6f@mail.gmail.com>

On Mon, Jun 30, 2008 at 4:53 PM, Guido van Rossum <guido at python.org> wrote:
> FWIW, I'm fine with making these methods on float -- a class method
> float.fromhex(...) echoes e.g. dict.fromkeys(...) and
> datetime.fromordinal(...). The to-hex conversion could be x.hex() --
> we don't tend to use ".toxyz()" as a naming convention much in Python.

Would it be totally outrageous for the float constructor to accept
hex strings directly?

Mark

From chris at atlee.ca  Mon Jun 30 19:18:17 2008
From: chris at atlee.ca (Chris AtLee)
Date: Mon, 30 Jun 2008 13:18:17 -0400
Subject: [Python-Dev] urllib,
	multipart/form-data encoding and file uploads
In-Reply-To: <4865F2E5.4000506@v.loewis.de>
References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>
	<4865F2E5.4000506@v.loewis.de>
Message-ID: <7790b6530806301018v733955a4mfa050381a73d922a@mail.gmail.com>

On Sat, Jun 28, 2008 at 4:14 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> I didn't see any recent discussion about this so I thought I'd ask
>> here: do you think this would make a good addition to the new urllib
>> package?
>
> Just in case that isn't clear: any such change must be delayed for
> 2.7/3.1. That is not to say that you couldn't start implementing it
> now, of course.

I like a challenge :)

As discussed previously, there are two parts to this: handling
streaming HTTP requests, and multipart/form-data encoding.

I notice that support for file objects has already been added to 2.6's
httplib.  The changes required to support iterable objects are very
minimal:

Index: Lib/httplib.py
===================================================================
--- Lib/httplib.py	(revision 64600)
+++ Lib/httplib.py	(working copy)
@@ -688,7 +688,12 @@
         self.__state = _CS_IDLE

     def send(self, str):
-        """Send `str' to the server."""
+        """Send `str` to the server.
+
+        ``str`` can be a string object, a file-like object that supports
+        a .read() method, or an iterable object that supports a .next()
+        method.
+        """
         if self.sock is None:
             if self.auto_open:
                 self.connect()
@@ -710,6 +715,10 @@
                 while data:
                     self.sock.sendall(data)
                     data=str.read(blocksize)
+            elif hasattr(str,'next'):
+                if self.debuglevel > 0: print "sendIng an iterable"
+                for data in str:
+                    self.sock.sendall(data)
             else:
                 self.sock.sendall(str)
         except socket.error, v:

(small aside, should the parameter called 'str' be renamed to
something else to avoid conflicts with the 'str' builtin?)

All regression tests continue to pass with this change applied.

If this change is not applied, then we have to jump through a couple
of hoops to support iterable HTTP request bodies:

- Provide our own httplib.HTTP(S)Connection classes that override
send() to do exactly what the patch above accomplishes

- Provide our own urllib2.HTTP(S)Handler classes that will use the new
HTTP(S)Connection classes

- Register the new HTTP(S)Handler classes with urllib2 so they take
priority over the standard handlers

I've created the necessary sub-classes, as well as several classes and
functions to do multipart/form-data encoding of strings and files.  My
current work is available online here: http://atlee.ca/software/poster
(tarball here: http://atlee.ca/software/poster/dist/0.1/poster-0.1dev.tar.gz)

Cheers,
Chris

From chris at atlee.ca  Mon Jun 30 20:05:25 2008
From: chris at atlee.ca (Chris AtLee)
Date: Mon, 30 Jun 2008 14:05:25 -0400
Subject: [Python-Dev] urllib,
	multipart/form-data encoding and file uploads
In-Reply-To: <48658EA5.4000803@gmail.com>
References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com>
	<4610679334594632865@unknownmsgid>
	<7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com>
	<48658EA5.4000803@gmail.com>
Message-ID: <7790b6530806301105t106d8d91j822c327f3df0ae06@mail.gmail.com>

On Fri, Jun 27, 2008 at 9:06 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Chris,
>
> To avoid losing these ideas, could you add them to the issue tracker as
> feature requests? It's too late to get them into 2.6/3.0 but they may make
> good additions for the next release cycle.
>
> Cheers,
> Nick.

Issues #3243 and #3244 created.

Cheers,
Chris