From greg.ewing at canterbury.ac.nz  Tue Aug  1 02:30:09 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 01 Aug 2006 12:30:09 +1200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44CE4D32.6040909@ewtllc.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
Message-ID: <44CEA091.1010406@canterbury.ac.nz>

Raymond Hettinger wrote:

> I think this would harm more than it would help.  It more confusing to 
> have several rounding-thingies to choose from than it is have an 
> explicit two-step.

But is it more confusing enough to be worth forcing
everyone to pay two function calls instead of one
in the most common case?

If I'm right that rounding-to-int is much more
commonly needed than rounding-to-float, the least
confusing thing would be for the builtin round()
to return an int, and have something somewhere
else, such as math.fround(), for round-to-float.

> BTW,  I thought the traditional idiom (for positive numbers) was:  
> int(x+.5)

It's the "for positive numbers" that's the problem
there. Most of my uses for round() involve graphics
coordinates, which I can't be sure won't be negative.
It's not immediately obvious what this will do with
negative numbers. It's not even immediately obvious
that it's doing round-to-nearest-integer unless
you're familiar with the idiom. A single well-named
function would be much more explicit.

--
Greg

From mal at egenix.com  Tue Aug  1 11:37:50 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 01 Aug 2006 11:37:50 +0200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44CEA091.1010406@canterbury.ac.nz>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>	<44CAD9A1.8080000@canterbury.ac.nz>
	<44CE4D32.6040909@ewtllc.com> <44CEA091.1010406@canterbury.ac.nz>
Message-ID: <44CF20EE.70103@egenix.com>

Greg Ewing wrote:
> Raymond Hettinger wrote:
> 
>> I think this would harm more than it would help.  It more confusing to 
>> have several rounding-thingies to choose from than it is have an 
>> explicit two-step.
> 
> But is it more confusing enough to be worth forcing
> everyone to pay two function calls instead of one
> in the most common case?

I suppose you don't know about the optional argument
to round that lets you round up to a certain decimal ?!

If we were to follow your suggestion, we'd have round()
return an integer when called without the argument and
a float when called with the argument.

Dependency of the return type on arguments to a function
should be avoided, if possible.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 01 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Tue Aug  1 11:50:19 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 01 Aug 2006 11:50:19 +0200
Subject: [Python-Dev] Which version of distutils to ship with Python 2.5?
In-Reply-To: <44CCB0A5.30706@v.loewis.de>
References: <43aa6ff70607261924w65e01325o854f87d4c109026a@mail.gmail.com>	<44C85FF2.3030701@v.loewis.de>	<200607271701.58223.anthony@interlink.com.au>
	<44CCB0A5.30706@v.loewis.de>
Message-ID: <44CF23DB.90909@egenix.com>

Martin v. L?wis wrote:
> Anthony Baxter schrieb:
>>> In any case, I bumped the version number to 2.5, according to the
>>> policy discussed in
>>>
>> Could this not simply use the Python version number directly, instead?
> 
> See the prior discussion at
> 
> http://mail.python.org/pipermail/distutils-sig/2005-January/004366.html
> 
> Some people still believe (at least, believed in January 2005), that
> distutils is developed independently of Python, and thus deserves
> its own version number.

They still do.

Even if distutils is part of the stdlib, it is
a software package that's intended to be used for more than just
the latest Python release.

The fact that were no official separate distutils releases
shouldn't be regarded as meaning that there's no interest
in using distutils with older Python versions.

> Of course, Andrew Kuchling officially declared in r1982 of pep 291
> that there won't be any further stand-alone distutils releases,
> and therefore, backwards compatibility with 2.1 is not necessary
> anymore.

AFAIK, there was no public discussion about this on the
distutils SIG list. While I don't think that Python 2.1
compatibility is still a requirement, Python 2.3 compatibility
should be maintained in order to make SVN distutils work
with all versions of Python since 2.3.

distutils should then be added back to PEP 291.

> So I changed distutils.__version__ again, to be derived from
> sys.version_info.

See the discussion on the checkins list for why this will
only cause problems instead of fixing any.

http://mail.python.org/pipermail/python-checkins/2006-July/055139.html

> I left the numerous comments still in distutils that compatibility
> with 2.1 is desired. We should remove these after 2.5 is released
> (or perhaps even before that).

Please don't. Instead we should have a discussion about this
whole issue on the distutils list and then decide.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 01 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From ncoghlan at gmail.com  Tue Aug  1 13:00:33 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 01 Aug 2006 21:00:33 +1000
Subject: [Python-Dev] Bad interaction of __index__ and sequence repeat
In-Reply-To: <44CE4BB9.60903@ieee.org>
References: <20060728101133.GA339@code0.codespeak.net>
	<44CB6B7D.1040001@gmail.com> <44CB92F1.9040304@iinet.net.au>
	<44CE4BB9.60903@ieee.org>
Message-ID: <44CF3451.4060500@gmail.com>

Travis Oliphant wrote:
>> Probably the most interesting thing now would be for Travis to review 
>> it, and see whether it makes things easier to handle for the Numeric 
>> scalar types (given the amount of code the patch deleted from the 
>> builtin and standard library data types, hopefully the benefits to 
>> Numeric will be comparable).
> 
> 
> I noticed most of the checks for PyInt where removed in the patch.  If I 
> remember correctly, I left these in for "optimization."   Other than 
> that, I think the patch is great.

You're right - there was a fast path based on PyInt_Check in 
_PyEval_SliceIndex that got lost, which I'll add back in. I'll also add fast 
paths for PyInt_Check to the functions in abstract.c, too.

The other PyInt_Check's (in slot_nb_index and instance_index) were there to 
check that __index__ returned the right thing. The check was still there in 
slot_nb_index, but I'd incorrectly removed it from instance_index. I'll add 
that one back in, too.

Once that's done, I'll update the tracker item and reassign to Tim for a review.

Cheers,
Nick.

> As far as helping with NumPy,  I think it will help to be able to remove 
> special-checks for all the different integer-types.  But, this has not 
> yet been done in the NumPy code.
> 
> -Travis
> 
> 
> 


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From greg.ewing at canterbury.ac.nz  Tue Aug  1 13:19:03 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 01 Aug 2006 23:19:03 +1200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44CF20EE.70103@egenix.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
Message-ID: <44CF38A7.8040304@canterbury.ac.nz>

M.-A. Lemburg wrote:

> I suppose you don't know about the optional argument
> to round that lets you round up to a certain decimal ?!

Yes, I know about, but I rarely if ever use it.
Rounding a binary float to a number of decimal
places seems a fundamentally ill-considered thing
to do anyway. What are the use cases for it,
given that one can easily select a number of decimal
places when formatting a number for display?

> If we were to follow your suggestion, we'd have round()
> return an integer when called without the argument and
> a float when called with the argument.

No, round() wouldn't have that option at all. If
you wanted it, you would use fround() instead,
which would have the option and return a float
always.

This would be a Py3k thing, obviously. If done
before then, the new function would have to be
given a different name.

--
Greg

From mal at egenix.com  Tue Aug  1 13:40:06 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 01 Aug 2006 13:40:06 +0200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and	coercing floats to integers)
In-Reply-To: <44CF38A7.8040304@canterbury.ac.nz>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>	<44CAD9A1.8080000@canterbury.ac.nz>
	<44CE4D32.6040909@ewtllc.com>	<44CEA091.1010406@canterbury.ac.nz>
	<44CF20EE.70103@egenix.com> <44CF38A7.8040304@canterbury.ac.nz>
Message-ID: <44CF3D96.7090101@egenix.com>

Greg Ewing wrote:
> M.-A. Lemburg wrote:
> 
>> I suppose you don't know about the optional argument
>> to round that lets you round up to a certain decimal ?!
> 
> Yes, I know about, but I rarely if ever use it.
> Rounding a binary float to a number of decimal
> places seems a fundamentally ill-considered thing
> to do anyway. What are the use cases for it,
> given that one can easily select a number of decimal
> places when formatting a number for display?

You often have a need for controlled rounding when doing
financial calculations or in situations where you want to
compare two floats with a given accuracy, e.g. to work
around rounding problems ;-)

The usual approach is to use full float accuracy
throughout the calculation and then apply rounding
a certain key places.

Float formatting is an entirely different issue.

>> If we were to follow your suggestion, we'd have round()
>> return an integer when called without the argument and
>> a float when called with the argument.
> 
> No, round() wouldn't have that option at all. If
> you wanted it, you would use fround() instead,
> which would have the option and return a float
> always.
> 
> This would be a Py3k thing, obviously. If done
> before then, the new function would have to be
> given a different name.

Hmm, looks like a YAGNI to me, but perhaps I'm missing
something :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 01 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From nmm1 at cus.cam.ac.uk  Tue Aug  1 15:12:38 2006
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Tue, 01 Aug 2006 14:12:38 +0100
Subject: [Python-Dev] Rounding float to int directly ...
Message-ID: <E1G7u34-0004Ka-N0@draco.cus.cam.ac.uk>

"M.-A. Lemburg" <mal at egenix.com> wrote:
>
> You often have a need for controlled rounding when doing
> financial calculations or in situations where you want to
> compare two floats with a given accuracy, e.g. to work
> around rounding problems ;-)

The latter is a crude hack, and was traditionally used to save cycles
when floating-point division was very slow.  There are better ways,
and have been for decades.

> Float formatting is an entirely different issue.

Not really.  You need controlled rounding to a fixed precision in
the other base.  But I agree that controlled rounding in binary
does not help with controlled rounding in decimal.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From aahz at pythoncraft.com  Tue Aug  1 16:09:24 2006
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 1 Aug 2006 07:09:24 -0700
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
	and	coercing floats to integers)
In-Reply-To: <44CF3D96.7090101@egenix.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
Message-ID: <20060801140923.GA19963@panix.com>

On Tue, Aug 01, 2006, M.-A. Lemburg wrote:
>
> You often have a need for controlled rounding when doing financial
> calculations or in situations where you want to compare two floats
> with a given accuracy, e.g. to work around rounding problems ;-)
>
> The usual approach is to use full float accuracy throughout the
> calculation and then apply rounding a certain key places.

That's what Decimal() is for.

(Note that I don't care all that much about round(), but I do think we
want to canonicalize Decimal() for all purposes in standard Python where
people care about accuracy.  Additional float features can go into
NumPy.)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it."  --Brian W. Kernighan

From nmm1 at cus.cam.ac.uk  Tue Aug  1 16:53:14 2006
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Tue, 01 Aug 2006 15:53:14 +0100
Subject: [Python-Dev] Rounding float to int directly ...
Message-ID: <E1G7vcQ-0005G0-Pc@draco.cus.cam.ac.uk>

Aahz <aahz at pythoncraft.com> wrote:
> On Tue, Aug 01, 2006, M.-A. Lemburg wrote:
> >
> > You often have a need for controlled rounding when doing financial
> > calculations or in situations where you want to compare two floats
> > with a given accuracy, e.g. to work around rounding problems ;-)
> >
> > The usual approach is to use full float accuracy throughout the
> > calculation and then apply rounding a certain key places.
> 
> That's what Decimal() is for.

Well, maybe.  There are other approaches, too, and Decimal has its
problems with that.  In particular, when people need precisely
defined decimal rounding, they ALWAYS need fixed-point and not
floating-point.

> (Note that I don't care all that much about round(), but I do think we
> want to canonicalize Decimal() for all purposes in standard Python where
> people care about accuracy.  Additional float features can go into
> NumPy.)

Really?  That sounds like dogma, not science.

Decimal doesn't even help people who care about accuracy.  At most
(and with the reservation mentioned above), it means that you can
can map external decimal formats to internal ones without loss of
precision.  Not a big deal, as there ARE no requirements for doing
that for floating-point, and there are plenty of other solutions for
fixed-point.

People who care about the accuracy of calculations prefer binary,
as it is a more accurate model.  That isn't a big deal, either.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From rhettinger at ewtllc.com  Tue Aug  1 17:21:15 2006
From: rhettinger at ewtllc.com (Raymond Hettinger)
Date: Tue, 01 Aug 2006 08:21:15 -0700
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: <E1G7vcQ-0005G0-Pc@draco.cus.cam.ac.uk>
References: <E1G7vcQ-0005G0-Pc@draco.cus.cam.ac.uk>
Message-ID: <44CF716B.6000603@ewtllc.com>

An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060801/0beb65ca/attachment.html 

From nmm1 at cus.cam.ac.uk  Tue Aug  1 17:37:29 2006
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Tue, 01 Aug 2006 16:37:29 +0100
Subject: [Python-Dev] Rounding float to int directly ...
Message-ID: <E1G7wJF-0005d5-2G@draco.cus.cam.ac.uk>

Raymond Hettinger <rhettinger at ewtllc.com> wrote:
>
> Hogwash.&nbsp; The only issues with decimal are ease-of-use and speed.<br>

I suggest that you get hold of a good 1960s or 1970s book on computer
arithmetic, and read up about "wobbling precision".  While it is not
a big deal, it was regarded as such, and is important enough to cause
significant numerical problems to the unwary - which means 99.99% of
modern programmers :-(

And, as I am sure that Aahz could point out, there are significant
reliability issues concerned with frequent base changes where any loss
of precision is unacceptable.  Yes, it can always be done, but only a
few people are likely to do it correctly in all cases.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From ncoghlan at gmail.com  Tue Aug  1 17:42:46 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 02 Aug 2006 01:42:46 +1000
Subject: [Python-Dev] Bad interaction of __index__ and sequence repeat
In-Reply-To: <44CF3451.4060500@gmail.com>
References: <20060728101133.GA339@code0.codespeak.net>	<44CB6B7D.1040001@gmail.com>
	<44CB92F1.9040304@iinet.net.au>	<44CE4BB9.60903@ieee.org>
	<44CF3451.4060500@gmail.com>
Message-ID: <44CF7676.2010102@gmail.com>

Nick Coghlan wrote:
> Travis Oliphant wrote:
>>> Probably the most interesting thing now would be for Travis to review 
>>> it, and see whether it makes things easier to handle for the Numeric 
>>> scalar types (given the amount of code the patch deleted from the 
>>> builtin and standard library data types, hopefully the benefits to 
>>> Numeric will be comparable).
>>
>> I noticed most of the checks for PyInt where removed in the patch.  If I 
>> remember correctly, I left these in for "optimization."   Other than 
>> that, I think the patch is great.
> 
> You're right - there was a fast path based on PyInt_Check in 
> _PyEval_SliceIndex that got lost, which I'll add back in. I'll also add fast 
> paths for PyInt_Check to the functions in abstract.c, too.
> 
> The other PyInt_Check's (in slot_nb_index and instance_index) were there to 
> check that __index__ returned the right thing. The check was still there in 
> slot_nb_index, but I'd incorrectly removed it from instance_index. I'll add 
> that one back in, too.
> 
> Once that's done, I'll update the tracker item and reassign to Tim for a review.

Aside from the 'assign to Tim' part, this is done now (pep357-fix-v3 on the 
tracker item). I also tweaked the clipping version of the C API to optionally 
expose the overflow flag that abstract.c was using internally so the client 
code can find out whether or not clipping actually occurred.

The patch also updates the index unit tests to:
   - cover the various type errors that Travis picked up
   - actually run the full set of unit tests under -O
   - only run the basic sequence independent tests once (instead of once per 
sequence type)

Neal also had a number of questions that I responded to in the tracker comments.

The most significant question related to the API signature, which Neal 
considered potentially confusing:
   PyNumber_Index(PyObject *item, int *is_index)

I added the is_index flag to the function signatures instead of providing a 
separate PyNumber_IndexCheck function in order to avoid doing the same type 
check twice in the typical mp_subscript implementation cases.

One possibility would be to invert the sense of that flag and call it 
"typeerror", which probably more accurately reflects what it's intended for - 
it's a way of telling the function "if this object does not have the correct 
type, tell me by setting this flag instead of by setting the Python error state".

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From skip at pobox.com  Tue Aug  1 20:25:56 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 1 Aug 2006 13:25:56 -0500
Subject: [Python-Dev] clock_gettime() vs. gettimeofday()?
Message-ID: <17615.40116.461692.83632@montanaro.dyndns.org>


This has probably been discussed before, however ...

Is there any reason to use clock_gettime() in preference to gettimeofday()
if it exists?  It pretends to at least return seconds + nanoseconds, where
gettimeofday() returns seconds + microseconds.  Are there perhaps some
platforms where it's possible to actually get more useful bits?  On my
Solaris 10/Intel desktop box clock_getres shows 0.01s resolution (not
obviously better) while on a Linux box here it shows 0.001s resolution.  Is
that better than what gettimeofday() does?

Skip

From rrr at ronadam.com  Tue Aug  1 23:07:57 2006
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 01 Aug 2006 16:07:57 -0500
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: <E1G7u34-0004Ka-N0@draco.cus.cam.ac.uk>
References: <E1G7u34-0004Ka-N0@draco.cus.cam.ac.uk>
Message-ID: <44CFC2AD.2070205@ronadam.com>

Nick Maclaren wrote:
> "M.-A. Lemburg" <mal at egenix.com> wrote:
>> You often have a need for controlled rounding when doing
>> financial calculations or in situations where you want to
>> compare two floats with a given accuracy, e.g. to work
>> around rounding problems ;-)
> 
> The latter is a crude hack, and was traditionally used to save cycles
> when floating-point division was very slow.  There are better ways,
> and have been for decades.
> 
>> Float formatting is an entirely different issue.
> 
> Not really.  You need controlled rounding to a fixed precision in
> the other base.  But I agree that controlled rounding in binary
> does not help with controlled rounding in decimal.


I'm -1 on implicitly converting to an int when rounding.

One reason is if your rounded (to int type) number is then used in an 
equation you my end up with a integer result when you wanted a floating 
point result.

 >>> 23.5/5.2
4.5192307692307692
 >>> round(23.5)/round(5.2)
4.7999999999999998
 >>> round(23.5/5.2)
5.0
 >>> int(round(23.5))/int(round(5.2))
4

Which one of these is correct probably depends on what you are using the 
  result for.  If it's an intermediate calculation you may not want an 
integer (type) result just yet.

Rounding is used in various differing ways in regard to significant 
digits and accuracy.  While it seems like there should be one easy way 
of handling rounding and significant digits, that would be too narrow a 
view.

See the following page (for starters) to see various uses and rules of 
significant digits and rounding and how they change depending on the 
type of calculation and weather or not its a pure or exact number:

     http://www.epcc.edu/faculty/victors/sigfig.htm



I think it's better to view that the common operation (being discussed) 
is rounding when converting to an int, and not converting to an int when 
rounding.  So the standard idiom mentioned earlier by Raymond ...

    int(x+.5)

If it could be made faster by having a round argument on int for such 
operations, it may be worth while.

   int(x, round=mode)

Where mode could be one of 'up','down','standard','toeven','toodd',  or 
whatever are useful methods that might be used.

If there is no performance advantage to combining the two, I think it 
would be better to keep them separate. So I'm -0 on giving the int 
constructor a rounding argument.

Just my 2 cents,
   Ron










From t-bruch at microsoft.com  Tue Aug  1 23:52:42 2006
From: t-bruch at microsoft.com (Bruce Christensen)
Date: Tue, 1 Aug 2006 14:52:42 -0700
Subject: [Python-Dev] Strange memo behavior from cPickle
Message-ID: <3581AA168D87A2479D88EA319BDF7D3201069E15@RED-MSG-80.redmond.corp.microsoft.com>

We seem to have stumbled upon some strange behavior in cPickle's memo
use when pickling instances.

Here's the repro:

[mymodule.py]
class C:
    def __getstate__(self): return ('s1', 's2', 's3')

[interactive interpreter]
Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cPickle
>>> import mymodule
>>> class C:
...     def __getstate__(self): return ('s1', 's2', 's3')
...
>>> for x in mymodule.C(), C(): cPickle.dumps(x)
...
"(imymodule\nC\np1\n(S's1'\nS's2'\np2\nS's3'\np3\ntp4\nb."
"(i__main__\nC\np1\n(S's1'\nS's2'\nS's3'\ntp2\nb."
>>>

Note that the second and third strings in the instance's state are
memoized in the first case, but not in the second. Any idea why this
occurs (and why the first element is never memoized)?

--Bruce

From tim.peters at gmail.com  Wed Aug  2 02:40:13 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 1 Aug 2006 20:40:13 -0400
Subject: [Python-Dev] Strange memo behavior from cPickle
In-Reply-To: <3581AA168D87A2479D88EA319BDF7D3201069E15@RED-MSG-80.redmond.corp.microsoft.com>
References: <3581AA168D87A2479D88EA319BDF7D3201069E15@RED-MSG-80.redmond.corp.microsoft.com>
Message-ID: <1f7befae0608011740t41dc76cchb86124144375fc00@mail.gmail.com>

[Bruce Christensen]
> We seem to have stumbled upon some strange behavior in cPickle's memo
> use when pickling instances.
>
> Here's the repro:
>
> [mymodule.py]
> class C:
>     def __getstate__(self): return ('s1', 's2', 's3')
>
> [interactive interpreter]
> Python 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import cPickle
> >>> import mymodule
> >>> class C:
> ...     def __getstate__(self): return ('s1', 's2', 's3')
> ...
> >>> for x in mymodule.C(), C(): cPickle.dumps(x)
> ...
> "(imymodule\nC\np1\n(S's1'\nS's2'\np2\nS's3'\np3\ntp4\nb."
> "(i__main__\nC\np1\n(S's1'\nS's2'\nS's3'\ntp2\nb."
> >>>
>
> Note that the second and third strings in the instance's state are
> memoized in the first case, but not in the second. Any idea why this
> occurs (and why the first element is never memoized)?

Ideally, a pickle would never contain a `PUT i` unless i was
referenced by a `GET i` later.  So, ideally, there would be no PUT
opcodes in either of these pickles.

cPickle is a little bit smarter than pickle.py here, in that cPickle
suppresses a PUT if the reference count on the object is less than 2
(in which case the structure being pickled can't possibly reference
the sub-object a second time, so it's impossible that a later GET will
want to reference the same sub-object).  So all you're seeing here is
refcount accidents, complicated by accidents concerning exactly which
strings get interned.

Use pickle.py instead (which doesn't do this refcount
micro-optimization), and you'll see the same number of PUTs in both.

They're all correct.  What would be incorrect is seeing a `GET i`
without a preceding `PUT i`   using the same `i`.

From greg.ewing at canterbury.ac.nz  Wed Aug  2 03:02:49 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 02 Aug 2006 13:02:49 +1200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44CF3D96.7090101@egenix.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
Message-ID: <44CFF9B9.3010402@canterbury.ac.nz>

M.-A. Lemburg wrote:

> You often have a need for controlled rounding when doing
> financial calculations 

You should NOT be using binary floats for money
in the first place.

 > or in situations where you want to
 > compare two floats with a given accuracy,

Pseudo-rounding to decimal places is not
the right way to do that. The right way is
to compare the difference to a tolerance.

> e.g. to work
> around rounding problems ;-)

As Tim Peters has pointed out many times,
you can't fix binary/decimal representation
problems by rounding. There are always cases
that give a different result than you would
want.

> Hmm, looks like a YAGNI to me,

In all my programming so far I've found
numerous uses for round-to-int, and exactly
zero uses for round-binary-float-to-decimal-
places. So from my point of view, the YAGNI
applies to the *current* behavour of round()!

--
Greg

From simon at arrowtheory.com  Wed Aug  2 03:58:06 2006
From: simon at arrowtheory.com (Simon Burton)
Date: Wed, 2 Aug 2006 11:58:06 +1000
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44CAD9A1.8080000@canterbury.ac.nz>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz>
Message-ID: <20060802115806.7d568586.simon@arrowtheory.com>

On Sat, 29 Jul 2006 15:44:33 +1200
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Michael Urman wrote:
> > The fact that
> > round returns a float rather than an int, while intentional, does not
> > feature prominently in one's mine when the first version yielded the
> > expected results.
> 
> As an aside, does anyone else think that it would be
> useful to have a builtin which rounds and converts to
> an int in one go? Whenever I use round(), I almost
> always want the result as an int, and making me do
> it in two steps seems unnecessarily bothersome.

It's not even clear to me that int(round(x)) is always the
nearest integer to x.
Is it always true that float(some_int)>=some_int ?  (for positive values).

(ie. I am wondering if truncating the float representation
of an int always gives back the original int).

Simon.

-- 
Simon Burton, B.Sc.
Licensed PO Box 8066
ANU Canberra 2601
Australia
Ph. 61 2 6249 6940
http://arrowtheory.com 

From nnorwitz at gmail.com  Wed Aug  2 06:36:25 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 1 Aug 2006 21:36:25 -0700
Subject: [Python-Dev] internal weakref API should be Py_ssize_t?
Message-ID: <ee2a432c0608012136q4fa232b5rd3b31cba55ad8332@mail.gmail.com>

I'm wondering if the following change should be made to Include/weakrefobject.h:

-PyAPI_FUNC(long) _PyWeakref_GetWeakrefCount(PyWeakReference *head);
+PyAPI_FUNC(Py_ssize_t) _PyWeakref_GetWeakrefCount(PyWeakReference *head);

And the 2 other files which use this (weakref obj and module).  Should
this be changed now, wait for 2.6, or not worry about it?

n

From tim.peters at gmail.com  Wed Aug  2 06:53:59 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 2 Aug 2006 00:53:59 -0400
Subject: [Python-Dev] internal weakref API should be Py_ssize_t?
In-Reply-To: <ee2a432c0608012136q4fa232b5rd3b31cba55ad8332@mail.gmail.com>
References: <ee2a432c0608012136q4fa232b5rd3b31cba55ad8332@mail.gmail.com>
Message-ID: <1f7befae0608012153y7eacfbceke93fcbe1d1c72e32@mail.gmail.com>

[Neal Norwitz]
> I'm wondering if the following change should be made to Include/weakrefobject.h:

Yes.

> -PyAPI_FUNC(long) _PyWeakref_GetWeakrefCount(PyWeakReference *head);
> +PyAPI_FUNC(Py_ssize_t) _PyWeakref_GetWeakrefCount(PyWeakReference *head);
>
> And the 2 other files which use this (weakref obj and module).  Should
> this be changed now, wait for 2.6, or not worry about it?

+1 on biting the bullet for 2.5.

From fdrake at acm.org  Wed Aug  2 07:24:44 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 2 Aug 2006 01:24:44 -0400
Subject: [Python-Dev] internal weakref API should be Py_ssize_t?
In-Reply-To: <1f7befae0608012153y7eacfbceke93fcbe1d1c72e32@mail.gmail.com>
References: <ee2a432c0608012136q4fa232b5rd3b31cba55ad8332@mail.gmail.com>
	<1f7befae0608012153y7eacfbceke93fcbe1d1c72e32@mail.gmail.com>
Message-ID: <200608020124.44806.fdrake@acm.org>

Neal Norwitz wrote:
 > I'm wondering if the following change should be made to
 > Include/weakrefobject.h:

On Wednesday 02 August 2006 00:53, Tim Peters wrote:
 > Yes.
...
 > +1 on biting the bullet for 2.5.

Agreed.  This should definately go with the rest of the Py_ssize_t changes.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From kbk at shore.net  Wed Aug  2 07:49:06 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed, 2 Aug 2006 01:49:06 -0400 (EDT)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200608020549.k725n6cP029299@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  396 open ( -5) /  3354 closed (+12) /  3750 total ( +7)
Bugs    :  864 open (-32) /  6087 closed (+52) /  6951 total (+20)
RFE     :  226 open ( +2) /   234 closed ( +1) /   460 total ( +3)

New / Reopened Patches
______________________

Move firewall warning to "about" menu  (2006-07-26)
       http://python.org/sf/1529018  opened by  Tal Einat

Allowing multiple instances of IDLE with sub-processes  (2006-07-26)
       http://python.org/sf/1529142  opened by  Tal Einat

Squeezer - squeeze large output in the interpreter  (2006-07-27)
       http://python.org/sf/1529353  opened by  Tal Einat

More openbsd targets for ctypes  (2006-07-27)
CLOSED http://python.org/sf/1529514  opened by  Thomas Heller

test_defaultdict not integrated into test sute  (2006-07-27)
CLOSED http://python.org/sf/1529686  opened by  ?iga Seilnacht

misleading info about tarfile.py's "r|*" mode  (2006-07-27)
CLOSED http://python.org/sf/1529811  opened by  Lars Gust?bel

pydoc render_doc  (2006-07-28)
       http://python.org/sf/1530482  opened by  ganges master

test_email_codecs not integrated into test sute  (2006-07-27)
CLOSED http://python.org/sf/1529686  reopened by  zseil

Fix __index__() clipping really big numbers  (2006-07-29)
       http://python.org/sf/1530738  opened by  Nick Coghlan

"a +  (2006-07-29)
CLOSED http://python.org/sf/1531113  opened by  Christopher Tur Lesniewski-Laas

Tracing and profiling functions can cause hangs in threads  (2006-07-31)
       http://python.org/sf/1531859  opened by  Matt Fleming

Support for case-insensitivity in string.Template  (2006-07-25)
CLOSED http://python.org/sf/1528167  reopened by  whit537

Patches Closed
______________

Expose case-insensitivity of string.Template  (2006-07-25)
       http://python.org/sf/1528167  closed by  bwarsaw

Fix for #1513611 and #1511497; xml.sax imports  (2006-07-10)
       http://python.org/sf/1519796  closed by  zseil

More openbsd targets for ctypes  (2006-07-27)
       http://python.org/sf/1529514  closed by  theller

test_defaultdict not integrated into test sute  (2006-07-27)
       http://python.org/sf/1529686  closed by  gbrandl

misleading info about tarfile.py's "r|*" mode  (2006-07-27)
       http://python.org/sf/1529811  closed by  akuchling

Support for PyGetSetDefs in pydoc, inspect, and types  (2006-07-10)
       http://python.org/sf/1520294  closed by  bwarsaw

Fix socketmodule compile on NetBSD  (2006-07-21)
       http://python.org/sf/1526460  closed by  splitscreen

test_email_codecs not integrated into test sute  (2006-07-27)
       http://python.org/sf/1529686  closed by  gbrandl

Describe Py_DEBUG and friends...  (2006-05-18)
       http://python.org/sf/1490989  closed by  akuchling

os.path.exists returns False if no permission  (2004-11-17)
       http://python.org/sf/1068277  closed by  akuchling

"a +  (2006-07-29)
       http://python.org/sf/1531113  closed by  nnorwitz

Fix zip file header format in zipfile.py  (2003-07-30)
       http://python.org/sf/780595  closed by  gbrandl

Don't produce core file in test_subprocess.py  (2006-07-11)
       http://python.org/sf/1520905  closed by  akuchling

New / Reopened Bugs
___________________

readline module description should mention raw_input()  (2006-07-26)
CLOSED http://python.org/sf/1529157  opened by  Thom Harp

possible bug in mystrtol.c with recent gcc  (2006-07-13)
CLOSED http://python.org/sf/1521947  reopened by  nnorwitz

unrelated variable messing up doctests  (2006-07-26)
CLOSED http://python.org/sf/1529297  reopened by  macquigg

BROWSER path with capital letters  (2006-07-28)
       http://python.org/sf/1529655  opened by  Ken McGaugh

doc string of read-only properties  (2006-07-27)
CLOSED http://python.org/sf/1529717  opened by  Michael Amrhein

distutils regression related to Distribution.run_command  (2006-07-27)
CLOSED http://python.org/sf/1529871  opened by  Collin Winter

bz2 lib missing from source distribution  (2006-07-27)
CLOSED http://python.org/sf/1529998  opened by  james yoo

Mac Universal Build of Python confuses distutils  (2006-07-28)
       http://python.org/sf/1530142  opened by  Richard Jones

ssl object documentation lacks a couple of methods  (2006-07-28)
CLOSED http://python.org/sf/1530382  opened by  Lawrence Oluyede

file.seek() influelce write() when opened with a+ mode  (2006-07-12)
       http://python.org/sf/1521491  reopened by  rudnik_lior

ctypes build fails on Solaris 10  (2006-07-28)
       http://python.org/sf/1530448  opened by  Skip Montanaro

struct.pack raises TypeError where it used to convert  (2006-07-28)
       http://python.org/sf/1530559  opened by  Joe Wreschnig

distutils doesn't notice --with-pydebug  (2006-07-29)
       http://python.org/sf/1530959  opened by  Collin Winter

__weaklist__ in typeobject.c  (2006-07-29)
       http://python.org/sf/1531003  opened by  Fred L. Drake, Jr.

Comman not allowed at the end of argument list for **argumen  (2006-07-29)
       http://python.org/sf/1531016  opened by  Roman Suzi

Reflected (swapped) operations overridden by subclasses  (2006-07-31)
CLOSED http://python.org/sf/1531349  opened by  Andrei Polushin

format_exception raises if str(exception) raises  (2006-07-30)
       http://python.org/sf/1531405  opened by  Jp Calderone

parsetok.c emits warnings by writing to stderr  (2006-07-30)
       http://python.org/sf/1531415  opened by  Jp Calderone

Build fails on MacOSX with missing symbol  (2006-07-31)
       http://python.org/sf/1531662  opened by  gideon may

HTTPSConnection request hangs  (2006-07-31)
       http://python.org/sf/1531775  opened by  Noam Taich

subprocess.Popen(cmd, stdout=sys.stdout) fails  (2006-07-31)
       http://python.org/sf/1531862  opened by  John A Meinel

SocketServer.TCPServer returns different ports  (2006-07-31)
       http://python.org/sf/1531963  opened by  hakker.de

the csv module writes files that Excel sees as SYLK files  (2006-08-01)
       http://python.org/sf/1532483  opened by  Vincent Povirk

incorrect behaviour of PyUnicode_EncodeMBCS?  (2006-08-01)
       http://python.org/sf/1532726  opened by  Jan-Willem

Bugs Closed
___________

Build fails on OS X with case sensitive fs  (2006-07-19)
       http://python.org/sf/1525447  closed by  ronaldoussoren

readline module description should mention raw_input()  (2006-07-26)
       http://python.org/sf/1529157  closed by  akuchling

possible bug in mystrtol.c with recent gcc  (2006-07-13)
       http://python.org/sf/1521947  closed by  nnorwitz

xml.sax.ParseException weirdness in python 2.5b1  (2006-06-27)
       http://python.org/sf/1513611  closed by  fdrake

Python 2.4c1 test_univnewlines.py fails within IDLE IDE  (2004-11-19)
       http://python.org/sf/1069433  closed by  kbk

unrelated variable messing up doctests  (2006-07-26)
       http://python.org/sf/1529297  closed by  nnorwitz

unrelated variable messing up doctests  (2006-07-26)
       http://python.org/sf/1529297  closed by  tim_one

wrong handling of character buffers in stringobject  (2006-07-01)
       http://python.org/sf/1515471  closed by  nnorwitz

xml.sax.expatreader is missing  (2006-06-23)
       http://python.org/sf/1511497  closed by  fdrake

FTP modue functions are not re-entrant,give odd exceptions  (2006-04-13)
       http://python.org/sf/1469557  closed by  gbrandl

turtle.py Docs still incomplete  (2006-07-09)
       http://python.org/sf/1519571  closed by  akuchling

doc string of read-only properties  (2006-07-27)
       http://python.org/sf/1529717  closed by  gbrandl

distutils regression related to Distribution.run_command  (2006-07-27)
       http://python.org/sf/1529871  closed by  pje

bz2 lib missing from source distribution  (2006-07-27)
       http://python.org/sf/1529998  closed by  nnorwitz

missing module names in email package  (2005-12-27)
       http://python.org/sf/1391608  closed by  bwarsaw

mimetools message's To field can't be changed  (2006-06-28)
       http://python.org/sf/1513913  closed by  bwarsaw

Docs on import of email.MIMExxx classes wrong  (2006-05-04)
       http://python.org/sf/1481650  closed by  bwarsaw

email.Utils.py: UnicodeError in RFC2322 header  (2006-01-24)
       http://python.org/sf/1414018  closed by  bwarsaw

ssl object documentation lacks a couple of methods  (2006-07-28)
       http://python.org/sf/1530382  closed by  akuchling

new.function() docs out of date  (2003-11-03)
       http://python.org/sf/835255  closed by  gbrandl

compiler module loses docstrings  (2006-03-02)
       http://python.org/sf/1441397  closed by  gbrandl

inconsistency in help(set)  (2006-01-25)
       http://python.org/sf/1414697  closed by  akuchling

xmlcore needs to be documented  (2006-06-11)
       http://python.org/sf/1504456  closed by  fdrake

urllib2 data argument  (2006-07-25)
       http://python.org/sf/1528258  closed by  akuchling

Finish PEP 343 terminology cleanup  (2006-04-26)
       http://python.org/sf/1476845  closed by  ncoghlan

set documentation deficiencies  (2006-02-10)
       http://python.org/sf/1429053  closed by  akuchling

Weakref types documentation bugs  (2005-05-02)
       http://python.org/sf/1193966  closed by  fdrake

suspect documentation of operator.isNumberType   (2004-01-17)
       http://python.org/sf/878908  closed by  gbrandl

Wrong documentation of __radd__ etc.  (2004-08-03)
       http://python.org/sf/1002453  closed by  gbrandl

Documented grammar for List displays incorrect (testlist)  (2004-11-22)
       http://python.org/sf/1071248  closed by  gbrandl

os.path.sameopenfile documentation wrong.  (2004-08-03)
       http://python.org/sf/1002398  closed by  gbrandl

mode 't' not documented as posssible mode for file built-in  (2005-12-15)
       http://python.org/sf/1381717  closed by  gbrandl

Strange os.path.exists() results with invalid chars  (2005-04-30)
       http://python.org/sf/1193180  closed by  gbrandl

popen3 on windows loses environment variables  (2005-01-13)
       http://python.org/sf/1101667  closed by  gbrandl

Reflected (swapped) operations overridden by subclasses  (2006-07-30)
       http://python.org/sf/1531349  closed by  gbrandl

String Methods 2.3.6.1 not in TOC  (2006-06-29)
       http://python.org/sf/1514540  closed by  akuchling

PEP 307 pickle extensions not documented  (2004-07-04)
       http://python.org/sf/984952  closed by  akuchling

4.2.6 (re) Examples: float regexp exponential on failure  (2003-11-24)
       http://python.org/sf/848556  closed by  akuchling

pydoc fails on package in ZIP archive  (2006-06-15)
       http://python.org/sf/1506951  closed by  gbrandl

Installing 2.3b2 on to non-system disk fails  (2003-07-03)
       http://python.org/sf/764975  closed by  ronaldoussoren

New / Reopened RFE
__________________

distutils 'register' command and windows home directories  (2006-07-30)
       http://python.org/sf/1531505  opened by  Josiah Carlson

RFE Closed
__________

No simple example for pickle  (2006-03-01)
       http://python.org/sf/1441072  closed by  gbrandl


From theller at python.net  Wed Aug  2 10:00:25 2006
From: theller at python.net (Thomas Heller)
Date: Wed, 02 Aug 2006 10:00:25 +0200
Subject: [Python-Dev] Releasemanager, please approve #1532975
Message-ID: <eapm2p$mnm$1@sea.gmane.org>

Approval requested for patch:
http://python.org/sf/1532975

Thanks,

Thomas


From nmm1 at cus.cam.ac.uk  Wed Aug  2 11:09:30 2006
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Wed, 02 Aug 2006 10:09:30 +0100
Subject: [Python-Dev] Rounding float to int directly ...
Message-ID: <E1G8CjK-0002fL-9l@virgo.cus.cam.ac.uk>

Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> You should NOT be using binary floats for money
> in the first place.

Or floating-point at all, actually.  But binary floating-point is
definitely unsuited for such a use.

> Pseudo-rounding to decimal places is not
> the right way to do that. The right way is
> to compare the difference to a tolerance.

Right.  Where the tolerance should be a combination of relative
and absolute accuracy.  1.0e-300 should usually be 'similar' to 0.0.


Simon Burton <simon at arrowtheory.com> wrote:
>
> It's not even clear to me that int(round(x)) is always the
> nearest integer to x.

There is a sense in which this is either true or overflow occurs.

> Is it always true that float(some_int)>=some_int ?  (for positive values).
>
> (ie. I am wondering if truncating the float representation
> of an int always gives back the original int).

No.

Consider 'standard' Python representations on a 64-bit system.  There
are only 53 bits in the mantissa, but an integer can have up to 63.
Very large integers need to be rounded, and can be rounded up or down.


Please note that I am not arguing against an int_rounded() function.
There is as much reason to want one as an int_truncated() one, but
there is no very good reason to to want more than one of the two.
int_expanded() [i.e. ceiling] is much less useful.

For people interested in historical trivia, the dominance of the
truncating form of integer conversion over the rounding form seems to
be yet another side-effect of the Fortran / IBM 370 dominance over
the Algol / other hardware, despite the fact that most modern
languages are rooted in CPL rather than Fortran.  I am unaware of
any technical grounds to prefer one over the other (i.e. the reasons
for wanting each are equally balanced).

It all comes down to the simple question "Do we regard a single
primitive for int(round()) as important enough to provide?"

I abstain :-)


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From mcherm at mcherm.com  Wed Aug  2 12:53:38 2006
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed, 02 Aug 2006 03:53:38 -0700
Subject: [Python-Dev] Rounding float to int directly ...
Message-ID: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com>

Greg Ewing writes:
> [I propose for Py3k that] builtin round()
> to return an int, and have something somewhere
> else, such as math.fround(), for round-to-float.

+1.

The need to convert float to int is pervasive. The need to round (binary)
floats to integer (or other decimal) floats is _much_ less common.
Furthermore, the need for "round to nearest int" is much more common
than the need for "integer floor" (although as pointed out, each could
be defined in terms of the other except for certain rules for rounding
exact halves). And most telling of all, if I were to stop 10 random
programmers on the street, most of them not familiar with Python, and
ask them to guess what the function named "round" did, I'm guessing
the majority would think of float-to-int. It's a nice feature of Python
that "it just works".


Marc-Andre Lemburg writes:
> You often have a need for controlled rounding when doing
> financial calculations or [other reason snipped]

Hmm. Not in the banks _I_ have worked at! We *never* use binary
floating point for money. The decimal class is fairly useful in
that regard.

Nick Maclaren write:
> Decimal doesn't even help people who care about accuracy.

Not true! The float class is incapable of maintaining 700 digits of
precision, but Decimal handles it just fine. (Why you would WANT
more than around 53 bits of precision is a different question, but
Decimal handles it.)

> People who care about the accuracy of calculations prefer binary,
> as it is a more accurate model.

Huh? That doesn't even make sense! A model is not inherently accurate
or inaccurate, it is only an accurate or inaccurate representation
of some "real" system. Neither binary nor decimal is a better
representation of either rational or real numbers, the first
candidates for "real" system I thought of. Financial accounting rules
tend to be based on paper-and-pencil calculations for which
decimal is usually a far better representation.

If you had said that binary floats squeeze out more digits of
precision per bit of storage than decimal floats, or that binary
floats are faster because they are supported by specialized hardware,
then I'd go along, but they're not a "better model".

-- Michael Chermside




From Sjaak.van.Werkhoven at nl.fortis.com  Wed Aug  2 13:01:36 2006
From: Sjaak.van.Werkhoven at nl.fortis.com (Werkhoven J.P. van (Sjaak))
Date: Wed, 2 Aug 2006 13:01:36 +0200 
Subject: [Python-Dev] FW: using globals
Message-ID: <AB7FC09A43F1EB46B9A11FB45F2B2FD507148EE0@dbacaa45.nl.fortis.bank>

> Hi,
> 
> I have got a problem with importing global variables. For instance I have
> got two files:
> 
> # t1.py							#t2.py
> 
> counter = 1				
> 
> def counter_adder(filenr):					def
> show_adder():
>    global counter						    import
> t1
>    counter+= 1
> print("adder is %d" % t1.counter)
>    if filenr == 1:
>       show_adder()
>    else:
>       import t2
>       t2.show_adder()
> 
> def show_adder():
>    print("adder is %d" % counter)
> 
> >>counter_adder(1)
> adder is 2
> >>counter_adder(2)
> adder is 1
> 
> When I look at the outcome of two function calls I expected no differences
> but much to my surprise it goes wrong when the global variable is
> imported. It doesn't give the updated value but the value it get when
> instantiated. Who come? What should I do to get the updated value
> 
> greetings,
> 
> Sjaak van Werkhoven
>  
> 


****************************DISCLAIMER***********************************
Deze e-mail is uitsluitend bestemd voor de geadresseerde(n). Verstrekking aan en gebruik door anderen is niet toegestaan. Fortis sluit iedere aansprakelijkheid uit die voortvloeit uit electronische verzending. 

This e-mail is intended exclusively for the addressee(s), and may not be passed on to, or made available for use by any person other than the addressee(s). Fortis rules out any and every liability resulting from any electronic transmission. 
******************************************************************************

From nmm1 at cus.cam.ac.uk  Wed Aug  2 13:03:53 2006
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Wed, 02 Aug 2006 12:03:53 +0100
Subject: [Python-Dev] Rounding float to int directly ...
Message-ID: <E1G8EW1-0003oe-1R@virgo.cus.cam.ac.uk>

Michael Chermside <mcherm at mcherm.com> wrote:
>
> > Decimal doesn't even help people who care about accuracy.
>
> Not true! The float class is incapable of maintaining 700 digits of
> precision, but Decimal handles it just fine. (Why you would WANT
> more than around 53 bits of precision is a different question, but
> Decimal handles it.)

Oh, yes, the CURRENT decimal class is potentially more accurate than
the CURRENT floating class, but that has nothing to do with the
intrinsic differences in the base.

> > People who care about the accuracy of calculations prefer binary,
> > as it is a more accurate model.
> 
> Huh? That doesn't even make sense! A model is not inherently accurate
> or inaccurate, it is only an accurate or inaccurate representation
> of some "real" system. Neither binary nor decimal is a better
> representation of either rational or real numbers, the first
> candidates for "real" system I thought of. Financial accounting rules
> tend to be based on paper-and-pencil calculations for which
> decimal is usually a far better representation.
> 
> If you had said that binary floats squeeze out more digits of
> precision per bit of storage than decimal floats, or that binary
> floats are faster because they are supported by specialized hardware,
> then I'd go along, but they're not a "better model".

No, that isn't true.  The "wobbling precision" effect may be overstated,
but is real, and gets worse the larger the base is.  To the best of my
knowledge, that is almost the only way in which binary is more accurate
than decimal, in absolute terms, and it is a marginal difference.  Note
that I said "prefer", not "require" :-)

For example, calculating the relative difference between two close
numbers is sensitive to whether you are using the numbers in their
normal or inverse forms (by a factor on N in base N), and this is a
common cause of incorrect answers.  A factor of 2 is better than one
of 10.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From mcherm at mcherm.com  Wed Aug  2 13:18:34 2006
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed, 02 Aug 2006 04:18:34 -0700
Subject: [Python-Dev] Rounding float to int directly ...
Message-ID: <20060802041834.1400idlbu6o0wk08@login.werra.lunarpages.com>

Nick Maclaren:
> The "wobbling precision" effect may be overstated,
> but is real, and gets worse the larger the base is.  To the best of my
> knowledge, that is almost the only way in which binary is more accurate
> than decimal, in absolute terms, and it is a marginal difference.

Okay, that makes sense.

However, if I understand "wobbling precision" correctly (and it's a BIG
if) then it refers to the fact that interpolating from the last digit of
precision doesn't work so well (intentionally vague phrase). So I
would think (but could be wrong) that it is always trumped by adding
another digit of precision to your numbers / lookup tables / whatever.
If so, then even more credit needs to be given to the system that
supports adjustable precision (the Decimal module).

-- Michael Chermside


From ronaldoussoren at mac.com  Wed Aug  2 13:30:57 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 02 Aug 2006 13:30:57 +0200
Subject: [Python-Dev] FW: using globals
In-Reply-To: <AB7FC09A43F1EB46B9A11FB45F2B2FD507148EE0@dbacaa45.nl.fortis.bank>
References: <AB7FC09A43F1EB46B9A11FB45F2B2FD507148EE0@dbacaa45.nl.fortis.bank>
Message-ID: <13237035.1154518257044.JavaMail.ronaldoussoren@mac.com>

 
On Wednesday, August 02, 2006, at 01:02PM, Werkhoven J.P. van (Sjaak) <Sjaak.van.Werkhoven at nl.fortis.com> wrote:

>> Hi,
>> 
>> I have got a problem with importing global variables. For instance I have
>> got two files:

Your message is off-topic for python-dev, this list is for the development OF python, not development WITH python. You're much likier to get an answer on python-list or comp.lang.python (which are the same).

BTW. The formatting of the source-code in you're e-mail was such that I have no idea how it is supposed to look :-(

Regards,
  Ronald

From ncoghlan at gmail.com  Wed Aug  2 14:02:41 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 02 Aug 2006 22:02:41 +1000
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com>
References: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com>
Message-ID: <44D09461.7060308@gmail.com>

Michael Chermside wrote:
> Greg Ewing writes:
>> [I propose for Py3k that] builtin round()
>> to return an int, and have something somewhere
>> else, such as math.fround(), for round-to-float.
> 
> +1.
> 
> The need to convert float to int is pervasive. The need to round (binary)
> floats to integer (or other decimal) floats is _much_ less common.
> Furthermore, the need for "round to nearest int" is much more common
> than the need for "integer floor" (although as pointed out, each could
> be defined in terms of the other except for certain rules for rounding
> exact halves). And most telling of all, if I were to stop 10 random
> programmers on the street, most of them not familiar with Python, and
> ask them to guess what the function named "round" did, I'm guessing
> the majority would think of float-to-int. It's a nice feature of Python
> that "it just works".

The potential issue with accidentally triggering truncating division also goes 
away in Py3k, so that's another point in favour of switching round() to return 
an integer result.

Another option would be to provide this as a method of float objects (similar 
to decimal).


> Nick Maclaren write:
>> Decimal doesn't even help people who care about accuracy.
> 
> Not true! The float class is incapable of maintaining 700 digits of
> precision, but Decimal handles it just fine. (Why you would WANT
> more than around 53 bits of precision is a different question, but
> Decimal handles it.)

The decimal module's default settings actually provide 93 bits or so of 
precision (28 decimal digits).

Sure, there may still be cases where you have to care about dynamic range, but 
if two numbers are separated by 28 orders of magnitude the smaller one really 
is pretty damn close to zero as far as the second one is concerned (in a fixed 
point system, the second one usually *would* be zero).

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mal at egenix.com  Wed Aug  2 14:28:12 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 02 Aug 2006 14:28:12 +0200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and	coercing floats to integers)
In-Reply-To: <44CFF9B9.3010402@canterbury.ac.nz>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>	<44CAD9A1.8080000@canterbury.ac.nz>
	<44CE4D32.6040909@ewtllc.com>	<44CEA091.1010406@canterbury.ac.nz>
	<44CF20EE.70103@egenix.com>	<44CF38A7.8040304@canterbury.ac.nz>
	<44CF3D96.7090101@egenix.com> <44CFF9B9.3010402@canterbury.ac.nz>
Message-ID: <44D09A5C.4030905@egenix.com>

Greg Ewing wrote:
> M.-A. Lemburg wrote:
> 
>> You often have a need for controlled rounding when doing
>> financial calculations 
> 
> You should NOT be using binary floats for money
> in the first place.

Believe me: you have to if you want to do more
advanced calculus related to pricing and risk
analysis of derivatives.

>> or in situations where you want to
>> compare two floats with a given accuracy,
> 
> Pseudo-rounding to decimal places is not
> the right way to do that. The right way is
> to compare the difference to a tolerance.

This is not the same: if you round both value
and then compare, you test a different interval
than by taking the difference and applying
a tolerance value comparison:

>>> x = 1.12
>>> y = 1.124999
>>> round(x,2) == round(y,2)
True
>>> abs(x-y) < 1e-2
True

yet:

>>> y = 1.1275
>>> round(x,2) == round(y,2)
False
>>> abs(x-y) < 1e-2
True

>> e.g. to work
>> around rounding problems ;-)
> 
> As Tim Peters has pointed out many times,
> you can't fix binary/decimal representation
> problems by rounding. There are always cases
> that give a different result than you would
> want.

Right, but in most real-world cases it works :-)

>> Hmm, looks like a YAGNI to me,
> 
> In all my programming so far I've found
> numerous uses for round-to-int, and exactly
> zero uses for round-binary-float-to-decimal-
> places. So from my point of view, the YAGNI
> applies to the *current* behavour of round()!

Most typical uses of round() don't use the
optional argument, true, but I still fail
to see what returning an integer instead of
a float would buy you.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 02 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From anthony at interlink.com.au  Wed Aug  2 14:29:44 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 2 Aug 2006 22:29:44 +1000
Subject: [Python-Dev] TRUNK FREEZE 2006-07-03, 00:00 UTC for 2.5b3
Message-ID: <200608022229.46957.anthony@interlink.com.au>

The trunk is FROZEN from 00:00 UTC/GMT on 03 July 2006. That's in about 12 
hours from now. This is for the third (and final, hopefully!) beta of 2.5. 

Please help the release team out by not making checkins on the trunk after 
that time until I send out an email that the release is done. 

Thanks,
Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From exarkun at divmod.com  Wed Aug  2 14:36:54 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 2 Aug 2006 08:36:54 -0400
Subject: [Python-Dev] TRUNK FREEZE 2006-07-03, 00:00 UTC for 2.5b3
In-Reply-To: <200608022229.46957.anthony@interlink.com.au>
Message-ID: <20060802123654.1717.1171821069.divmod.quotient.12149@ohm>

On Wed, 2 Aug 2006 22:29:44 +1000, Anthony Baxter <anthony at interlink.com.au> wrote:
>The trunk is FROZEN from 00:00 UTC/GMT on 03 July 2006. That's in about 12
>hours from now. This is for the third (and final, hopefully!) beta of 2.5.
>
>Please help the release team out by not making checkins on the trunk after
>that time until I send out an email that the release is done.

Does this mean that there is no further opportunity to resolve regressions
such as <http://sourceforge.net/tracker/index.php?func=detail&aid=1531405&group_id=5470&atid=105470>, or will changes be allowed between b3 and final?

Jean-Paul

From skip at pobox.com  Wed Aug  2 14:47:32 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 2 Aug 2006 07:47:32 -0500
Subject: [Python-Dev] TRUNK FREEZE 2006-07-03, 00:00 UTC for 2.5b3
In-Reply-To: <20060802123654.1717.1171821069.divmod.quotient.12149@ohm>
References: <200608022229.46957.anthony@interlink.com.au>
	<20060802123654.1717.1171821069.divmod.quotient.12149@ohm>
Message-ID: <17616.40676.620565.678809@montanaro.dyndns.org>

    >> Please help the release team out by not making checkins on the trunk
    >> after that time until I send out an email that the release is done.

    Jean-Paul> Does this mean that there is no further opportunity to
    Jean-Paul> resolve regressions such as
    Jean-Paul> <http://sourceforge.net/tracker/index.php?func=detail&aid=1531405&group_id=5470&atid=105470>,
    Jean-Paul> or will changes be allowed between b3 and final?

The latter.  One would hope bugfixes only.

Skip

From mal at egenix.com  Wed Aug  2 14:48:13 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 02 Aug 2006 14:48:13 +0200
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com>
References: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com>
Message-ID: <44D09F0D.5090802@egenix.com>

Michael Chermside wrote:
> Marc-Andre Lemburg writes:
>> You often have a need for controlled rounding when doing
>> financial calculations or [other reason snipped]
> 
> Hmm. Not in the banks _I_ have worked at! We *never* use binary
> floating point for money. The decimal class is fairly useful in
> that regard.

For accounting, I agree - banks would rather not lose too many
cents :-)

However, banks also tend to do more exotic things, esp. in investment
banking, where prices are not determined by taking the sum of a list
of values, but rather turn out to be the result of some calculus.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 02 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bjourne at gmail.com  Wed Aug  2 18:06:40 2006
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Wed, 2 Aug 2006 18:06:40 +0200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
	and coercing floats to integers)
In-Reply-To: <44D09A5C.4030905@egenix.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
	<44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com>
Message-ID: <740c3aec0608020906q437927ddp175d6843719d26d0@mail.gmail.com>

On 8/2/06, M.-A. Lemburg <mal at egenix.com> wrote:
> Greg Ewing wrote:
> > In all my programming so far I've found
> > numerous uses for round-to-int, and exactly
> > zero uses for round-binary-float-to-decimal-
> > places. So from my point of view, the YAGNI
> > applies to the *current* behavour of round()!
>
> Most typical uses of round() don't use the
> optional argument, true, but I still fail
> to see what returning an integer instead of
> a float would buy you.

One hundred and sixty two type conversions!

http://www.google.se/search?hl=sv&q=filetype%3Apy+%22int%28round%22&btnG=S%C3%B6k&meta=

-- 
mvh Bj?rn

From nnorwitz at gmail.com  Wed Aug  2 19:26:21 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 2 Aug 2006 10:26:21 -0700
Subject: [Python-Dev] TRUNK FREEZE 2006-07-03, 00:00 UTC for 2.5b3
In-Reply-To: <20060802123654.1717.1171821069.divmod.quotient.12149@ohm>
References: <200608022229.46957.anthony@interlink.com.au>
	<20060802123654.1717.1171821069.divmod.quotient.12149@ohm>
Message-ID: <ee2a432c0608021026pb283308u427705c60b44188a@mail.gmail.com>

This looks like it needs to be fixed as it's a regression.  As Skip
said, bug fixes are allowed.  There will still be at least one release
candidate.  It's looking like c1 will be around Aug 18 and final
around Sept 12.  I'll update the PEP when I get a chance and confirm
all the dates.

n
--

On 8/2/06, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> On Wed, 2 Aug 2006 22:29:44 +1000, Anthony Baxter <anthony at interlink.com.au> wrote:
> >The trunk is FROZEN from 00:00 UTC/GMT on 03 July 2006. That's in about 12
> >hours from now. This is for the third (and final, hopefully!) beta of 2.5.
> >
> >Please help the release team out by not making checkins on the trunk after
> >that time until I send out an email that the release is done.
>
> Does this mean that there is no further opportunity to resolve regressions
> such as <http://sourceforge.net/tracker/index.php?func=detail&aid=1531405&group_id=5470&atid=105470>, or will changes be allowed between b3 and final?
>
> Jean-Paul
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>

From guido at python.org  Wed Aug  2 19:58:20 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 2 Aug 2006 10:58:20 -0700
Subject: [Python-Dev] Bad interaction of __index__ and sequence repeat
In-Reply-To: <44CF7676.2010102@gmail.com>
References: <20060728101133.GA339@code0.codespeak.net>
	<44CB6B7D.1040001@gmail.com> <44CB92F1.9040304@iinet.net.au>
	<44CE4BB9.60903@ieee.org> <44CF3451.4060500@gmail.com>
	<44CF7676.2010102@gmail.com>
Message-ID: <ca471dc20608021058g265528d7nda2ca60da439180c@mail.gmail.com>

All, please decide without me.

On 8/1/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Nick Coghlan wrote:
> > Travis Oliphant wrote:
> >>> Probably the most interesting thing now would be for Travis to review
> >>> it, and see whether it makes things easier to handle for the Numeric
> >>> scalar types (given the amount of code the patch deleted from the
> >>> builtin and standard library data types, hopefully the benefits to
> >>> Numeric will be comparable).
> >>
> >> I noticed most of the checks for PyInt where removed in the patch.  If I
> >> remember correctly, I left these in for "optimization."   Other than
> >> that, I think the patch is great.
> >
> > You're right - there was a fast path based on PyInt_Check in
> > _PyEval_SliceIndex that got lost, which I'll add back in. I'll also add
> fast
> > paths for PyInt_Check to the functions in abstract.c, too.
> >
> > The other PyInt_Check's (in slot_nb_index and instance_index) were there
> to
> > check that __index__ returned the right thing. The check was still there
> in
> > slot_nb_index, but I'd incorrectly removed it from instance_index. I'll
> add
> > that one back in, too.
> >
> > Once that's done, I'll update the tracker item and reassign to Tim for a
> review.
>
> Aside from the 'assign to Tim' part, this is done now (pep357-fix-v3 on the
> tracker item). I also tweaked the clipping version of the C API to
> optionally
> expose the overflow flag that abstract.c was using internally so the client
> code can find out whether or not clipping actually occurred.
>
> The patch also updates the index unit tests to:
>    - cover the various type errors that Travis picked up
>    - actually run the full set of unit tests under -O
>    - only run the basic sequence independent tests once (instead of once per
> sequence type)
>
> Neal also had a number of questions that I responded to in the tracker
> comments.
>
> The most significant question related to the API signature, which Neal
> considered potentially confusing:
>    PyNumber_Index(PyObject *item, int *is_index)
>
> I added the is_index flag to the function signatures instead of providing a
> separate PyNumber_IndexCheck function in order to avoid doing the same type
> check twice in the typical mp_subscript implementation cases.
>
> One possibility would be to invert the sense of that flag and call it
> "typeerror", which probably more accurately reflects what it's intended for
> -
> it's a way of telling the function "if this object does not have the correct
> type, tell me by setting this flag instead of by setting the Python error
> state".
>
> Regards,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>              http://www.boredomandlaziness.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From oliphant.travis at ieee.org  Wed Aug  2 21:15:24 2006
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Wed, 02 Aug 2006 13:15:24 -0600
Subject: [Python-Dev] Bad interaction of __index__ and sequence repeat
In-Reply-To: <44CF7676.2010102@gmail.com>
References: <20060728101133.GA339@code0.codespeak.net>	<44CB6B7D.1040001@gmail.com>	<44CB92F1.9040304@iinet.net.au>	<44CE4BB9.60903@ieee.org>	<44CF3451.4060500@gmail.com>
	<44CF7676.2010102@gmail.com>
Message-ID: <eaqtkc$5sj$1@sea.gmane.org>

Nick Coghlan wrote:
> 
> 
> One possibility would be to invert the sense of that flag and call it 
> "typeerror", which probably more accurately reflects what it's intended for - 
> it's a way of telling the function "if this object does not have the correct 
> type, tell me by setting this flag instead of by setting the Python error state".

+1 on changing the name (type_error might be a better spelling)

-Travis


From simon at arrowtheory.com  Thu Aug  3 03:39:26 2006
From: simon at arrowtheory.com (Simon Burton)
Date: Thu, 3 Aug 2006 11:39:26 +1000
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <324634B71B159D469BCEB616678A6B94F94B1A@ingdexs1.ingdirect.com>
References: <324634B71B159D469BCEB616678A6B94F94B1A@ingdexs1.ingdirect.com>
Message-ID: <20060803113926.39fd374a.simon@arrowtheory.com>

On Wed, 2 Aug 2006 07:21:02 -0400
"Chermside, Michael" <mchermside at ingdirect.com> wrote:

> 
> You wrote:
> > (ie. I am wondering if truncating the float representation
> > of an int always gives back the original int).
> 
> Yes it does, because all integers that can be expressed as floats
> can be expressed EXACTLY as floats.

Yes, OK this makes sense.

My point is that, regardless of this answer, as a python user I should not
be worried by such details. At least: the doc string for round() should be
expanded to mention the int(round(x)) use case.

Simon.

-- 
Simon Burton, B.Sc.
Licensed PO Box 8066
ANU Canberra 2601
Australia
Ph. 61 2 6249 6940
http://arrowtheory.com 

From greg.ewing at canterbury.ac.nz  Thu Aug  3 04:03:19 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Aug 2006 14:03:19 +1200
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: <E1G8CjK-0002fL-9l@virgo.cus.cam.ac.uk>
References: <E1G8CjK-0002fL-9l@virgo.cus.cam.ac.uk>
Message-ID: <44D15967.5090603@canterbury.ac.nz>

Nick Maclaren wrote:
> I am unaware of
> any technical grounds to prefer one over the other (i.e. the reasons
> for wanting each are equally balanced).

If they're equally balanced in the sense of there
being equal numbers of use cases for both, that
would seem to be a reason to *have* both.

What's the feeling about this? If, e.g. int()
were changed in Py3k to round instead of truncate,
would it cause anyone substantial pain?

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Aug  3 04:14:53 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Aug 2006 14:14:53 +1200
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com>
References: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com>
Message-ID: <44D15C1D.5030800@canterbury.ac.nz>

Michael Chermside wrote:
> (Why you would WANT
> more than around 53 bits of precision is a different question, but
> Decimal handles it.)

You can't have travelled far beyond Alpha Centauri
recently. The major interstellar banking corporations
need *quite* a lot of bits for dealing with the
economy of a medium-sized galaxy. The Decimal module
has been a godsend to them -- they were all making
do with long integers until that came along.

Of course they've had to make some modifications
to handle the number bases used by some of the
more unusual species out there, such as the
13-fingered Quezorpians, but that's just a simple
matter of programming.

Always remember where your towel is,
Zaphod

From greg.ewing at canterbury.ac.nz  Thu Aug  3 04:21:05 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Aug 2006 14:21:05 +1200
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: <44D09461.7060308@gmail.com>
References: <20060802035338.fxzcn0gcvsv4kso4@login.werra.lunarpages.com>
	<44D09461.7060308@gmail.com>
Message-ID: <44D15D91.6000401@canterbury.ac.nz>

Nick Coghlan wrote:

> Another option would be to provide this as a method of float objects 
> (similar to decimal).

That wouldn't be so good. Either kind of rounding ought to be
a function that you can apply without knowing what kind of
number you've got -- int, float, Decimal, or something else.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Aug  3 04:30:12 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Aug 2006 14:30:12 +1200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44D09A5C.4030905@egenix.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
	<44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com>
Message-ID: <44D15FB4.5020605@canterbury.ac.nz>

M.-A. Lemburg wrote:

> Believe me: you have to if you want to do more
> advanced calculus related to pricing and risk
> analysis of derivatives.

When you do things like that, you're treating
money as though it were a continuous quantity.
This is an approximation, so you can tolerate
having small inaccuracies in the result.

But when adding up actual, real amounts of
money, where the result must be exact, using
binary fractions is a very bad idea.

> This is not the same: if you round both value
> and then compare, you test a different interval
> than by taking the difference and applying
> a tolerance value comparison:

That's exactly my point. Chopping decimal
places is not a substitute for doing tolerance
testing properly.

> Most typical uses of round() don't use the
> optional argument, true, but I still fail
> to see what returning an integer instead of
> a float would buy you.

It saves you a function call in the vast
majority of cases, where an int is what
you ultimately want.

--
Greg



From bob at redivi.com  Thu Aug  3 05:20:40 2006
From: bob at redivi.com (Bob Ippolito)
Date: Wed, 2 Aug 2006 20:20:40 -0700
Subject: [Python-Dev] struct module and coercing floats to integers
In-Reply-To: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
Message-ID: <E2C65422-D1CE-46E7-AFD7-B5BE3EB624D8@redivi.com>


On Jul 28, 2006, at 1:35 PM, Bob Ippolito wrote:

> It seems that the pre-2.5 struct module has some additional
> undocumented behavior[1] that didn't percolate into the new version:
> http://python.org/sf/1530559
>
> Python 2.4 and previous will coerce floats to integers when necessary
> as such without any kind of complaint:
>
> $ python2.4 -c "import struct; print repr(struct.pack('>H',
> 0.9999999999999999))"
> '\x00\x00'
>
> Python 2.5 refuses to coerce float to int:
>
> $ python2.5 -c "import struct; print repr(struct.pack('>H',
> 0.9999999999999999))"
> Traceback (most recent call last):
>    File "<string>", line 1, in <module>
>    File "/Users/bob/src/python/Lib/struct.py", line 63, in pack
>      return o.pack(*args)
> TypeError: unsupported operand type(s) for &: 'float' and 'long'
>
> The available options are to:
>
> 1. Reinstate the pre-2.5 weirdness
> 2. Reinstate the pre-2.5 weirdness with a DeprecationWarning
> 3. Break existing code that relies on undocumented behavior (seems
> more like a bug than lack of specification)

There's a patch in the tracker for 2. It should get applied when the  
trunk freeze is over.

-bob


From rhettinger at ewtllc.com  Thu Aug  3 05:26:55 2006
From: rhettinger at ewtllc.com (Raymond Hettinger)
Date: Wed, 02 Aug 2006 20:26:55 -0700
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44D15FB4.5020605@canterbury.ac.nz>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
	<44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com>
	<44D15FB4.5020605@canterbury.ac.nz>
Message-ID: <44D16CFF.7090001@ewtllc.com>


>> Most typical uses of round() don't use the
>> optional argument, true, but I still fail
>> to see what returning an integer instead of
>> a float would buy you.
>
>
> It saves you a function call in the vast
> majority of cases, where an int is what
> you ultimately want.
>

-1 on an extra built-in just to save the time for function call -- this 
is a false optimization, one that is unlikely to ever be the bottleneck 
in any real program's running time.  If it isn't time you care about but 
just hate writing it, what's wrong with a simple helper function like:   
rndint=lambda x:  int(round(x)) ?  There are no shortage of possible 
function pairs that could be proposed (perhaps list(set(x)) or 
int(abs(x)) for example).  Of course, it would be silly to make new 
builtins for them.  IMHO, the case is weak for adding a new variant of 
round().

Also, -10 on changing the semantics of int() to round instead of 
truncate.  The truncating version is found is so many other languages 
and book examples, that it would be a disaster for us to choose a 
different meaning.


Raymond

From foom at fuhm.net  Thu Aug  3 05:39:43 2006
From: foom at fuhm.net (James Y Knight)
Date: Wed, 2 Aug 2006 23:39:43 -0400
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
	and coercing floats to integers)
In-Reply-To: <44D16CFF.7090001@ewtllc.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
	<44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com>
	<44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com>
Message-ID: <9E3C8812-877A-4923-A4AD-884B81AE1CDF@fuhm.net>


On Aug 2, 2006, at 11:26 PM, Raymond Hettinger wrote:
> Also, -10 on changing the semantics of int() to round instead of
> truncate.  The truncating version is found is so many other languages
> and book examples, that it would be a disaster for us to choose a
> different meaning.

I'd be happy to see floats lose their __int__ method entirely,  
replaced by an explicit truncate function.

I've always thought it quite a hack that python floats have implicit  
truncation to ints, and then a random smattering of APIs go to extra  
lengths to explicitly prevent float.__int__ from being called because  
people thought "passing a float makes no sense!". That's right, it  
doesn't, and it _never_ should happen implicitly, not just in those  
particular few cases. Explicit is better than implicit.

James


From greg.ewing at canterbury.ac.nz  Thu Aug  3 08:34:26 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 03 Aug 2006 18:34:26 +1200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44D16CFF.7090001@ewtllc.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
	<44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com>
	<44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com>
Message-ID: <44D198F2.9020702@canterbury.ac.nz>

Raymond Hettinger wrote:

> -1 on an extra built-in just to save the time for function call

The time isn't the main issue. The main issue
is that almost all the use cases for round()
involve doing an int() on it afterwards. At
least nobody has put forward an argument to
the contrary yet.

Sure you can define your own function to do
this. But that's still a considerable burden
when you add up the effort over all the
programs that use int(round()).

--
Greg



From tanzer at swing.co.at  Thu Aug  3 08:34:31 2006
From: tanzer at swing.co.at (Christian Tanzer)
Date: Thu, 03 Aug 2006 08:34:31 +0200
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: Your message of "Thu, 03 Aug 2006 14:03:19 +1200."
	<44D15967.5090603@canterbury.ac.nz>
Message-ID: <E1G8Wmt-0001Am-5D@swing.co.at>


Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> What's the feeling about this? If, e.g. int()
> were changed in Py3k to round instead of truncate,
> would it cause anyone substantial pain?

Gratuitous breakage!

I shudder at the thought of checking hundreds of int-calls to see if
they'd still be correct under such a change.

-- 
Christian Tanzer                                    http://www.c-tanzer.at/


From mal at egenix.com  Thu Aug  3 10:32:54 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 03 Aug 2006 10:32:54 +0200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and	coercing floats to integers)
In-Reply-To: <44D15FB4.5020605@canterbury.ac.nz>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>	<44CAD9A1.8080000@canterbury.ac.nz>
	<44CE4D32.6040909@ewtllc.com>	<44CEA091.1010406@canterbury.ac.nz>
	<44CF20EE.70103@egenix.com>	<44CF38A7.8040304@canterbury.ac.nz>
	<44CF3D96.7090101@egenix.com>	<44CFF9B9.3010402@canterbury.ac.nz>
	<44D09A5C.4030905@egenix.com> <44D15FB4.5020605@canterbury.ac.nz>
Message-ID: <44D1B4B6.2080403@egenix.com>

Greg Ewing wrote:
> M.-A. Lemburg wrote:
> 
>> Believe me: you have to if you want to do more
>> advanced calculus related to pricing and risk
>> analysis of derivatives.
> 
> When you do things like that, you're treating
> money as though it were a continuous quantity.
> This is an approximation, so you can tolerate
> having small inaccuracies in the result.
> 
> But when adding up actual, real amounts of
> money, where the result must be exact, using
> binary fractions is a very bad idea.

Agreed. Those are different use cases, though.

>> This is not the same: if you round both value
>> and then compare, you test a different interval
>> than by taking the difference and applying
>> a tolerance value comparison:
> 
> That's exactly my point. Chopping decimal
> places is not a substitute for doing tolerance
> testing properly.

As I said: those two are different ways of doing
comparisons.

If you are eventually rounding to say 2 decimal
places in the end of the calculation, you won't
want to find yourself presenting the user 1.12
and 1.13 as equal values :-)

>> Most typical uses of round() don't use the
>> optional argument, true, but I still fail
>> to see what returning an integer instead of
>> a float would buy you.
> 
> It saves you a function call in the vast
> majority of cases, where an int is what
> you ultimately want.

-1 on that. Just saving a single function call
isn't really enough to break all sorts of
applications out there that use round() in
calculations (with or without argument). Most
such calculations do work with floats, so having
round() return an int would just add another
coercion to a float for those use-cases.

Seriously, rounding is a complicated business.
round() just implements one way of doing it.
Perhaps a module providing different rounding
models would be of use ?!

The decimal module already has a few of those
implemented, so it could benefit from such a
module as well.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 03 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From rrr at ronadam.com  Thu Aug  3 10:31:58 2006
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 03 Aug 2006 03:31:58 -0500
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44D198F2.9020702@canterbury.ac.nz>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>	<44CAD9A1.8080000@canterbury.ac.nz>
	<44CE4D32.6040909@ewtllc.com>	<44CEA091.1010406@canterbury.ac.nz>
	<44CF20EE.70103@egenix.com>	<44CF38A7.8040304@canterbury.ac.nz>
	<44CF3D96.7090101@egenix.com>	<44CFF9B9.3010402@canterbury.ac.nz>
	<44D09A5C.4030905@egenix.com>	<44D15FB4.5020605@canterbury.ac.nz>
	<44D16CFF.7090001@ewtllc.com> <44D198F2.9020702@canterbury.ac.nz>
Message-ID: <44D1B47E.5020506@ronadam.com>

Greg Ewing wrote:
> Raymond Hettinger wrote:
> 
>> -1 on an extra built-in just to save the time for function call
> 
> The time isn't the main issue. The main issue
> is that almost all the use cases for round()
> involve doing an int() on it afterwards. At
> least nobody has put forward an argument to
> the contrary yet.

I'll try to give a valid argument of the contrary.

I think you are only considering part of the usefulness of round().  In 
statistics and science math equations, rounding isn't used to get an 
integer, but instead used to give an answer that is meaningful within 
the margin of error of the data being used.

Consider an example where you are combining data that had different 
number of significant digits.  Keeping all the digits of your answer 
gives a false since of accuracy.  The extra digits are meaningless 
because the margin of error is greater than any of the digits that 
exceed the minimum number of significant digits in your data.  So you 
need to remove the extraneous digits by rounding.  Then this answer can 
be further used as data, and sense the number of significant digits is 
maintained, the margin of error can be calculated accurately.


[ NOTE: While floating point may contribute a significant error to value 
that are very large or very small, in most cases it is a very small 
amount in relation to the precision of the calculation and so the error 
can be managed without too much trouble.  In cases where it is a 
significant factor, another internal representation should probably be 
considered.  I only mention it here because someone would point it out 
(again) anyway. The accuracy of floating point is not the issue being 
discussed in this thread. ]


It makes since for round have an argument to specify the number of 
digits of precision to round to and also for it to return floating point 
numbers because rounding results to a float of n digits of precision is 
a necessary operation.

If you have the default round() case of 0 (and negative) digits of 
precision return integers, you then have a function that may sometimes 
returns integers, and sometimes returns floats. This can be problematic 
if the values are further used in division operations.  Which will 
result in many cases of.. float(round(x)) in order to convert integer 
results back to floats.

      See pep-0238:  Changing the division operator.

      http://www.python.org/dev/peps/pep-0238/

While it is true we often will use round along with converting to an 
integer, it is also true that functions that are predictable and return 
a single type are easier to use and learn.  So the question of which is 
better is a matter of point of view in this respect.

And in regard to using round() combined with division operators in 
floating point math, it is an issue of least surprises.


> Sure you can define your own function to do
> this. But that's still a considerable burden
> when you add up the effort over all the
> programs that use int(round()).

I think the only time it would be a burden is if a single program uses 
int(round()) many times, otherwise it is only a very small additional 
amount to type for each program.  In the case of single program using it 
many times, a helper function is a very practical solution.


Cheers,
   Ron




From nmm1 at cus.cam.ac.uk  Thu Aug  3 10:35:23 2006
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Thu, 03 Aug 2006 09:35:23 +0100
Subject: [Python-Dev] Rounding float to int directly ...
Message-ID: <E1G8Yfr-0004Ne-CH@draco.cus.cam.ac.uk>

James Y Knight <foom at fuhm.net> wrote:
>
> I'd be happy to see floats lose their __int__ method entirely,  
> replaced by an explicit truncate function.

Come back Algol - all is forgiven :-)  Yes, indeed.  I have favoured
that view for 35 years - anything that can lose information quietly
should be explicit.


tanzer at swing.co.at (Christian Tanzer) wrote:
> Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
> > What's the feeling about this? If, e.g. int()
> > were changed in Py3k to round instead of truncate,
> > would it cause anyone substantial pain?
> 
> Gratuitous breakage!
>         
> I shudder at the thought of checking hundreds of int-calls to see if
> they'd still be correct under such a change.

My experience of doing that when compilers sometimes did one and sometimes
the other is that such breakages are rarer than the conversions to integer
that are broken with both rules!  And both are rarer than the code that
works with either rule.

However, a 5% breakage rate is still enough to be of concern.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From tanzer at swing.co.at  Thu Aug  3 10:44:51 2006
From: tanzer at swing.co.at (Christian Tanzer)
Date: Thu, 03 Aug 2006 10:44:51 +0200
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: Your message of "Thu, 03 Aug 2006 09:35:23 BST."
	<E1G8Yfr-0004Ne-CH@draco.cus.cam.ac.uk>
Message-ID: <E1G8Yp1-0004BN-EC@swing.co.at>


Nick Maclaren <nmm1 at cus.cam.ac.uk> wrote:

> tanzer at swing.co.at (Christian Tanzer) wrote:
> > Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> >
> > > What's the feeling about this? If, e.g. int()
> > > were changed in Py3k to round instead of truncate,
> > > would it cause anyone substantial pain?
> >
> > Gratuitous breakage!
> >
> > I shudder at the thought of checking hundreds of int-calls to see if
> > they'd still be correct under such a change.
>
> My experience of doing that when compilers sometimes did one and
> sometimes the other is that such breakages are rarer than the
> conversions to integer that are broken with both rules! And both are
> rarer than the code that works with either rule.
>
> However, a 5% breakage rate is still enough to be of concern.

I couldn't care less about how many calls would break -- I'd still
need to look at each and every one. And I know that quite a number of
calls need the truncation semantics (I just don't know which without
looking).

-- 
Christian Tanzer                                    http://www.c-tanzer.at/


From ncoghlan at gmail.com  Thu Aug  3 13:43:27 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 03 Aug 2006 21:43:27 +1000
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44D1B47E.5020506@ronadam.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>	<44CAD9A1.8080000@canterbury.ac.nz>	<44CE4D32.6040909@ewtllc.com>	<44CEA091.1010406@canterbury.ac.nz>	<44CF20EE.70103@egenix.com>	<44CF38A7.8040304@canterbury.ac.nz>	<44CF3D96.7090101@egenix.com>	<44CFF9B9.3010402@canterbury.ac.nz>	<44D09A5C.4030905@egenix.com>	<44D15FB4.5020605@canterbury.ac.nz>	<44D16CFF.7090001@ewtllc.com>
	<44D198F2.9020702@canterbury.ac.nz> <44D1B47E.5020506@ronadam.com>
Message-ID: <44D1E15F.9060908@gmail.com>

Ron Adam wrote:
> Consider an example where you are combining data that had different 
> number of significant digits.  Keeping all the digits of your answer 
> gives a false since of accuracy.  The extra digits are meaningless 
> because the margin of error is greater than any of the digits that 
> exceed the minimum number of significant digits in your data.  So you 
> need to remove the extraneous digits by rounding.  Then this answer can 
> be further used as data, and sense the number of significant digits is 
> maintained, the margin of error can be calculated accurately.

This is a fallacy though - add two numbers together each with an error of +/- 
0.05, and the error in the total is +/- 0.1. The approach you propose gives 
the misleading impression that the error in the total is still only +/- 0.05 
(as the answer will contain 1 decimal place).

If you really care about error, you need to do it properly (e.g. stddev or 
adding the error margins).

Anyway, it's not proposed to remove the "round to x decimal places" capability 
entirely - merely remove it from the builtin round() so that the return type 
can be changed.

> It makes since for round have an argument to specify the number of 
> digits of precision to round to and also for it to return floating point 
> numbers because rounding results to a float of n digits of precision is 
> a necessary operation.
> 
> If you have the default round() case of 0 (and negative) digits of 
> precision return integers, you then have a function that may sometimes 
> returns integers, and sometimes returns floats.

That's not the proposal, as Greg has already said (the fround() mentioned 
would probably be 'math.fround' rather than a builtin):

[Greg Ewing]
> No, round() wouldn't have that option at all. If
> you wanted it, you would use fround() instead,
> which would have the option and return a float
> always.
> 
> This would be a Py3k thing, obviously. If done
> before then, the new function would have to be
> given a different name.

[Ron Adam]
> This can be problematic 
> if the values are further used in division operations.  Which will 
> result in many cases of.. float(round(x)) in order to convert integer 
> results back to floats.

Not in Py3k where int/int will return a float as needed.

> And in regard to using round() combined with division operators in 
> floating point math, it is an issue of least surprises.

And in Py3k, with a round that always returned an integer there wouldn't be 
any surprises at all:
   - division would do the right thing, because true division becomes the only 
behaviour for '/'
   - the result of the builtin rounding operation would be usable directly as 
an index without requiring subsequent coercion to an integer

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ralf at brainbot.com  Thu Aug  3 15:02:11 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 03 Aug 2006 15:02:11 +0200
Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
Message-ID: <eass46$ups$1@sea.gmane.org>

Hi all,

I've been trying to port our software to python 2.5.
unfortunately I'm getting constantly hit by segfaults.

I've boiled it down to the following code:

rs at schrotti:~/bug$ cat t.py
import array

class Indexer(object):
     maximumForwardSize = property(array.array.fromstring)
rs at schrotti:~/bug$ python t.py
Segmentation fault
rs at schrotti:~/bug$ python
Python 2.5b3 (trunk:51066, Aug  3 2006, 13:18:16)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>>
[18733 refs]
[7182 refs]
rs at schrotti:~/bug$ gdb python
GNU gdb 6.4-debian
Copyright 2005 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...Using host libthread_db 
library "/lib/tls/i686/cmov/libthread_db.so.1".

(gdb) r t.py
Starting program: /home/rs/exp/bin/python t.py
[Thread debugging using libthread_db enabled]
[New Thread -1210304832 (LWP 21660)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1210304832 (LWP 21660)]
0x0811b68a in visit_decref (op=0xb7db21d8, data=0x0) at 
Modules/gcmodule.c:270
270		if (PyObject_IS_GC(op)) {
(gdb) bt
#0  0x0811b68a in visit_decref (op=0xb7db21d8, data=0x0) at 
Modules/gcmodule.c:270
#1  0x0814043d in property_traverse (self=0xb7d7b5b4, visit=0x811b654 
<visit_decref>, arg=0x0) at Objects/descrobject.c:1235
#2  0x0811b753 in subtract_refs (containers=0x8197d88) at 
Modules/gcmodule.c:295
#3  0x0811c453 in collect (generation=2) at Modules/gcmodule.c:790
#4  0x0811cf76 in PyGC_Collect () at Modules/gcmodule.c:1264
#5  0x0810d5da in Py_Finalize () at Python/pythonrun.c:377
#6  0x08057556 in Py_Main (argc=2, argv=0xbfb5a6f4) at Modules/main.c:516
#7  0x080565ba in main (argc=2, argv=0xbfb5a6f4) at ./Modules/python.c:23
(gdb)


From theller at python.net  Thu Aug  3 15:46:28 2006
From: theller at python.net (Thomas Heller)
Date: Thu, 03 Aug 2006 15:46:28 +0200
Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
In-Reply-To: <eass46$ups$1@sea.gmane.org>
References: <eass46$ups$1@sea.gmane.org>
Message-ID: <easun9$8q1$1@sea.gmane.org>

Ralf Schmitt schrieb:
> Hi all,
> 
> I've been trying to port our software to python 2.5.
> unfortunately I'm getting constantly hit by segfaults.
> 
> I've boiled it down to the following code:
> 
> rs at schrotti:~/bug$ cat t.py
> import array
> 
> class Indexer(object):
>      maximumForwardSize = property(array.array.fromstring)
> rs at schrotti:~/bug$ python t.py
> Segmentation fault

Confirmed under Windows.  Here's the problem (Objects/descrobject.c, near line 1200,
in property_init(...):


	/* if no docstring given and the getter has one, use that one */
	if ((doc == NULL || doc == Py_None) && get != NULL && 
	    PyObject_HasAttrString(get, "__doc__")) {
		if (!(get_doc = PyObject_GetAttrString(get, "__doc__")))
			return -1;
		Py_DECREF(get_doc); /* it is INCREF'd again below */
                ^^^^^^^^^^^^^^^^^^
		doc = get_doc;
	}

	Py_XINCREF(get);
	Py_XINCREF(set);
	Py_XINCREF(del);
	Py_XINCREF(doc);

If the refcount of get_doc drops to zero in the Py_DECREF(), the Py_XINCREF()
shortly after doesn't help ;-).

Thomas


From theller at python.net  Thu Aug  3 15:56:48 2006
From: theller at python.net (Thomas Heller)
Date: Thu, 03 Aug 2006 15:56:48 +0200
Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
In-Reply-To: <easun9$8q1$1@sea.gmane.org>
References: <eass46$ups$1@sea.gmane.org> <easun9$8q1$1@sea.gmane.org>
Message-ID: <easvak$89h$1@sea.gmane.org>

> 	/* if no docstring given and the getter has one, use that one */
> 	if ((doc == NULL || doc == Py_None) && get != NULL && 
> 	    PyObject_HasAttrString(get, "__doc__")) {
> 		if (!(get_doc = PyObject_GetAttrString(get, "__doc__")))
> 			return -1;
> 		Py_DECREF(get_doc); /* it is INCREF'd again below */
>                 ^^^^^^^^^^^^^^^^^^
> 		doc = get_doc;
> 	}
> 
> 	Py_XINCREF(get);
> 	Py_XINCREF(set);
> 	Py_XINCREF(del);
> 	Py_XINCREF(doc);
> 

A strange programming style, if you ask me, and I wonder why Coverity doesn't complain
about it.

Thomas


From duncan.booth at suttoncourtenay.org.uk  Thu Aug  3 16:41:45 2006
From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth)
Date: Thu, 3 Aug 2006 15:41:45 +0100
Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
References: <eass46$ups$1@sea.gmane.org> <easun9$8q1$1@sea.gmane.org>
	<easvak$89h$1@sea.gmane.org>
Message-ID: <n2m-g.Xns98149FAE6AB78duncanrcpcouk@127.0.0.1>

Thomas Heller <theller at python.net> wrote in
news:easvak$89h$1 at sea.gmane.org: 

>>      /* if no docstring given and the getter has one, use that one */
>>      if ((doc == NULL || doc == Py_None) && get != NULL && 
>>          PyObject_HasAttrString(get, "__doc__")) {
>>           if (!(get_doc = PyObject_GetAttrString(get, "__doc__")))
>>                return -1;
>>           Py_DECREF(get_doc); /* it is INCREF'd again below */
>>                 ^^^^^^^^^^^^^^^^^^
>>           doc = get_doc;
>>      }
>> 
>>      Py_XINCREF(get);
>>      Py_XINCREF(set);
>>      Py_XINCREF(del);
>>      Py_XINCREF(doc);
>> 
> 
> A strange programming style, if you ask me, and I wonder why Coverity
> doesn't complain about it.
> 
Does Coverity recognise objects on Python's internal pools as deallocated? 

If not it wouldn't complain because all that the Py_DECREF does is link the 
block into a pool's freeblock list so any subsequent reference from the 
Py_XINCREF is still a reference to allocated memory.


[Off topic:
Microsoft have (or had?) a similarly screwy bit in their ActiveX atl 
libraries: a newly created ActiveX object has its reference count 
incremented before calling FinalConstruct and then  decremented to 0  
(using a method which decrements the reference count but doesn't free it) 
before being incremented again. If in the meanwhile you increment and 
decrement the reference count in another thread then it goes bang.]

The moral is to regard the reference counting rules as law: no matter how 
sure you are that you can cheat, don't or you'll regret it.


From ralf at brainbot.com  Thu Aug  3 16:59:08 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 03 Aug 2006 16:59:08 +0200
Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
In-Reply-To: <n2m-g.Xns98149FAE6AB78duncanrcpcouk@127.0.0.1>
References: <eass46$ups$1@sea.gmane.org>
	<easun9$8q1$1@sea.gmane.org>	<easvak$89h$1@sea.gmane.org>
	<n2m-g.Xns98149FAE6AB78duncanrcpcouk@127.0.0.1>
Message-ID: <44D20F3C.9040602@brainbot.com>

Duncan Booth wrote:
> 
> 
> The moral is to regard the reference counting rules as law: no matter how 
> sure you are that you can cheat, don't or you'll regret it.
> 

Or someone else will regret it, just like in this case. :)

- Ralf


From ralf at brainbot.com  Thu Aug  3 17:40:57 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 03 Aug 2006 17:40:57 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
Message-ID: <eat5ds$37j$1@sea.gmane.org>

Still trying to port our software. here's another thing I noticed:

d = {}
d[u'm\xe1s'] = 1
d['m\xe1s'] = 1
print d

With python 2.4 I can add those two keys to the dictionary and get:
$ python2.4 t2.py
{u'm\xe1s': 1, 'm\xe1s': 1}

With python 2.5 I get:

$ python2.5 t2.py
Traceback (most recent call last):
   File "t2.py", line 3, in <module>
     d['m\xe1s'] = 1
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: 
ordinal not in range(128)

Is this intended behaviour? I guess this might break lots of programs 
and the way python 2.4 works looks right to me.
I think it should be possible to mix str/unicode keys in dicts and let 
non-ascii strings compare not-equal to any unicode string.

At least it should be documented prominently in the "what's new" document.

- Ralf



From ralf at brainbot.com  Thu Aug  3 17:47:37 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 03 Aug 2006 17:47:37 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <eat5ds$37j$1@sea.gmane.org>
References: <eat5ds$37j$1@sea.gmane.org>
Message-ID: <44D21A99.2080801@brainbot.com>

Ralf Schmitt wrote:
> Still trying to port our software. here's another thing I noticed:
> 
> d = {}
> d[u'm\xe1s'] = 1
> d['m\xe1s'] = 1
> print d
> 
> With python 2.4 I can add those two keys to the dictionary and get:
> $ python2.4 t2.py
> {u'm\xe1s': 1, 'm\xe1s': 1}
> 
> With python 2.5 I get:
> 
> $ python2.5 t2.py
> Traceback (most recent call last):
>    File "t2.py", line 3, in <module>
>      d['m\xe1s'] = 1
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: 
> ordinal not in range(128)
> 
> Is this intended behaviour? I guess this might break lots of programs 
> and the way python 2.4 works looks right to me.
> I think it should be possible to mix str/unicode keys in dicts and let 
> non-ascii strings compare not-equal to any unicode string.

Also this behaviour makes your programs break randomly, that is, it will 
break when the string you add hashes to the same value that the unicode 
string has (at least that's what I guess..)

- Ralf



From mwh at python.net  Thu Aug  3 17:51:29 2006
From: mwh at python.net (Michael Hudson)
Date: Thu, 03 Aug 2006 16:51:29 +0100
Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
In-Reply-To: <n2m-g.Xns98149FAE6AB78duncanrcpcouk@127.0.0.1> (Duncan Booth's
	message of "Thu, 3 Aug 2006 15:41:45 +0100")
References: <eass46$ups$1@sea.gmane.org> <easun9$8q1$1@sea.gmane.org>
	<easvak$89h$1@sea.gmane.org>
	<n2m-g.Xns98149FAE6AB78duncanrcpcouk@127.0.0.1>
Message-ID: <2mmzalx1fi.fsf@starship.python.net>

Duncan Booth <duncan.booth at suttoncourtenay.org.uk> writes:

> Does Coverity recognise objects on Python's internal pools as deallocated? 

Coverity doesn't work on that level; it analyzes source code, and
knows about Python's INCREFs and DECREFs.

> The moral is to regard the reference counting rules as law: no matter how 
> sure you are that you can cheat, don't or you'll regret it.

This is the truth.

Cheers,
mwh

-- 
 As it seems to me, in Perl you have to be an expert to correctly make
 a nested data structure like, say, a list of hashes of instances.  In
 Python, you have to be an idiot not  to be able to do it, because you
 just write it down.             -- Peter Norvig, comp.lang.functional

From mal at egenix.com  Thu Aug  3 18:51:21 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 03 Aug 2006 18:51:21 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D21A99.2080801@brainbot.com>
References: <eat5ds$37j$1@sea.gmane.org> <44D21A99.2080801@brainbot.com>
Message-ID: <44D22989.7000202@egenix.com>

Ralf Schmitt wrote:
> Ralf Schmitt wrote:
>> Still trying to port our software. here's another thing I noticed:
>>
>> d = {}
>> d[u'm\xe1s'] = 1
>> d['m\xe1s'] = 1
>> print d
>>
>> With python 2.4 I can add those two keys to the dictionary and get:
>> $ python2.4 t2.py
>> {u'm\xe1s': 1, 'm\xe1s': 1}
>>
>> With python 2.5 I get:
>>
>> $ python2.5 t2.py
>> Traceback (most recent call last):
>>    File "t2.py", line 3, in <module>
>>      d['m\xe1s'] = 1
>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: 
>> ordinal not in range(128)
>>
>> Is this intended behaviour? I guess this might break lots of programs 
>> and the way python 2.4 works looks right to me.
>> I think it should be possible to mix str/unicode keys in dicts and let 
>> non-ascii strings compare not-equal to any unicode string.
> 
> Also this behaviour makes your programs break randomly, that is, it will 
> break when the string you add hashes to the same value that the unicode 
> string has (at least that's what I guess..)

This is because Unicode and 8-bit string keys only work
in the same way if and only if they are plain ASCII.

The reason lies in the hash function used by Unicode: it is
crafted to make hash(u) == hash(s) for all ASCII s, such
that s == u.

For non-ASCII strings, there are no guarantees as to the
hash value of the strings or whether they match or not.

This has been like that since Unicode was introduced, so it's
not new in Python 2.5.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 03 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bob at redivi.com  Thu Aug  3 19:03:08 2006
From: bob at redivi.com (Bob Ippolito)
Date: Thu, 3 Aug 2006 10:03:08 -0700
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <44D22989.7000202@egenix.com>
References: <eat5ds$37j$1@sea.gmane.org> <44D21A99.2080801@brainbot.com>
	<44D22989.7000202@egenix.com>
Message-ID: <1009DE87-F7CD-41F0-8409-BA3599ACB65E@redivi.com>


On Aug 3, 2006, at 9:51 AM, M.-A. Lemburg wrote:

> Ralf Schmitt wrote:
>> Ralf Schmitt wrote:
>>> Still trying to port our software. here's another thing I noticed:
>>>
>>> d = {}
>>> d[u'm\xe1s'] = 1
>>> d['m\xe1s'] = 1
>>> print d
>>>
>>> With python 2.4 I can add those two keys to the dictionary and get:
>>> $ python2.4 t2.py
>>> {u'm\xe1s': 1, 'm\xe1s': 1}
>>>
>>> With python 2.5 I get:
>>>
>>> $ python2.5 t2.py
>>> Traceback (most recent call last):
>>>    File "t2.py", line 3, in <module>
>>>      d['m\xe1s'] = 1
>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in  
>>> position 1:
>>> ordinal not in range(128)
>>>
>>> Is this intended behaviour? I guess this might break lots of  
>>> programs
>>> and the way python 2.4 works looks right to me.
>>> I think it should be possible to mix str/unicode keys in dicts  
>>> and let
>>> non-ascii strings compare not-equal to any unicode string.
>>
>> Also this behaviour makes your programs break randomly, that is,  
>> it will
>> break when the string you add hashes to the same value that the  
>> unicode
>> string has (at least that's what I guess..)
>
> This is because Unicode and 8-bit string keys only work
> in the same way if and only if they are plain ASCII.
>
> The reason lies in the hash function used by Unicode: it is
> crafted to make hash(u) == hash(s) for all ASCII s, such
> that s == u.
>
> For non-ASCII strings, there are no guarantees as to the
> hash value of the strings or whether they match or not.
>
> This has been like that since Unicode was introduced, so it's
> not new in Python 2.5.

What is new is that the exception raised on "u == s" after hash  
collision is no longer silently swallowed.

-bob


From martin at v.loewis.de  Thu Aug  3 19:15:59 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 03 Aug 2006 19:15:59 +0200
Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
In-Reply-To: <eass46$ups$1@sea.gmane.org>
References: <eass46$ups$1@sea.gmane.org>
Message-ID: <44D22F4F.4060707@v.loewis.de>

Ralf Schmitt schrieb:
> I've been trying to port our software to python 2.5.
> unfortunately I'm getting constantly hit by segfaults.

I understand it's unfortunate for you, but it is fortunate
for us that you have been trying to port your application
before 2.5 was released, and reported the bug when you
discovered it. So thanks for reporting it.

FWIW, the bug that Thomas found was introduced by
patch #1434038, and committed as r42916.

Regards,
Martin

From ralf at brainbot.com  Thu Aug  3 19:17:27 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 03 Aug 2006 19:17:27 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <44D22989.7000202@egenix.com>
References: <eat5ds$37j$1@sea.gmane.org> <44D21A99.2080801@brainbot.com>
	<44D22989.7000202@egenix.com>
Message-ID: <44D22FA7.7030905@brainbot.com>

M.-A. Lemburg wrote:
> Ralf Schmitt wrote:
>> Ralf Schmitt wrote:
>>> Still trying to port our software. here's another thing I noticed:
>>>
>>> d = {}
>>> d[u'm\xe1s'] = 1
>>> d['m\xe1s'] = 1
>>> print d
>>>
>>> With python 2.4 I can add those two keys to the dictionary and get:
>>> $ python2.4 t2.py
>>> {u'm\xe1s': 1, 'm\xe1s': 1}
>>>
>>> With python 2.5 I get:
>>>
>>> $ python2.5 t2.py
>>> Traceback (most recent call last):
>>>    File "t2.py", line 3, in <module>
>>>      d['m\xe1s'] = 1
>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: 
>>> ordinal not in range(128)
>>>
>>> Is this intended behaviour? I guess this might break lots of programs 
>>> and the way python 2.4 works looks right to me.
>>> I think it should be possible to mix str/unicode keys in dicts and let 
>>> non-ascii strings compare not-equal to any unicode string.
>> Also this behaviour makes your programs break randomly, that is, it will 
>> break when the string you add hashes to the same value that the unicode 
>> string has (at least that's what I guess..)
> 
> This is because Unicode and 8-bit string keys only work
> in the same way if and only if they are plain ASCII.

This is okay. But in the case where one is not ASCII I would prefer to 
be able to compare them (not equal) instead of getting a UnicodeError.
I know it's too late to change this, ...

> 
> The reason lies in the hash function used by Unicode: it is
> crafted to make hash(u) == hash(s) for all ASCII s, such
> that s == u.
> 
> For non-ASCII strings, there are no guarantees as to the
> hash value of the strings or whether they match or not.
> 
> This has been like that since Unicode was introduced, so it's
> not new in Python 2.5.
> 

...but in the case of dictionaries this behaviour has changed and in 
prior versions of python dictionaries did work as I expected them to.
Now they don't.

When working with unicode strings and (accidently) mixing with str 
strings, things might seem to work until the first non-ascii string
is given to some code and one gets that UnicodeDecodeError (e.g. when 
comparing them).

If one mixes unicode strings and str strings as keys in a dictionary 
things might seem to work far longer until he tries to put in some non 
ASCII string with the "wrong" hash value and suddenly things go boom.
I'd rather keep the pre 2.5 behaviour.

- Ralf


From jimjjewett at gmail.com  Thu Aug  3 19:36:30 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 3 Aug 2006 13:36:30 -0400
Subject: [Python-Dev] Rounding float to int directly (Re: struct module and
	coercing floats to integers)
Message-ID: <fb6fbf560608031036r5abcf70bm5552529c73b7c3a4@mail.gmail.com>

Greg Ewing wrote:

> The time isn't the main issue. The main issue
> is that almost all the use cases for round()
> involve doing an int() on it afterwards. At
> least nobody has put forward an argument to
> the contrary yet.

Or, more accurately, they *should* involve an int
afterwards, but often don't, because people expect
round(something) to produce an integer.

Looking only at the one-argument round() calls not
surrounded by int(), I would bet that most are bugs
rather than intentional.  The proportion is certainly
high enough that I can't just trust it.

Ron Adam wrote:

> I think you are only considering part of the usefulness of round().  In
> statistics and science math equations, rounding isn't used to get an
> integer, but instead used to give an answer that is meaningful within
> the margin of error of the data being used.

Yes, but

(1)  Only for display (or storage).  This rounding should explicitly
not be done on intermediate values, as it introduces another source
of error.

(2)  You need to pass the precision, so this isn't the one-argument round().

(3)  As a side note, even the two-argument round may not be what you
want; it isn't uncommon to have accuracy to a number of significant
figures, rather than to an absolute "three places past the decimal point".
This gets complicated enough that it is reasonable to use math.fround()
or some such instead of a builtin.

-jJ

From mal at egenix.com  Thu Aug  3 19:39:15 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 03 Aug 2006 19:39:15 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D22FA7.7030905@brainbot.com>
References: <eat5ds$37j$1@sea.gmane.org>
	<44D21A99.2080801@brainbot.com>	<44D22989.7000202@egenix.com>
	<44D22FA7.7030905@brainbot.com>
Message-ID: <44D234C3.4010404@egenix.com>

Ralf Schmitt wrote:
>>>> Still trying to port our software. here's another thing I noticed:
>>>>
>>>> d = {}
>>>> d[u'm\xe1s'] = 1
>>>> d['m\xe1s'] = 1
>>>> print d
>>>>
>>>> With python 2.5 I get:
>>>>
>>>> $ python2.5 t2.py
>>>> Traceback (most recent call last):
>>>>    File "t2.py", line 3, in <module>
>>>>      d['m\xe1s'] = 1
>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1: 
>>>> ordinal not in range(128)
>>>>
>> This is because Unicode and 8-bit string keys only work
>> in the same way if and only if they are plain ASCII.
> 
> This is okay. But in the case where one is not ASCII I would prefer to 
> be able to compare them (not equal) instead of getting a UnicodeError.
> I know it's too late to change this, ...

It is too late to change this, since it was always like this ;-)

Seriously, Unicode is doing the right thing here: you should
really always get an exception if you compare apples and
oranges, rather than reverting to comparing the ids of apples
and oranges as fall-back solution.

I believe that Py3k will implement this.

>> The reason lies in the hash function used by Unicode: it is
>> crafted to make hash(u) == hash(s) for all ASCII s, such
>> that s == u.
>>
>> For non-ASCII strings, there are no guarantees as to the
>> hash value of the strings or whether they match or not.
>>
>> This has been like that since Unicode was introduced, so it's
>> not new in Python 2.5.
>>
> 
> ...but in the case of dictionaries this behaviour has changed and in 
> prior versions of python dictionaries did work as I expected them to.
> Now they don't.

Let's put it this way: Python 2.5 uncovered a bug in your
application that has always been there. It's better to
fix your application than arguing to cover up the bug again.

> When working with unicode strings and (accidently) mixing with str 
> strings, things might seem to work until the first non-ascii string
> is given to some code and one gets that UnicodeDecodeError (e.g. when 
> comparing them).
> 
> If one mixes unicode strings and str strings as keys in a dictionary 
> things might seem to work far longer until he tries to put in some non 
> ASCII string with the "wrong" hash value and suddenly things go boom.
> I'd rather keep the pre 2.5 behaviour.

It's actually a good preparation for Py3k where 1 == u'abc' will
(likely) also raise an exception.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 03 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From jjl at pobox.com  Thu Aug  3 19:45:23 2006
From: jjl at pobox.com (John J Lee)
Date: Thu, 3 Aug 2006 18:45:23 +0100 (GMT Standard Time)
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D234C3.4010404@egenix.com>
References: <eat5ds$37j$1@sea.gmane.org> <44D21A99.2080801@brainbot.com>
	<44D22989.7000202@egenix.com> <44D22FA7.7030905@brainbot.com>
	<44D234C3.4010404@egenix.com>
Message-ID: <Pine.WNT.4.64.0608031842340.2112@shaolin>

On Thu, 3 Aug 2006, M.-A. Lemburg wrote:
[...]
> It's actually a good preparation for Py3k where 1 == u'abc' will
> (likely) also raise an exception.

I though I'd heard (from Guido here or on the py3k list) that it was only 
1 < u'abc' that would raise an exception, and that 1 == u'abc' would still 
evaluate to False.  Did I misunderstand?


John

From g.brandl at gmx.net  Thu Aug  3 19:50:58 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 03 Aug 2006 19:50:58 +0200
Subject: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
In-Reply-To: <44D22F4F.4060707@v.loewis.de>
References: <eass46$ups$1@sea.gmane.org> <44D22F4F.4060707@v.loewis.de>
Message-ID: <eatcvn$ueh$1@sea.gmane.org>

Martin v. L?wis wrote:
> Ralf Schmitt schrieb:
>> I've been trying to port our software to python 2.5.
>> unfortunately I'm getting constantly hit by segfaults.
> 
> I understand it's unfortunate for you, but it is fortunate
> for us that you have been trying to port your application
> before 2.5 was released, and reported the bug when you
> discovered it. So thanks for reporting it.
> 
> FWIW, the bug that Thomas found was introduced by
> patch #1434038, and committed as r42916.

I already fixed it locally and will commit as soon as trunk is
unfrozen.

Georg


From mal at egenix.com  Thu Aug  3 19:53:11 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 03 Aug 2006 19:53:11 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <Pine.WNT.4.64.0608031842340.2112@shaolin>
References: <eat5ds$37j$1@sea.gmane.org>
	<44D21A99.2080801@brainbot.com>	<44D22989.7000202@egenix.com>
	<44D22FA7.7030905@brainbot.com>	<44D234C3.4010404@egenix.com>
	<Pine.WNT.4.64.0608031842340.2112@shaolin>
Message-ID: <44D23807.4050604@egenix.com>

John J Lee wrote:
> On Thu, 3 Aug 2006, M.-A. Lemburg wrote:
> [...]
>> It's actually a good preparation for Py3k where 1 == u'abc' will
>> (likely) also raise an exception.
> 
> I though I'd heard (from Guido here or on the py3k list) that it was only 
> 1 < u'abc' that would raise an exception, and that 1 == u'abc' would still 
> evaluate to False.  Did I misunderstand?

Could be that I'm wrong.

All such mixed-type compares are wrong depending on the scale of
brokenness you apply ;-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 03 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From python-dev at zesty.ca  Thu Aug  3 21:02:45 2006
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Thu, 3 Aug 2006 14:02:45 -0500 (CDT)
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44D198F2.9020702@canterbury.ac.nz>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
	<44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com>
	<44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com>
	<44D198F2.9020702@canterbury.ac.nz>
Message-ID: <Pine.LNX.4.58.0608031353580.32166@server1.LFW.org>

On Thu, 3 Aug 2006, Greg Ewing wrote:
> The time isn't the main issue. The main issue
> is that almost all the use cases for round()
> involve doing an int() on it afterwards.

That's my experience as well.  In my opinion, the purpose of round()
is most commonly described as "to make an integer".  So it should
yield an integer.

As other comments in this discussion imply, whether you want round()
to return an integer really depends on your intended audience and
their pattern of usage.

It seems to me that for round() to produce an integer is about as
intuitive as for 1/2 to produce 0.5.  That is, if you bring basic
mathematical intuition to the table instead of a background in how
C behaves, then "1/2" suggests one-half and "rounding" suggests
"convert to an integer".  If you have a background in programming
languages and you understand the value of type consistency, then
you would see why dividing integers should yield integers and
rounding floats should yield floats.

Since Python was indeed changed to make 1/2 evaluate to 0.5, that
suggests that it is Pythonic to target the first audience rather
than the second.  And that implies that it would also be Pythonic
to make round() return an int.


-- ?!ng

From nmm1 at cus.cam.ac.uk  Thu Aug  3 21:43:07 2006
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Thu, 03 Aug 2006 20:43:07 +0100
Subject: [Python-Dev] Rounding float to int directly ...
Message-ID: <E1G8j63-00048w-Ku@draco.cus.cam.ac.uk>

Ka-Ping Yee <python-dev at zesty.ca> wrote:
>
> That's my experience as well.  In my opinion, the purpose of round()
> is most commonly described as "to make an integer".  So it should
> yield an integer.

Grrk.  No, that logic is flawed.

There are algorithms where the operation of rounding (or truncation)
is needed, but where the value may be larger than can be held in an
integer, and that is not an error.  If the only rounding or truncation
primitive converts to an integer, those algorithms are unimplementable.
You need at least one primitive that converts a float to an integer,
held as a float.

Which is independent of whether THIS particular facility should yield
an integer or float!


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From ronaldoussoren at mac.com  Thu Aug  3 22:12:11 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 3 Aug 2006 22:12:11 +0200
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: <E1G8j63-00048w-Ku@draco.cus.cam.ac.uk>
References: <E1G8j63-00048w-Ku@draco.cus.cam.ac.uk>
Message-ID: <9AD92B78-1723-43D4-B1F5-0D6147E59CEE@mac.com>


On Aug 3, 2006, at 9:43 PM, Nick Maclaren wrote:

> Ka-Ping Yee <python-dev at zesty.ca> wrote:
>>
>> That's my experience as well.  In my opinion, the purpose of round()
>> is most commonly described as "to make an integer".  So it should
>> yield an integer.
>
> Grrk.  No, that logic is flawed.
>
> There are algorithms where the operation of rounding (or truncation)
> is needed, but where the value may be larger than can be held in an
> integer, and that is not an error.

Is that really true for python? Python integers are unbounded in  
magnitute, they are not the same as int or long in C, therefore any  
float except exceptional values like NaN can be converted to an  
integer value. The converse is not true, python integers can contain  
values that are larger than any float (aka C's double).

 >>> v = 1e200
 >>> int(v)
999999999999999969733122212510361659474503275455023626482417509503468484 
355540755341963384047062518680275124159738824081821357343682784846393850 
41047239877871023591066789981811181813306167128854888448L
 >>> type(v)
<type 'float'>
 >>> t = int(v)
 >>> t ** 2
999999999999999939466244425020724235032897553743712247233978162062705420 
868772363027380308001932133054230558394675289323324880702327952854432161 
552216024892912466614409626956153314556116473848998339762109232220813863 
099472521374735119038509661875525607726747258646821773646868361139842288 
412173261267039669530389442594522433115448347796339690544576171593343952 
0020822843337114038314499908946523848704L
 >>> float(t**2)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
OverflowError: long int too large to convert to float


Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2157 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060803/5e233fa2/attachment.bin 

From nmm1 at cus.cam.ac.uk  Thu Aug  3 22:49:30 2006
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Thu, 03 Aug 2006 21:49:30 +0100
Subject: [Python-Dev] Rounding float to int directly ...
Message-ID: <E1G8k8I-0004W8-P7@draco.cus.cam.ac.uk>

Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> 
> > There are algorithms where the operation of rounding (or truncation)
> > is needed, but where the value may be larger than can be held in an
> > integer, and that is not an error.
> 
> Is that really true for python? Python integers are unbounded in  
> magnitute, they are not the same as int or long in C, therefore any  
> float except exceptional values like NaN can be converted to an  
> integer value. The converse is not true, python integers can contain  
> values that are larger than any float (aka C's double).

It depends a great deal on what you mean by a Python integer!  Yes,
I was assuming the (old) Python model, where it is a C long, but so
were many (most?) of the other postings.

If you are assuming the (future?) model, where there is a single
integer type of unlimited size, then that is true.  There is still
an efficiency point, in that such algorithms positively don't want
a float value like 1.0e300 (or, worse, 1.0e4000) expanded to its
full decimal representation as an intermediate step.

Whatever.  There should still be at least one operation that rounds
or truncates a float value, returning a float of the same type, on
either functionality or efficiency grounds.  I and most programmers
of such algorithms don't give a damn which it does, provided that it
is clearly documented, at least half-sane and doesn't change with
versions of Python.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From jimjjewett at gmail.com  Thu Aug  3 23:10:43 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 3 Aug 2006 17:10:43 -0400
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
Message-ID: <fb6fbf560608031410h3a72ae72meb885856b0922ee3@mail.gmail.com>

http://mail.python.org/pipermail/python-dev/2006-August/067934.html
M.-A. Lemburg mal at egenix.com

> Ralf Schmitt wrote:
>> Still trying to port our software. here's another thing I noticed:

>> d = {}
>> d[u'm\xe1s'] = 1
>> d['m\xe1s'] = 1
>> print d

(a 2-element dictionary, because they are not equal)

>> With python 2.5 I get: [ a traceback ending in ]

>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1:
>> ordinal not in range(128)

> Let's put it this way: Python 2.5 uncovered a bug in your
> application that has always been there.

No; he application would only have a bug if he expected those two
objects to compare equal.  Trying to stick something hashable into a
dictionary should not raise an Exception just because there is already
a similar key, (regardless of whether or not the other key is equal or
identical).

The only way this error could be the right thing is if you were trying
to suggest that he shouldn't mix unicode and bytestrings at all.
There is a command line switch for that, but it doesn't get much use.

-jJ

From mal at egenix.com  Thu Aug  3 23:47:10 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 03 Aug 2006 23:47:10 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <fb6fbf560608031410h3a72ae72meb885856b0922ee3@mail.gmail.com>
References: <fb6fbf560608031410h3a72ae72meb885856b0922ee3@mail.gmail.com>
Message-ID: <44D26EDE.6030907@egenix.com>

Jim Jewett wrote:
> http://mail.python.org/pipermail/python-dev/2006-August/067934.html
> M.-A. Lemburg mal at egenix.com
> 
>> Ralf Schmitt wrote:
>>> Still trying to port our software. here's another thing I noticed:
> 
>>> d = {}
>>> d[u'm\xe1s'] = 1
>>> d['m\xe1s'] = 1
>>> print d
> 
> (a 2-element dictionary, because they are not equal)
> 
>>> With python 2.5 I get: [ a traceback ending in ]
> 
>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1:
>>> ordinal not in range(128)
> 
>> Let's put it this way: Python 2.5 uncovered a bug in your
>> application that has always been there.
> 
> No; he application would only have a bug if he expected those two
> objects to compare equal.  Trying to stick something hashable into a
> dictionary should not raise an Exception just because there is already
> a similar key, (regardless of whether or not the other key is equal or
> identical).

Hmm, you have a point there...

>>> d = {}

# Two different objects
>>> x = 'a'
>>> y = hash(x)
>>> x
'a'
>>> y
12416037344

# ... with the same hash value
>>> hash(x)
12416037344
>>> hash(y)
12416037344

# Put them in the dictionary, causing a hash collision ...
>>> d[x] = 1
>>> d[y] = 2

# ... which is resolved by comparing the two for equality
# and assigning them to two different slots:
>>> d
{'a': 1, 12416037344: 2}

Since Python 2.5 propagates the compare exception, you get the
exception. Python 2.4 silenced the exception.

> The only way this error could be the right thing is if you were trying
> to suggest that he shouldn't mix unicode and bytestrings at all.

Good question. I wonder whether that's a reasonable approach for
Python 2.x (I'd say it is for Py3k).

Currently you can't safely mix non-ASCII string with Unicode
keys in the same dictionary.

Perhaps we ought to add an exception to the dict lookup mechanism
and continue to silence UnicodeErrors ?!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 03 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From rrr at ronadam.com  Fri Aug  4 00:00:04 2006
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 03 Aug 2006 17:00:04 -0500
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44D1E15F.9060908@gmail.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>	<44CAD9A1.8080000@canterbury.ac.nz>	<44CE4D32.6040909@ewtllc.com>	<44CEA091.1010406@canterbury.ac.nz>	<44CF20EE.70103@egenix.com>	<44CF38A7.8040304@canterbury.ac.nz>	<44CF3D96.7090101@egenix.com>	<44CFF9B9.3010402@canterbury.ac.nz>	<44D09A5C.4030905@egenix.com>	<44D15FB4.5020605@canterbury.ac.nz>	<44D16CFF.7090001@ewtllc.com>
	<44D198F2.9020702@canterbury.ac.nz>
	<44D1B47E.5020506@ronadam.com> <44D1E15F.9060908@gmail.com>
Message-ID: <44D271E4.9000009@ronadam.com>

Nick Coghlan wrote:
> Ron Adam wrote:
>> Consider an example where you are combining data that had different 
>> number of significant digits.  Keeping all the digits of your answer 
>> gives a false since of accuracy.  The extra digits are meaningless 
>> because the margin of error is greater than any of the digits that 
>> exceed the minimum number of significant digits in your data.  So you 
>> need to remove the extraneous digits by rounding.  Then this answer 
>> can be further used as data, and sense the number of significant 
>> digits is maintained, the margin of error can be calculated accurately.
> 
> This is a fallacy though - add two numbers together each with an error 
> of +/- 0.05, and the error in the total is +/- 0.1. The approach you 
> propose gives the misleading impression that the error in the total is 
> still only +/- 0.05 (as the answer will contain 1 decimal place).

I did say "the margin of error can be calculated accurately".  I did not 
go into detail on the correct method to do that.  You do need to round 
to the correct number of significant digits AND yes, you do need to also 
take into account the number of items in your data set.

If you don't round, the value of your answer would be shifted up or down 
by the amount of the non significant digits.  And the error range would 
also be shifted up or down by that amount.

In any case, my point is (and my opinion) that rounding to floating 
point types is very common and not something that is needed rarely as it 
has been expressed in several messages.  I don't want to debate the 
correct method of calculating errors and maintaining accuracy in 
statistical equations. I'll let the mathematicians do that.


> If you really care about error, you need to do it properly (e.g. stddev 
> or adding the error margins).

Yes


> Anyway, it's not proposed to remove the "round to x decimal places" 
> capability entirely - merely remove it from the builtin round() so that 
> the return type can be changed.

That would work.



This is one of those places where I wonder when should a function be 
included in a types methods and called by either using arguments with 
__init__, an external function to call a private method, or by having it 
be a public method to be called with the dot syntax like is common for 
strings?

If you look at the Decimal type, it has quite of few rounding methods, 
(as well as other non-underscored methods), while int, float, and long, 
do not have them.  Also the Decimal _round() method returns the number 
unchanged (max precision) if no precision argument is given where 
round() has a default precision of 0.

Will there be more efforts (in Py3k to unifiy Decimal and the other 
types so they are more alike in both the way they work and in how they 
are used?



Would it make since for the numeric type to have methods (other than the 
underscored ones) that can be called with the dot syntax like we do in 
unicode and the Decimal types that do not have the underscores?  Or 
would it be better to remove those in favor of builtin or module 
functions?  The extremes of both might be a foolish consistency, but 
this may also be a good area to make some basic improvements for python 
3k.  (I'm sure Guido and others here have though some on this in general.)

If these types gain a round method that all work the same way, then 
round() can then use those and it would return the same type as it is 
given. So then round(float(x)), round(int(x)), round(long(x)) and 
round(Decimal(x)) will return their own type respectfully.

And to get a specific type, you would call the types method directly.

    int.round(x)  -> integer

Which is nicer than int(round(x)).


And the round methods could have arguments to designate the type of 
rounding to use. Where 'f','c', and 'e' are floor, ceiling, and even 
rounding.

    int.round(x, 'c')     # 'f' is the default rounding mode

The functions round(), roundcieling(), and roundeven() could then call 
the method of the type with the appropriate argument, although I'd be 
happy without these round functions if there are round methods are 
consistent across types.


Well, this is the direction I would go in.  Hopefully by Python 9000 the 
decimal type or something equivalent will be as fast as all the others 
and/or computers will be so fast as to make it irrelevant,  then we can 
have just one scaler base type. ;-)

Cheers,
    Ron














From tdelaney at avaya.com  Fri Aug  4 01:24:12 2006
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Fri, 4 Aug 2006 09:24:12 +1000
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
Message-ID: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>

M.-A. Lemburg wrote:

> Perhaps we ought to add an exception to the dict lookup mechanism
> and continue to silence UnicodeErrors ?!

I'd definitely consider a UnicodeError to be an indication that two
objects are not equal. At the very least, in the context of a dictionary
lookup.

Tim Delaney

From Sjaak.van.Werkhoven at nl.fortis.com  Wed Aug  2 10:02:20 2006
From: Sjaak.van.Werkhoven at nl.fortis.com (Werkhoven J.P. van (Sjaak))
Date: Wed, 2 Aug 2006 10:02:20 +0200 
Subject: [Python-Dev] using globals
Message-ID: <AB7FC09A43F1EB46B9A11FB45F2B2FD507148EDA@dbacaa45.nl.fortis.bank>

Hi,

I have got a problem with importing global variables. For instance I have
got two files:

# t1.py							#t2.py

counter = 1				

def counter_adder(filenr):					def
show_adder():
   global counter						    import
t1
   counter+= 1						    print("adder is
%d" % t1.counter)
   if filenr == 1:
      show_adder()
   else:
      import t2
      t2.show_adder()

def show_adder():
   print("adder is %d" % counter)

>>counter_adder(1)
adder is 2
>>counter_adder(2)
adder is 1

When I look at the outcome of two function calls I expected no differences
but much to my surprise it goes wrong when the global variable is imported.
It doesn't give the updated value but the value it get when instantiated.
Who come? What should I do to get the updated value

greetings,

Sjaak van Werkhoven
 



****************************DISCLAIMER***********************************
Deze e-mail is uitsluitend bestemd voor de geadresseerde(n). Verstrekking aan en gebruik door anderen is niet toegestaan. Fortis sluit iedere aansprakelijkheid uit die voortvloeit uit electronische verzending. 

This e-mail is intended exclusively for the addressee(s), and may not be passed on to, or made available for use by any person other than the addressee(s). Fortis rules out any and every liability resulting from any electronic transmission. 
******************************************************************************

From oren at hishome.net  Thu Aug  3 22:11:16 2006
From: oren at hishome.net (Oren Tirosh)
Date: Thu, 3 Aug 2006 23:11:16 +0300
Subject: [Python-Dev] uuid module - byte order issue
Message-ID: <7168d65a0608031311g3479824ds18a9ede4a33cae7e@mail.gmail.com>

The UUID module uses network byte order, regardless of the platform
byte order. On little-endian platforms like Windows the ".bytes"
property of UUID objects is not compatible with the memory layout of
UUIDs:

>>> import uuid
>>> import pywintypes
>>> s = '{00112233-4455-6677-8899-aabbccddeeff}'
>>> uuid.UUID(s).bytes.encode('hex')
'00112233445566778899aabbccddeeff'
>>> str(buffer(pywintypes.IID(s))).encode('hex')
'33221100554477668899aabbccddeeff'
>>>

Ka-Ping Yee writes* that the Windows UUID generation calls are not RFC
4122 compliant and have an illegal version field. If the correct byte
order is used the UUIDs generated by Windows XP are valid version 4
UUIDs:

>>> parts = struct.unpack('<LHH8s', pywintypes.CreateGuid())
>>> parts[2] >> 12    # version
4
>>> ord(parts[3][0]) & 0xC0    # variant
128

The first three fields (32 bit time-low, 16 bit time-mid and
time-high-and-version) are stored in the platform byte order while the
remainder is stored as a vector of 8 bytes.

The bytes property and bytes argument to the constructor should use
the platform byte order. It would be nice to have explicit little
endian and big endian versions available on platforms of either
endianness for compatibility in communication and disk formats.


There is another issue with version 1 uuid generation:
>>> len(set(uuid.uuid1() for i in range(1000)))
992

The problem is that the random clock_seq field is only 14 bits long.
If enough UUIDs are generated within the same system clock tick there
will be collisions. Suggested solution: use the high-resolution of the
time field (100ns) to generate a monotonically increasing timestamp
that advances at least by 1 for each call, when time.time() returns
the same value on subsequent calls.

  Oren

[*] http://mail.python.org/pipermail/python-dev/2006-June/065869.html

From tjreedy at udel.edu  Fri Aug  4 02:50:06 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 3 Aug 2006 20:50:06 -0400
Subject: [Python-Dev] using globals
References: <AB7FC09A43F1EB46B9A11FB45F2B2FD507148EDA@dbacaa45.nl.fortis.bank>
Message-ID: <eau5jt$814$1@sea.gmane.org>


"Werkhoven J.P. van (Sjaak)" <Sjaak.van.Werkhoven at nl.fortis.com> wrote in 
news:AB7FC09A43F1EB46B9A11FB45F2B2FD507148EDA at dbacaa45.nl.fortis.bank...
> I have got a problem with importing global variables.

Questions about using Python belong on comp.lang.python or the general 
python mailing list.




From murman at gmail.com  Fri Aug  4 03:37:54 2006
From: murman at gmail.com (Michael Urman)
Date: Thu, 3 Aug 2006 20:37:54 -0500
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <44D234C3.4010404@egenix.com>
References: <eat5ds$37j$1@sea.gmane.org> <44D21A99.2080801@brainbot.com>
	<44D22989.7000202@egenix.com> <44D22FA7.7030905@brainbot.com>
	<44D234C3.4010404@egenix.com>
Message-ID: <dcbbbb410608031837k4d651da3n6ba96dcc5279acf5@mail.gmail.com>

On 8/3/06, M.-A. Lemburg <mal at egenix.com> wrote:
> > ...but in the case of dictionaries this behaviour has changed and in
> > prior versions of python dictionaries did work as I expected them to.
> > Now they don't.
>
> Let's put it this way: Python 2.5 uncovered a bug in your
> application that has always been there. It's better to
> fix your application than arguing to cover up the bug again.

I would understand this assertion if Ralf were expecting dictionaries
to consider
    { u'm\xe1s': 1, 'm\xe1s': 1 } == { u'm\xe1s': 1 } == { 'm\xe1s': 1 }
This is clearly a mess waiting to explode.

But that's not what he said. He expects, as is the case in python2.4,
    len({ u'm\xe1s': 1, 'm\xe1s': 1 }) == 2
because u'm\xe1s' clearly does not equal 'm\xe1s'. Because it raises
an exception, the dictionary shouldn't consider it equal, so there
should be the two keys which happen to be somewhat equivalent.

While this is in fact in the NEWS (Patch #1497053 & bug #1275608), I
think this should be raised for further discussion. Raising the
exception is good for debugging mistakes, but bad for dictionaries
holding holding inequal objects that happen to hash to the same value,
and correclty raise exceptions on comparison.

When we thought it was just a debugging tool, it made sense to put it
straight in to 2.5. Since it actually can adversely affect behavior in
only slightly edgy cases, perhaps it should go through a warning phase
(which ideally could show the exception that was thrown, thus yielding
most or all of the intended debugging advantage).

Michael
-- 
Michael Urman  http://www.tortall.net/mu/blog

From greg.ewing at canterbury.ac.nz  Fri Aug  4 03:51:14 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 04 Aug 2006 13:51:14 +1200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <44D1B4B6.2080403@egenix.com>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
	<44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com>
	<44D15FB4.5020605@canterbury.ac.nz> <44D1B4B6.2080403@egenix.com>
Message-ID: <44D2A812.20906@canterbury.ac.nz>

M.-A. Lemburg wrote:

> If you are eventually rounding to say 2 decimal
> places in the end of the calculation, you won't
> want to find yourself presenting the user 1.12
> and 1.13 as equal values :-)

Even if, before rounding, they were actually
1.12499999999 and 1.125000000001? And if the
difference were only due to the unrepresentability
of some decimal fraction exactly in binary?

I still maintain that (a) rounding a *binary*
float to *decimal* places is wrongheaded, and
(b) digit chopping is a bad way to decide
whether two inexact numbers should be
considered equal. Not just a different way,
but a poorer way.

> Most
> such calculations do work with floats, so having
> round() return an int would just add another
> coercion to a float for those use-cases.

I'm *not* proposing to eliminate round-to-float,
despite the fact that I can't see much use for
it personally.

I'm also *not* advocating changing the existing
behaviour of round() or int(). That was just
tentative speculation.

All I'm asking for is another function that does
round-to-int directly. I wouldn't have thought
that was such a controversial idea, given the
frequency of use for that operation.

--
Greg


From greg.ewing at canterbury.ac.nz  Fri Aug  4 03:51:32 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 04 Aug 2006 13:51:32 +1200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <44D26EDE.6030907@egenix.com>
References: <fb6fbf560608031410h3a72ae72meb885856b0922ee3@mail.gmail.com>
	<44D26EDE.6030907@egenix.com>
Message-ID: <44D2A824.6060502@canterbury.ac.nz>

M.-A. Lemburg wrote:

> Perhaps we ought to add an exception to the dict lookup mechanism
> and continue to silence UnicodeErrors ?!

Seems to be that comparison of unicode and non-unicode
strings for equality shouldn't raise exceptions in the
first place.

--
Greg

From foom at fuhm.net  Fri Aug  4 04:00:31 2006
From: foom at fuhm.net (James Y Knight)
Date: Thu, 3 Aug 2006 22:00:31 -0400
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <44D26EDE.6030907@egenix.com>
References: <fb6fbf560608031410h3a72ae72meb885856b0922ee3@mail.gmail.com>
	<44D26EDE.6030907@egenix.com>
Message-ID: <028CF4BD-EFF8-4D26-9918-C754AB7283D1@fuhm.net>

On Aug 3, 2006, at 5:47 PM, M.-A. Lemburg wrote:
>> The only way this error could be the right thing is if you were  
>> trying
>> to suggest that he shouldn't mix unicode and bytestrings at all.
>
> Good question. I wonder whether that's a reasonable approach for
> Python 2.x (I'd say it is for Py3k).

It's my understanding that in py3k, there will be no implicit  
conversion, bytestrings and unicodes will never be equal (no matter  
what the contents), and so this wouldn't be an issue. (as u"1" == "1"  
would be the same sort of situation as 1 == "1" is now)

James

From foom at fuhm.net  Fri Aug  4 04:02:34 2006
From: foom at fuhm.net (James Y Knight)
Date: Thu, 3 Aug 2006 22:02:34 -0400
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
	and coercing floats to integers)
In-Reply-To: <44D198F2.9020702@canterbury.ac.nz>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
	<44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com>
	<44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com>
	<44D198F2.9020702@canterbury.ac.nz>
Message-ID: <06264EFB-FE9A-4E94-B80D-C621D6BC7AD3@fuhm.net>


On Aug 3, 2006, at 2:34 AM, Greg Ewing wrote:

> Raymond Hettinger wrote:
>
>> -1 on an extra built-in just to save the time for function call
>
> The time isn't the main issue. The main issue
> is that almost all the use cases for round()
> involve doing an int() on it afterwards. At
> least nobody has put forward an argument to
> the contrary yet.

And I bet the main reason why round() in python returns a float is  
because it does in C.

And it does in C because C doesn't have arbitrary size integers, so  
if round returned integers, round(1e+308) couldn't work. In python,  
however, that's no problem, since python does have arbitrarily big  
integers.

There's also round(float("inf")), of course, which wouldn't be  
defined if the result was an integer, but I don't think rounding  
infinity is much of a use case.

And I do think the extension of round to allow the specification of  
number of decimal places was a mistake. If you want that, you  
probably really mean to do something like round(x * 10**y) instead.

James

From bob at redivi.com  Fri Aug  4 04:58:47 2006
From: bob at redivi.com (Bob Ippolito)
Date: Thu, 3 Aug 2006 19:58:47 -0700
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <44D2A824.6060502@canterbury.ac.nz>
References: <fb6fbf560608031410h3a72ae72meb885856b0922ee3@mail.gmail.com>
	<44D26EDE.6030907@egenix.com> <44D2A824.6060502@canterbury.ac.nz>
Message-ID: <DEA27711-509B-4F5F-9D49-ABF5ECD2BB20@redivi.com>


On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote:

> M.-A. Lemburg wrote:
>
>> Perhaps we ought to add an exception to the dict lookup mechanism
>> and continue to silence UnicodeErrors ?!
>
> Seems to be that comparison of unicode and non-unicode
> strings for equality shouldn't raise exceptions in the
> first place.

Seems like a slightly better idea than having dictionaries suppress  
exceptions. Still not ideal though because sticking non-ASCII strings  
that are supposed to be text and unicode in the same data structures  
is *probably* still an error.

-bob


From aahz at pythoncraft.com  Fri Aug  4 05:22:00 2006
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 3 Aug 2006 20:22:00 -0700
Subject: [Python-Dev] Rounding float to int directly ...
In-Reply-To: <44CFC2AD.2070205@ronadam.com>
References: <E1G7u34-0004Ka-N0@draco.cus.cam.ac.uk>
	<44CFC2AD.2070205@ronadam.com>
Message-ID: <20060804032200.GA6515@panix.com>

On Tue, Aug 01, 2006, Ron Adam wrote:
>
> I'm -1 on implicitly converting to an int when rounding.
> 
> One reason is if your rounded (to int type) number is then used in an 
> equation you my end up with a integer result when you wanted a floating 
> point result.
> 
>  >>> 23.5/5.2
> 4.5192307692307692
>  >>> round(23.5)/round(5.2)
> 4.7999999999999998
>  >>> round(23.5/5.2)
> 5.0
>  >>> int(round(23.5))/int(round(5.2))
> 4

That's not relevant to Py3K.  Which is where this discussion should
proably move because round() ain't changing in 2.x.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it."  --Brian W. Kernighan

From jcarlson at uci.edu  Fri Aug  4 06:34:04 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 03 Aug 2006 21:34:04 -0700
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <DEA27711-509B-4F5F-9D49-ABF5ECD2BB20@redivi.com>
References: <44D2A824.6060502@canterbury.ac.nz>
	<DEA27711-509B-4F5F-9D49-ABF5ECD2BB20@redivi.com>
Message-ID: <20060803212410.E6B5.JCARLSON@uci.edu>


Bob Ippolito <bob at redivi.com> wrote:
> On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote:
> 
> > M.-A. Lemburg wrote:
> >
> >> Perhaps we ought to add an exception to the dict lookup mechanism
> >> and continue to silence UnicodeErrors ?!
> >
> > Seems to be that comparison of unicode and non-unicode
> > strings for equality shouldn't raise exceptions in the
> > first place.
> 
> Seems like a slightly better idea than having dictionaries suppress  
> exceptions. Still not ideal though because sticking non-ASCII strings  
> that are supposed to be text and unicode in the same data structures  
> is *probably* still an error.

If/when 'python -U -c "import test.testall"' runs without unexpected
error (I doubt it will happen prior to the "all strings are unicode"
conversion), then I think that we can say that there aren't any
use-cases for strings and unicode being in the same dictionary.

As an alternate idea, rather than attempting to .decode('ascii') when
strings and unicode compare, why not .decode('latin-1')?  We lose the
unicode decoding error, but "the right thing" happens (in my opinion)
when u'\xa1' and '\xa1' compare.

 - Josiah


From foom at fuhm.net  Fri Aug  4 06:39:29 2006
From: foom at fuhm.net (James Y Knight)
Date: Fri, 4 Aug 2006 00:39:29 -0400
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <20060803212410.E6B5.JCARLSON@uci.edu>
References: <44D2A824.6060502@canterbury.ac.nz>
	<DEA27711-509B-4F5F-9D49-ABF5ECD2BB20@redivi.com>
	<20060803212410.E6B5.JCARLSON@uci.edu>
Message-ID: <1265F73D-C13D-4329-96C1-C8FD09D4F079@fuhm.net>


On Aug 4, 2006, at 12:34 AM, Josiah Carlson wrote:
> As an alternate idea, rather than attempting to .decode('ascii') when
> strings and unicode compare, why not .decode('latin-1')?  We lose the
> unicode decoding error, but "the right thing" happens (in my opinion)
> when u'\xa1' and '\xa1' compare.

Maybe you want those to compare equal, but _I_ want u'\xa1' and '\xc2 
\xa1' to compare equal, so it should obviously use .decode('utf-8')!

(okay, no, I don't really want that.)

James

From anthony at python.org  Fri Aug  4 06:41:24 2006
From: anthony at python.org (Anthony Baxter)
Date: Fri, 4 Aug 2006 14:41:24 +1000
Subject: [Python-Dev] RELEASED Python 2.5 (beta 3)
Message-ID: <200608041441.32253.anthony@python.org>

On behalf of the Python development team and the Python
community, I'm happy to announce the third BETA release
of Python 2.5.

This is an *beta* release of Python 2.5. As such, it is not
suitable for a production environment. It is being released
to solicit feedback and hopefully discover bugs, as well as
allowing you to determine how changes in 2.5 might impact
you. If you find things broken or incorrect, please log a
bug on Sourceforge.

In particular, note that changes to improve Python's support
of 64 bit systems might require authors of C extensions
to change their code. More information (as well as source
distributions and Windows and Universal Mac OSX installers) are
available from the 2.5 website:

    http://www.python.org/2.5/

There's been over 50 fixes since the second beta. This
large number of changes meant we felt more comfortable
cutting a third beta release, rather than charging ahead to
the release candidate.

As of this release, Python 2.5 is now in *feature freeze*.
Unless absolutely necessary, no functionality changes will
be made between now and the final release of Python 2.5.

The plan is that this will be the final beta release (no,
really, this time for sure (probably)). We should now move
to one or more release candidates, leading to a 2.5 final
release early August. PEP 356 includes the schedule and will
be updated as the schedule evolves. At this point, any
testing you can do would be greatly, greatly appreciated.

The new features in Python 2.5 are described in Andrew
Kuchling's What's New In Python 2.5. It's available from
the 2.5 web page.

Amongst the language features added include conditional
expressions, the with statement, the merge of try/except
and try/finally into try/except/finally, enhancements to
generators to produce a coroutine kind of functionality, and
a brand new AST-based compiler implementation.

New modules added include hashlib, ElementTree, sqlite3,
wsgiref and ctypes. In addition, a new profiling module
"cProfile" was added.

Enjoy this new release,
Anthony

Anthony Baxter
anthony at python.org
Python Release Manager
(on behalf of the entire python-dev team)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060804/5d50035d/attachment.pgp 

From exarkun at divmod.com  Fri Aug  4 06:43:23 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 4 Aug 2006 00:43:23 -0400
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <20060803212410.E6B5.JCARLSON@uci.edu>
Message-ID: <20060804044323.1717.140314887.divmod.quotient.13837@ohm>

On Thu, 03 Aug 2006 21:34:04 -0700, Josiah Carlson <jcarlson at uci.edu> wrote:
>
>Bob Ippolito <bob at redivi.com> wrote:
>> On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote:
>>
>> > M.-A. Lemburg wrote:
>> >
>> >> Perhaps we ought to add an exception to the dict lookup mechanism
>> >> and continue to silence UnicodeErrors ?!
>> >
>> > Seems to be that comparison of unicode and non-unicode
>> > strings for equality shouldn't raise exceptions in the
>> > first place.
>>
>> Seems like a slightly better idea than having dictionaries suppress
>> exceptions. Still not ideal though because sticking non-ASCII strings
>> that are supposed to be text and unicode in the same data structures
>> is *probably* still an error.
>
>If/when 'python -U -c "import test.testall"' runs without unexpected
>error (I doubt it will happen prior to the "all strings are unicode"
>conversion), then I think that we can say that there aren't any
>use-cases for strings and unicode being in the same dictionary.
>
>As an alternate idea, rather than attempting to .decode('ascii') when
>strings and unicode compare, why not .decode('latin-1')?  We lose the
>unicode decoding error, but "the right thing" happens (in my opinion)
>when u'\xa1' and '\xa1' compare.

It might be right for Latin-1 strings.

However, it would be even *more* surprising for the person who has to
figure out why his program works when his program gets a string containing
'\xc0' from one user but fails when it gets '\xe3\x81\x82' from another
user.

I like the exception that 2.5 raises.  I only wish it raised by default
when using 'ascii' and u'ascii' as keys in the same dictionary. ;)  Oh,
and that str and unicode did not hash like they do.  ;)

Jean-Paul

From anthony at interlink.com.au  Fri Aug  4 06:43:39 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri, 4 Aug 2006 14:43:39 +1000
Subject: [Python-Dev] TRUNK is UNFROZEN
In-Reply-To: <200608022229.46957.anthony@interlink.com.au>
References: <200608022229.46957.anthony@interlink.com.au>
Message-ID: <200608041443.40668.anthony@interlink.com.au>

The trunk is unfrozen now. Sorry about the delay - combination of a laptop 
motherboard replacement and a DSL provider with a network that was .. ahem... 
slightly slow. 

I'm still planning to branch at 2.5c1, the next release. That then opens the 
trunk up for all the bad craziness that's planned for 2.6.

Anthony

From murman at gmail.com  Fri Aug  4 06:45:05 2006
From: murman at gmail.com (Michael Urman)
Date: Thu, 3 Aug 2006 23:45:05 -0500
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <20060803212410.E6B5.JCARLSON@uci.edu>
References: <44D2A824.6060502@canterbury.ac.nz>
	<DEA27711-509B-4F5F-9D49-ABF5ECD2BB20@redivi.com>
	<20060803212410.E6B5.JCARLSON@uci.edu>
Message-ID: <dcbbbb410608032145t47d00943s9b292ad51c080539@mail.gmail.com>

On 8/3/06, Josiah Carlson <jcarlson at uci.edu> wrote:
> As an alternate idea, rather than attempting to .decode('ascii') when
> strings and unicode compare, why not .decode('latin-1')?  We lose the
> unicode decoding error, but "the right thing" happens (in my opinion)
> when u'\xa1' and '\xa1' compare.

Since I use utf-8 way more than I use latin-1, -1. Since others do
not, -1 on any not obviously correct encoding other than ascii, which
gets grandfathered.

This raises an exception for a good reason. Yes it's annoying at
times. We should fix those times, not the (unbroken) exception.

Michael
-- 
Michael Urman  http://www.tortall.net/mu/blog

From nnorwitz at gmail.com  Fri Aug  4 07:49:22 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 3 Aug 2006 22:49:22 -0700
Subject: [Python-Dev] 2.5 status
Message-ID: <ee2a432c0608032249i5779606btb24a2324204239e3@mail.gmail.com>

Things are getting better, but we still have some really important
outstanding issues.  PLEASE CONTINUE TESTING AS MUCH AS POSSIBLE.
Also, it would be great to use as many tools as possible to find bugs
and improve quality.  It would be especially nice to run Purify on
Windows.

I've updated PEP 356 with the current status.  The current schedule is:

    rc 1:    August 18, 2006 [planned]
    final:   September 12, 2006 [planned]

This somewhat strange schedule is to accommodate availability of
people cutting the release.  A branch will be made when the release
candidate is done.  Don't try to rush bug fixes in.  Let's try to keep
things stable.  Keep the doc fixes coming.

I believe all 3 outstanding issues (and solutions!) could use some
more discussion.  All bugs/patches blocking release are set to
priority 9.

        http://python.org/sf/1530559 - struct rejecting floats (patch pending)

        http://mail.python.org/pipermail/python-dev/2006-July/067774.html
                Problem with __index__ (patch pending)

        http://mail.python.org/pipermail/python-dev/2006-August/067926.html
                str/unicode dict keys can raise an exception

All the AST issues that I know about have been addressed.

There are various other issues that would be nice to fix for 2.5.
These are priority 8.  All exist in 2.4 and possibly earlier:

        http://python.org/sf/1526585 - SystemError concat long strings
        http://python.org/sf/1523610 - PyArg_ParseTupleAndKeywords
potential core dump
        http://python.org/sf/1517042 - Fix crashers/gc_inspection.py
        http://python.org/sf/1475523 - gettext.py bug
        http://python.org/sf/1467929 - %-formatting and dicts

I will continue to use priorities.  Please review bugs and patches and
let me know if anything else should be re-prioritized.

I will continue to stress test Python and use various tools to keep
quality up.  The current suite includes in no particular order:
valgrind, Coverity, Klocwork, failmalloc, pychecker, not to mention
the buildbot testing on commits and refleak test runs every 12 hours.

Cheers,
n

From mal at egenix.com  Fri Aug  4 09:25:07 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 04 Aug 2006 09:25:07 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>
Message-ID: <44D2F653.5090305@egenix.com>

Delaney, Timothy (Tim) wrote:
> M.-A. Lemburg wrote:
> 
>> Perhaps we ought to add an exception to the dict lookup mechanism
>> and continue to silence UnicodeErrors ?!
> 
> I'd definitely consider a UnicodeError to be an indication that two
> objects are not equal. 

Not really: Python expects all strings to be ASCII whenever they
meet Unicode strings and have to be converted to Unicode. If a string
is not ASCII and thus causes the exception, there's not a lot you
can say, since you don't know the encoding of the string. All you
know is that it's not ASCII. Instead of guessing, Python then raises
an exception to let the programmer decide.

> At the very least, in the context of a dictionary
> lookup.

The point here is that a typical user won't expect any comparisons
to be made when dealing with dictionaries, simply because the fact
that you do need to make comparisons is an implementation detail.

So in this particular case silencing the exception might be the
more user friendly way of dealing with the problem.

That said, the problem still lingers in that dictionary, so it may
bite you in some other context, e.g. when iterating over the list
of keys.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From rasky at develer.com  Fri Aug  4 10:58:16 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Fri, 4 Aug 2006 10:58:16 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>
	<44D2F653.5090305@egenix.com>
Message-ID: <141f01c6b7a4$2001b460$8cb22997@bagio>

M.-A. Lemburg <mal at egenix.com> wrote:

>> At the very least, in the context of a dictionary
>> lookup.
>
> The point here is that a typical user won't expect any comparisons
> to be made when dealing with dictionaries, simply because the fact
> that you do need to make comparisons is an implementation detail.


I'd also weight in the fact that this is a change of behaviour since the
introduction of unicode in 2.0, and it's break real-world applications (as
demonstrated by this thread). I think Python ought to revert to the previous
behaviour and, in case there's interest, the discussion moved to the py3k list.

Giovanni Bajo


From greg.ewing at canterbury.ac.nz  Fri Aug  4 11:14:57 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 04 Aug 2006 21:14:57 +1200
Subject: [Python-Dev] Rounding float to int directly (Re: struct module
 and coercing floats to integers)
In-Reply-To: <06264EFB-FE9A-4E94-B80D-C621D6BC7AD3@fuhm.net>
References: <9E51A030-B610-414B-98E0-1923F1C3E862@redivi.com>
	<dcbbbb410607282014w30459351ifb09c56075997be1@mail.gmail.com>
	<44CAD9A1.8080000@canterbury.ac.nz> <44CE4D32.6040909@ewtllc.com>
	<44CEA091.1010406@canterbury.ac.nz> <44CF20EE.70103@egenix.com>
	<44CF38A7.8040304@canterbury.ac.nz> <44CF3D96.7090101@egenix.com>
	<44CFF9B9.3010402@canterbury.ac.nz> <44D09A5C.4030905@egenix.com>
	<44D15FB4.5020605@canterbury.ac.nz> <44D16CFF.7090001@ewtllc.com>
	<44D198F2.9020702@canterbury.ac.nz>
	<06264EFB-FE9A-4E94-B80D-C621D6BC7AD3@fuhm.net>
Message-ID: <44D31011.4040607@canterbury.ac.nz>

James Y Knight wrote:

> And it does in C because C doesn't have arbitrary size integers, so  if 
> round returned integers, round(1e+308) couldn't work.

Also, in C you can use the result directly in an int
context without complaint. In Python these days, that
is usually disallowed.

--
Greg

From ralf at brainbot.com  Fri Aug  4 11:18:28 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Fri, 04 Aug 2006 11:18:28 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <20060804044323.1717.140314887.divmod.quotient.13837@ohm>
References: <20060803212410.E6B5.JCARLSON@uci.edu>
	<20060804044323.1717.140314887.divmod.quotient.13837@ohm>
Message-ID: <44D310E4.10304@brainbot.com>

Jean-Paul Calderone wrote:
> 
> I like the exception that 2.5 raises.  I only wish it raised by default
> when using 'ascii' and u'ascii' as keys in the same dictionary. ;)  Oh,
> and that str and unicode did not hash like they do.  ;)

No problem:

 >>> import sys
 >>> reload(sys)
<module 'sys' (built-in)>
 >>> sys.setdefaultencoding("base64")
 >>> "a"==u"a"
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/exp/lib/python2.5/encodings/base64_codec.py", line 42, in 
base64_decode
     output = base64.decodestring(input)
   File "/exp/lib/python2.5/base64.py", line 321, in decodestring
     return binascii.a2b_base64(s)
binascii.Error: Incorrect padding
 >>> "a"=="a"
True
 >>> d={u"a":1, "a":1}
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/exp/lib/python2.5/encodings/base64_codec.py", line 42, in 
base64_decode
     output = base64.decodestring(input)
   File "/exp/lib/python2.5/base64.py", line 321, in decodestring
     return binascii.a2b_base64(s)
binascii.Error: Incorrect padding

Maybe this is all just a matter of choosing the right defaultencoding ? :)


BTW, python 2.4 also suppresses this exception (when instantiating the 
dictionary)

Does python 2.4 catch any exception when comparing keys (which are not 
basestrings) in dictionaries?


- Ralf


From mal at egenix.com  Fri Aug  4 11:26:42 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 04 Aug 2006 11:26:42 +0200
Subject: [Python-Dev] 2.5 status
In-Reply-To: <ee2a432c0608032249i5779606btb24a2324204239e3@mail.gmail.com>
References: <ee2a432c0608032249i5779606btb24a2324204239e3@mail.gmail.com>
Message-ID: <44D312D2.9020608@egenix.com>

Neal Norwitz wrote:
> I believe all 3 outstanding issues (and solutions!) could use some
> more discussion.  All bugs/patches blocking release are set to
> priority 9.
> 
>         http://python.org/sf/1530559 - struct rejecting floats (patch pending)
> 
>         http://mail.python.org/pipermail/python-dev/2006-July/067774.html
>                 Problem with __index__ (patch pending)
> 
>         http://mail.python.org/pipermail/python-dev/2006-August/067926.html
>                 str/unicode dict keys can raise an exception

I'd like to add this issue:

	http://mail.python.org/pipermail/python-dev/2006-August/067869.html
	distutils version string should become static again

This is part of a longer discussion we can have after 2.6
final is out, but for now, I would like to request that the
change

http://mail.python.org/pipermail/python-checkins/2006-July/055139.html

be backed out again before c1 gets released.

In the above patch, the distutils version was made to dynamically
depend on the Python version running the code, effectively
making it impossible to query the true distutils version that
is being used, bypassing the whole purpose of having static
version in the code.

I don't really see manually bumping the version number every
now and then to be such a great issue and have also volunteered
to take care of that in the future.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Fri Aug  4 11:29:38 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 04 Aug 2006 11:29:38 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D310E4.10304@brainbot.com>
References: <20060803212410.E6B5.JCARLSON@uci.edu>	<20060804044323.1717.140314887.divmod.quotient.13837@ohm>
	<44D310E4.10304@brainbot.com>
Message-ID: <44D31382.80605@egenix.com>

Ralf Schmitt wrote:
> Does python 2.4 catch any exception when comparing keys (which are not 
> basestrings) in dictionaries?

Yes. It does so for all equality compares that need to be done
as part of the hash collision algorithm (not only w/r to strings
and Unicode, but in general).

This was changed in 2.5, which now reports the exception.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From greg.ewing at canterbury.ac.nz  Fri Aug  4 11:31:54 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 04 Aug 2006 21:31:54 +1200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <44D2F653.5090305@egenix.com>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>
	<44D2F653.5090305@egenix.com>
Message-ID: <44D3140A.2040600@canterbury.ac.nz>

M.-A. Lemburg wrote:

> If a string
> is not ASCII and thus causes the exception, there's not a lot you
> can say, since you don't know the encoding of the string.

That's one way of looking at it.

Another is that any string containing chars > 127 is not
text at all, but binary data, in which case it's not equal
to *any* unicode string -- just like bytes objects will
not be equal to strings in Py3k.

> All you
> know is that it's not ASCII. Instead of guessing, Python then raises
> an exception to let the programmer decide.

There's no disputing that an exception should be raised
if the string *must* be interpretable as characters in
order to continue. But that's not true here if you allow
for the interpretation that they're simply objects of
different (duck) type and therefore unequal.

--
Greg


From ralf at brainbot.com  Fri Aug  4 11:42:11 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Fri, 04 Aug 2006 11:42:11 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <44D31382.80605@egenix.com>
References: <20060803212410.E6B5.JCARLSON@uci.edu>	<20060804044323.1717.140314887.divmod.quotient.13837@ohm>	<44D310E4.10304@brainbot.com>
	<44D31382.80605@egenix.com>
Message-ID: <44D31673.9040307@brainbot.com>

M.-A. Lemburg wrote:
> Ralf Schmitt wrote:
>> Does python 2.4 catch any exception when comparing keys (which are not 
>> basestrings) in dictionaries?
> 
> Yes. It does so for all equality compares that need to be done
> as part of the hash collision algorithm (not only w/r to strings
> and Unicode, but in general).
> 
> This was changed in 2.5, which now reports the exception.
> 

So, this thread isn't about "unicode hell" at all.
I guess this change will break lots of code (or will reveal lots of 
broken code...as it did in my case actually).

- Ralf


From g.brandl at gmx.net  Fri Aug  4 11:49:24 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 04 Aug 2006 11:49:24 +0200
Subject: [Python-Dev] 2.5 status
In-Reply-To: <ee2a432c0608032249i5779606btb24a2324204239e3@mail.gmail.com>
References: <ee2a432c0608032249i5779606btb24a2324204239e3@mail.gmail.com>
Message-ID: <eav54q$jcp$1@sea.gmane.org>

Neal Norwitz wrote:
> Things are getting better, but we still have some really important
> outstanding issues.  PLEASE CONTINUE TESTING AS MUCH AS POSSIBLE.
> Also, it would be great to use as many tools as possible to find bugs
> and improve quality.  It would be especially nice to run Purify on
> Windows.
> 
> I've updated PEP 356 with the current status.  The current schedule is:
> 
>     rc 1:    August 18, 2006 [planned]
>     final:   September 12, 2006 [planned]
> 
> This somewhat strange schedule is to accommodate availability of
> people cutting the release.  A branch will be made when the release
> candidate is done.  Don't try to rush bug fixes in.  Let's try to keep
> things stable.  Keep the doc fixes coming.

>         http://python.org/sf/1523610 - PyArg_ParseTupleAndKeywords
> potential core dump

This one's almost fixed if we can decide what to do with "levels".
I wrote some time ago:

"""
With respect to this bug (which is about stack issues in Python/getargs.c
involving misuse of the "levels" array), I think that we can drop the
"levels" thing completely. It's only there to tell the user which exact item
passed as part of a tuple argument cannot be accepted (and only if that
function is implemented in C code). As tuple arguments
are very rare "argument x" should be enough to tell the user that
something's wrong with that tuple.
"""

Georg


From mwh at python.net  Fri Aug  4 12:06:11 2006
From: mwh at python.net (Michael Hudson)
Date: Fri, 04 Aug 2006 11:06:11 +0100
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D2F653.5090305@egenix.com> (M.'s message of "Fri, 04 Aug
	2006 09:25:07 +0200")
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>
	<44D2F653.5090305@egenix.com>
Message-ID: <2mejvwx1bg.fsf@starship.python.net>

"M.-A. Lemburg" <mal at egenix.com> writes:

> The point here is that a typical user won't expect any comparisons
> to be made when dealing with dictionaries, simply because the fact
> that you do need to make comparisons is an implementation detail.

Of course looking things up in a dictionary involves comparisons!  How
could it not?

> So in this particular case silencing the exception might be the
> more user friendly way of dealing with the problem.

Please, no.

> That said, the problem still lingers in that dictionary, so it may
> bite you in some other context, e.g. when iterating over the list
> of keys.

For this reason, and others.

Cheers,
mwh

-- 
  <dash> web in my head get it out get it out
                                                -- from Twisted.Quotes

From mal at egenix.com  Fri Aug  4 12:16:04 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 04 Aug 2006 12:16:04 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D3140A.2040600@canterbury.ac.nz>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>	<44D2F653.5090305@egenix.com>
	<44D3140A.2040600@canterbury.ac.nz>
Message-ID: <44D31E64.3070407@egenix.com>

Greg Ewing wrote:
> M.-A. Lemburg wrote:
> 
>> If a string
>> is not ASCII and thus causes the exception, there's not a lot you
>> can say, since you don't know the encoding of the string.
> 
> That's one way of looking at it.
> 
> Another is that any string containing chars > 127 is not
> text at all, but binary data, in which case it's not equal
> to *any* unicode string -- just like bytes objects will
> not be equal to strings in Py3k.
> 
>> All you
>> know is that it's not ASCII. Instead of guessing, Python then raises
>> an exception to let the programmer decide.
> 
> There's no disputing that an exception should be raised
> if the string *must* be interpretable as characters in
> order to continue. But that's not true here if you allow
> for the interpretation that they're simply objects of
> different (duck) type and therefore unequal.

Hmm, given that interpretation, 1 == 1.0 would have to be
False.

Note that you do have to interpret the string as characters
if you compare it to Unicode and there's nothing wrong with
that.

What's making this particular case interesting is that
the comparison is hidden in the dictionary implementation
and only triggers if you get a hash collision, which makes
the whole issue appear to be happening randomly.

This whole thread aside: it's never recommended to mix strings
and Unicode, unless you really have to.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Fri Aug  4 12:22:35 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 04 Aug 2006 12:22:35 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D31673.9040307@brainbot.com>
References: <20060803212410.E6B5.JCARLSON@uci.edu>	<20060804044323.1717.140314887.divmod.quotient.13837@ohm>	<44D310E4.10304@brainbot.com>	<44D31382.80605@egenix.com>
	<44D31673.9040307@brainbot.com>
Message-ID: <44D31FEB.9000806@egenix.com>

Ralf Schmitt wrote:
> M.-A. Lemburg wrote:
>> Ralf Schmitt wrote:
>>> Does python 2.4 catch any exception when comparing keys (which are not 
>>> basestrings) in dictionaries?
>> Yes. It does so for all equality compares that need to be done
>> as part of the hash collision algorithm (not only w/r to strings
>> and Unicode, but in general).
>>
>> This was changed in 2.5, which now reports the exception.
>>
> 
> So, this thread isn't about "unicode hell" at all.

For some reason people always think of hell when dealing with Unicode.
Instead, the should think of hell when dealing with strings.

> I guess this change will break lots of code (or will reveal lots of 
> broken code...as it did in my case actually).

I don't think it will break a lot of code that wasn't already
broken.

Let's see how many reports we get for 2.5b3 and then decide.

If things turn out bad, we might silence the UnicodeError
and instead issue a warning everytime this situation occurs.
In 2.6 we'd then revert to raising an exception.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From bob at redivi.com  Fri Aug  4 12:50:30 2006
From: bob at redivi.com (Bob Ippolito)
Date: Fri, 4 Aug 2006 03:50:30 -0700
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <20060803212410.E6B5.JCARLSON@uci.edu>
References: <44D2A824.6060502@canterbury.ac.nz>
	<DEA27711-509B-4F5F-9D49-ABF5ECD2BB20@redivi.com>
	<20060803212410.E6B5.JCARLSON@uci.edu>
Message-ID: <9A11B598-DEF0-4ED7-9D17-789BB89E05A5@redivi.com>


On Aug 3, 2006, at 9:34 PM, Josiah Carlson wrote:

>
> Bob Ippolito <bob at redivi.com> wrote:
>> On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote:
>>
>>> M.-A. Lemburg wrote:
>>>
>>>> Perhaps we ought to add an exception to the dict lookup mechanism
>>>> and continue to silence UnicodeErrors ?!
>>>
>>> Seems to be that comparison of unicode and non-unicode
>>> strings for equality shouldn't raise exceptions in the
>>> first place.
>>
>> Seems like a slightly better idea than having dictionaries suppress
>> exceptions. Still not ideal though because sticking non-ASCII strings
>> that are supposed to be text and unicode in the same data structures
>> is *probably* still an error.
>
> If/when 'python -U -c "import test.testall"' runs without unexpected
> error (I doubt it will happen prior to the "all strings are unicode"
> conversion), then I think that we can say that there aren't any
> use-cases for strings and unicode being in the same dictionary.
>
> As an alternate idea, rather than attempting to .decode('ascii') when
> strings and unicode compare, why not .decode('latin-1')?  We lose the
> unicode decoding error, but "the right thing" happens (in my opinion)
> when u'\xa1' and '\xa1' compare.

Well, in this case it would cause different behavior if u'\xa1' and  
'\xa1' compared equal. It'd just be an even more subtle error.

-bob


From kristjan at ccpgames.com  Fri Aug  4 13:41:21 2006
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_V=2E_J=F3nsson?=)
Date: Fri, 4 Aug 2006 11:41:21 -0000
Subject: [Python-Dev] unicode hell/mixing str and unicode as
	dictionarykeys
Message-ID: <129CEF95A523704B9D46959C922A280002FE9868@nemesis.central.ccp.cc>

The "string" isn?t necessarily text, so selecting latin-1 doesn?t help  (in fact, what happens is that the current default encoding is used, in his case this was ascii).  What if it is image data?  What if you are using a dict to implement a singleton set for arbitrary objects?

The point is that if the comparison operator raises an exception, the two objects are likely to be dissimilar.  We could even define that behaviour.  Propagating the exception means that you can't have objects as keys in a dictionary that raise an exception when compared.  This goes over and beyond any unicode vs. string question.

If the propagation of the exception was a concious change for debugging purposes, why not make that somehow optional?  A flag on the dict object?  Or special lookup mehtods for that?

Cheers,
Kristj?n

> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org 
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] 
> On Behalf Of Josiah Carlson
> Sent: 4. ?g?st 2006 04:34
> To: Bob Ippolito; python-dev at python.org
> Subject: Re: [Python-Dev] unicode hell/mixing str and unicode 
> as dictionarykeys
> 
> 
> Bob Ippolito <bob at redivi.com> wrote:
> > On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote:
> > 
> > > M.-A. Lemburg wrote:
> > >
> > >> Perhaps we ought to add an exception to the dict lookup 
> mechanism 
> > >> and continue to silence UnicodeErrors ?!
> > >
> > > Seems to be that comparison of unicode and non-unicode 
> strings for 
> > > equality shouldn't raise exceptions in the first place.
> > 
> > Seems like a slightly better idea than having dictionaries suppress 
> > exceptions. Still not ideal though because sticking 
> non-ASCII strings 
> > that are supposed to be text and unicode in the same data 
> structures 
> > is *probably* still an error.
> 
> If/when 'python -U -c "import test.testall"' runs without 
> unexpected error (I doubt it will happen prior to the "all 
> strings are unicode"
> conversion), then I think that we can say that there aren't 
> any use-cases for strings and unicode being in the same dictionary.
> 
> As an alternate idea, rather than attempting to 
> .decode('ascii') when strings and unicode compare, why not 
> .decode('latin-1')?  We lose the unicode decoding error, but 
> "the right thing" happens (in my opinion) when u'\xa1' and 
> '\xa1' compare.
> 
>  - Josiah
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/kristjan%40c
cpgames.com
> 

From mcherm at mcherm.com  Fri Aug  4 14:26:52 2006
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri, 04 Aug 2006 05:26:52 -0700
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
	unicode	asdictionarykeys
Message-ID: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>

I'm changing the subject line because I want to convince everyone that
the problem being discussed in the "unicode hell" thread has nothing
to do with unicode and strings. It's all about dicts.

I have not observed real breakage in my own code, but I will present
a sample of made-up-but-completely-reasonable code that works in
2.4, fails in 2.5, and arguably ought to work fine. I think we should
restore the behavior of dicts that when they compare keys for
equality they suppress exceptions (treating the objects as unequal),
or at LEAST retain the behavior for one more release making it a
warning this time.

Here is my sample code:

----------- problem_with_dicts.py ----------
# A sample program to demonstrate that the proposed behavior
# of dicts in Python 2.5 generates bugs. This is not code from
# an actual program, but it is completely reasonable.

# First import from
# http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413486
# the only 5-star recipe in the Python Cookbook for implementing
# enums.
import cookbookenum

# Then set up some reasonable enums. We'll say we're writing
# a program for dealing with dates.
DaysOfWeek = cookbookenum.Enum(
     'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun')
Months = cookbookenum.Enum(
     'Jan','Feb','Mar','Apr','May','Jun','Jul','Aug',
     'Sep','Oct','Nov','Dec')

# Let's say we also do some translations. Here is a
# useful dictionary:
translations = {}
# which we populate with values
translations[ DaysOfWeek.Mon ] = {
     'en': 'Monday',
     'es': 'Lunes',
     'fr': 'Lundi',
     }
# and assume we do the other days
translations[ Months.Jan ] = {
     'en': 'January',
     'es': 'Enero',
     'fr': 'Janvier',
     }
# and assume we do the other months

# ...then later in the code we could do things like this:
language = 'en'
dayOfWeek = DaysOfWeek.Mon
month = Months.Jan
dayOfMonth = 3
print '%s, %s %s' % (
     translations[dayOfWeek][language],
     translations[month][language],
     dayOfMonth)

# this works in 2.4 but fails in 2.5
--------- end problem_with_dicts.py --------

Please reconsider.

-- Michael Chermside


From radix at twistedmatrix.com  Fri Aug  4 16:01:51 2006
From: radix at twistedmatrix.com (Christopher Armstrong)
Date: Fri, 4 Aug 2006 10:01:51 -0400
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <44D310E4.10304@brainbot.com>
References: <20060803212410.E6B5.JCARLSON@uci.edu>
	<20060804044323.1717.140314887.divmod.quotient.13837@ohm>
	<44D310E4.10304@brainbot.com>
Message-ID: <60ed19d40608040701v59474febw94f7f374ad47d92a@mail.gmail.com>

On 8/4/06, Ralf Schmitt <ralf at brainbot.com> wrote:
>
> Jean-Paul Calderone wrote:
> >
> > I like the exception that 2.5 raises.  I only wish it raised by default
> > when using 'ascii' and u'ascii' as keys in the same dictionary. ;)  Oh,
> > and that str and unicode did not hash like they do.  ;)
>
> No problem:
>
> >>> import sys
> >>> reload(sys)
> <module 'sys' (built-in)>
> >>> sys.setdefaultencoding("base64")
> >>> "a"==u"a"
> Traceback (most recent call last):
> ...
> binascii.Error: Incorrect padding
>
>
> Maybe this is all just a matter of choosing the right defaultencoding ? :)



Doing this is amazingly stupid. I can't believe how often I hear this
suggestion. Apparently the fact that you have to "reload(sys)" to do it
isn't warning enough?

-- 
Christopher Armstrong
International Man of Twistery
http://radix.twistedmatrix.com/
http://twistedmatrix.com/
http://canonical.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060804/6f4e00fa/attachment.html 

From ralf at brainbot.com  Fri Aug  4 16:30:24 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Fri, 04 Aug 2006 16:30:24 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <60ed19d40608040701v59474febw94f7f374ad47d92a@mail.gmail.com>
References: <20060803212410.E6B5.JCARLSON@uci.edu>	<20060804044323.1717.140314887.divmod.quotient.13837@ohm>	<44D310E4.10304@brainbot.com>
	<60ed19d40608040701v59474febw94f7f374ad47d92a@mail.gmail.com>
Message-ID: <44D35A00.70107@brainbot.com>

Christopher Armstrong wrote:
> On 8/4/06, *Ralf Schmitt* <ralf at brainbot.com <mailto:ralf at brainbot.com>> 
> wrote:
> 
> 
>     Maybe this is all just a matter of choosing the right
>     defaultencoding ? :)
> 
> 
> 
> Doing this is amazingly stupid. I can't believe how often I hear this 
> suggestion. Apparently the fact that you have to "reload(sys)" to do it 
> isn't warning enough?

Ahh, the twisted people, friendly as I know them. Did you actually 
notice the smiley?

- Ralf



From mwh at python.net  Fri Aug  4 16:38:32 2006
From: mwh at python.net (Michael Hudson)
Date: Fri, 04 Aug 2006 15:38:32 +0100
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>
	(Michael Chermside's message of "Fri, 04 Aug 2006 05:26:52 -0700")
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>
Message-ID: <2m3bccwopj.fsf@starship.python.net>

Michael Chermside <mcherm at mcherm.com> writes:

> I'm changing the subject line because I want to convince everyone that
> the problem being discussed in the "unicode hell" thread has nothing
> to do with unicode and strings. It's all about dicts.

I'd say it's more to do with __eq__.  It's a strange __eq__ method
that raises an Exception, IMHO.

Please do realize that the motivation for this change was hours and
hours of tortous debugging caused by a buggy __eq__ method making keys
"impossibly" seem to not be in dictionaries.

> I have not observed real breakage in my own code, but I will present
> a sample of made-up-but-completely-reasonable code that works in
> 2.4, fails in 2.5, and arguably ought to work fine. I think we should
> restore the behavior of dicts that when they compare keys for
> equality they suppress exceptions (treating the objects as unequal),
> or at LEAST retain the behavior for one more release making it a
> warning this time.

Please no.  Here's just one piece of evidence that the 2.4 semantics
are pretty silly too:

>>> d = {u'\xe0':1, '\xe0\:2}
  File "<stdin>", line 1
    d = {u'\xe0':1, '\xe0\:2}
                            ^
SyntaxError: EOL while scanning single-quoted string
>>> d = {u'\xe0':1, '\xe0':2}
>>> '\xe0' in d
True
>>> '\xe0' in d.keys()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)

Cheers,
mwh

-- 
  same software, different verbosity settings (this one goes to
  eleven)                             -- the effbot on the martellibot

From tjreedy at udel.edu  Fri Aug  4 17:18:02 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 4 Aug 2006 11:18:02 -0400
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
	unicode	asdictionarykeys
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>
	<2m3bccwopj.fsf@starship.python.net>
Message-ID: <eavofa$n2f$1@sea.gmane.org>


"Michael Hudson" <mwh at python.net> wrote in message 
news:2m3bccwopj.fsf at starship.python.net...
> Michael Chermside <mcherm at mcherm.com> writes:
>
>> I'm changing the subject line because I want to convince everyone that
>> the problem being discussed in the "unicode hell" thread has nothing
>> to do with unicode and strings. It's all about dicts.
>
> I'd say it's more to do with __eq__.  It's a strange __eq__ method
> that raises an Exception, IMHO.

I agree; a == b should always work, certainly unless explicitly programmed 
otherwise in Python for a particular class.  So I think the proper solution 
is fix the buggy __eq__ method to return False instead.  If a byte string 
does not have a default (ascii) text interpretation, then it obviously is 
not equal to any particular unicode text.

The fundamental axiom of sets and hence of dict keys is that any 
object/value either is or is not a member (at any given time for 'mutable' 
set collections).  This requires that testing an object for possible 
membership by equality give a clean True or False answer.

> Please do realize that the motivation for this change was hours and
> hours of tortous debugging caused by a buggy __eq__ method making keys
> "impossibly" seem to not be in dictionaries.

So why not fix the buggy __eq__ method?

>> 2.4, fails in 2.5, and arguably ought to work fine. I think we should
>> restore the behavior of dicts that when they compare keys for
>> equality they suppress exceptions (treating the objects as unequal),
>> or at LEAST retain the behavior for one more release making it a
>> warning this time.
>
> Please no.  Here's just one piece of evidence that the 2.4 semantics
> are pretty silly too:

We mostly agreed half a decode ago that 1/2 should be .5 instead of 0, but 
to avoid breaking code, have (or Guido has) refrained from yet making the 
change the default.  To me, the same principle applies here at least as 
strongly.

Terry Jan Reedy




From mal at egenix.com  Fri Aug  4 17:43:50 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 04 Aug 2006 17:43:50 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <eavofa$n2f$1@sea.gmane.org>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>
	<eavofa$n2f$1@sea.gmane.org>
Message-ID: <44D36B36.8030202@egenix.com>

Terry Reedy wrote:
> "Michael Hudson" <mwh at python.net> wrote in message 
> news:2m3bccwopj.fsf at starship.python.net...
>> Michael Chermside <mcherm at mcherm.com> writes:
>>
>>> I'm changing the subject line because I want to convince everyone that
>>> the problem being discussed in the "unicode hell" thread has nothing
>>> to do with unicode and strings. It's all about dicts.
>> I'd say it's more to do with __eq__.  It's a strange __eq__ method
>> that raises an Exception, IMHO.
> 
> I agree; a == b should always work, certainly unless explicitly programmed 
> otherwise in Python for a particular class. 

... which this is.

> So I think the proper solution 
> is fix the buggy __eq__ method to return False instead.  If a byte string 
> does not have a default (ascii) text interpretation, then it obviously is 
> not equal to any particular unicode text.
> 
> The fundamental axiom of sets and hence of dict keys is that any 
> object/value either is or is not a member (at any given time for 'mutable' 
> set collections).  This requires that testing an object for possible 
> membership by equality give a clean True or False answer.
> 
>> Please do realize that the motivation for this change was hours and
>> hours of tortous debugging caused by a buggy __eq__ method making keys
>> "impossibly" seem to not be in dictionaries.
> 
> So why not fix the buggy __eq__ method?

Because it's not buggy.

Python just doesn't know the encoding of the 8-bit string, so can't
make any assumptions on it. As result, it raises an exception to inform
the programmer.

It is well possible that the string uses an encoding where the
Unicode string is indeed the equal to the string, assuming this
encoding, e.g.

>>> s = 'tr?r??'
>>> u = u'tr?r??'
>>> s == u
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 2:
ordinal not in range(128)
>>> hash(s)
673683206
>>> hash(u)
673683206

Here, the encoding that creates the match is Latin-1.

>>> 2.4, fails in 2.5, and arguably ought to work fine. I think we should
>>> restore the behavior of dicts that when they compare keys for
>>> equality they suppress exceptions (treating the objects as unequal),
>>> or at LEAST retain the behavior for one more release making it a
>>> warning this time.
>> Please no.  Here's just one piece of evidence that the 2.4 semantics
>> are pretty silly too:
> 
> We mostly agreed half a decode ago that 1/2 should be .5 instead of 0, but 
> to avoid breaking code, have (or Guido has) refrained from yet making the 
> change the default.  To me, the same principle applies here at least as 
> strongly.

I think that's a different category of semantic change: the integer
division change will cause applications to return wrong data (if not
fixed properly). The exception will just let the application refuse
to continue.

How about generating a warning instead and then go for the exception
in 2.6 ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 04 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From barry at python.org  Fri Aug  4 17:59:43 2006
From: barry at python.org (Barry Warsaw)
Date: Fri, 4 Aug 2006 11:59:43 -0400
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
	unicode	asdictionarykeys
In-Reply-To: <44D36B36.8030202@egenix.com>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>
	<eavofa$n2f$1@sea.gmane.org> <44D36B36.8030202@egenix.com>
Message-ID: <9AD92B30-3CF7-4357-9B64-D732F327C28F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Aug 4, 2006, at 11:43 AM, M.-A. Lemburg wrote:

> How about generating a warning instead and then go for the exception
> in 2.6 ?

 From the perspective of wanting to avoid blog entries in 2007  
railing against our gratuitous breakages in Python 2.5 but also  
wanting to avoid perpetuating broken code, this solution seems the  
most reasonable.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iQCVAwUBRNNu8HEjvBPtnXfVAQL9MQP/SCuLPFwS0m5vWJ3+i2iVZVCg21eXKQte
R8zoTngSx7unxfn5WQ7Bi8l9Ai1SkmZ7z0mOr6UbtRXmxM9+HwSFvpYpWazuaC4R
AylYA1ZbfsnzplHZW/TPxZKZJG++qWK2+mIcdHa9MS6OoBb583BQ4oXN8gs6tT2P
9VoUL5OW4Gw=
=IPeG
-----END PGP SIGNATURE-----

From mkent at emdeon.com  Fri Aug  4 16:55:45 2006
From: mkent at emdeon.com (Michael Kent)
Date: Fri, 4 Aug 2006 14:55:45 +0000 (UTC)
Subject: [Python-Dev] Python 2.5b3 and AIX 4.3 - It Works
Message-ID: <loom.20060804T164808-435@post.gmane.org>

Because of a requirement to remain compatible with AIX 4.3, I have been forced
to stay with Python 2.3, because while 2.4 would compile under AIX 4.3, it would
segfault immediately when run.

I'm happy to report that Python 2.5b3 compiles and runs fine under AIX 4.3, and
passes most of its test suite.  However, here are a few test failures.  I
realize AIX 4.3 is long obsolete, so is there any interest on the list for me to
report which tests failed and how?


From python-dev at zesty.ca  Fri Aug  4 19:32:32 2006
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Fri, 4 Aug 2006 12:32:32 -0500 (CDT)
Subject: [Python-Dev] uuid module - byte order issue
In-Reply-To: <7168d65a0608031311g3479824ds18a9ede4a33cae7e@mail.gmail.com>
References: <7168d65a0608031311g3479824ds18a9ede4a33cae7e@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0608041217290.32166@server1.LFW.org>

On Thu, 3 Aug 2006, Oren Tirosh wrote:
> The UUID module uses network byte order, regardless of the platform
> byte order. On little-endian platforms like Windows the ".bytes"
> property of UUID objects is not compatible with the memory layout

RFC 4122 says:

    In the absence of explicit application or presentation protocol
    specification to the contrary, a UUID is encoded as a 128-bit
    object, as follows:

    The fields are encoded as 16 octets, with the sizes and order of
    the fields defined above, and with each field encoded with the
    Most Significant Byte first (known as network byte order).

> Ka-Ping Yee writes* that the Windows UUID generation calls are not RFC
> 4122 compliant and have an illegal version field.
[...]
> If the correct byte order is used the UUIDs generated by Windows XP
> are valid version 4 UUIDs:

I see.  I stand corrected, then.  My interpretation of RFC 4122 would
be that the uuid.py module currently completely implements RFC 4122;
while Windows XP can't be said to violate RFC 4122, supporting this
alternative byte order would be an optional feature.

> The bytes property and bytes argument to the constructor should use
> the platform byte order.

I disagree; i think the main interface to uuid.py should behave
consistently independent of the platform, and should conform to the
default encoding in RFC 4122.  However, we could certainly consider
adding a 'bytes_le' field to support the functionality you describe.

I'll ask Anthony whether adding such a field would be permitted at
this stage of the release process.

> There is another issue with version 1 uuid generation:
> >>> len(set(uuid.uuid1() for i in range(1000)))
> 992
>
> The problem is that the random clock_seq field is only 14 bits long.
> If enough UUIDs are generated within the same system clock tick there
> will be collisions.

The clock field of the UUID has enough resolution to avoid collisions;
the problem you're describing is a limitation of the platform's clock,
not the UUID module.  It doesn't happen on Mac OS X, for example.

> Suggested solution: use the high-resolution of the
> time field (100ns) to generate a monotonically increasing timestamp
> that advances at least by 1 for each call, when time.time() returns
> the same value on subsequent calls.

That sounds like a fine solution to me.  I'll look into it.


-- ?!ng

From nnorwitz at gmail.com  Fri Aug  4 19:36:33 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 4 Aug 2006 10:36:33 -0700
Subject: [Python-Dev] Python 2.5b3 and AIX 4.3 - It Works
In-Reply-To: <loom.20060804T164808-435@post.gmane.org>
References: <loom.20060804T164808-435@post.gmane.org>
Message-ID: <ee2a432c0608041036k5b320463n5b2c41ad7a7aef10@mail.gmail.com>

There is at least one outstanding bug report for test_mmap failing on
AIX IIRC.  Possibly another for test_resource.  Please review bug
reports and file new ones/update old ones with the current status.
Unless if you provide patches, they probably won't be fixed though.
No one has access to AIX AFAIK.

n
--

On 8/4/06, Michael Kent <mkent at emdeon.com> wrote:
> Because of a requirement to remain compatible with AIX 4.3, I have been forced
> to stay with Python 2.3, because while 2.4 would compile under AIX 4.3, it would
> segfault immediately when run.
>
> I'm happy to report that Python 2.5b3 compiles and runs fine under AIX 4.3, and
> passes most of its test suite.  However, here are a few test failures.  I
> realize AIX 4.3 is long obsolete, so is there any interest on the list for me to
> report which tests failed and how?
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>

From mcherm at mcherm.com  Fri Aug  4 19:49:53 2006
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri, 04 Aug 2006 10:49:53 -0700
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str
	and	unicode	asdictionarykeys
Message-ID: <20060804104953.fhak73l8ffcw8w4o@login.werra.lunarpages.com>

Marc-Andre Lemburg writes:

> How about generating a warning instead and then go for the exception
> in 2.6 ?

Agreed. Michael Hudson's explanation convinced me.

-- Michael Chermside


From jcarlson at uci.edu  Fri Aug  4 20:23:10 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri, 04 Aug 2006 11:23:10 -0700
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
	unicode	asdictionarykeys
In-Reply-To: <9AD92B30-3CF7-4357-9B64-D732F327C28F@python.org>
References: <44D36B36.8030202@egenix.com>
	<9AD92B30-3CF7-4357-9B64-D732F327C28F@python.org>
Message-ID: <20060804112104.E6CD.JCARLSON@uci.edu>


There's one problem with generating a warning for 2.5, and that is the
same problem as generating a warning for possible packages that lack an
__init__.py; users may start to get a bunch of warnings, and be unaware
of how to suppress them.

All in all though, I'm +0 on the warning, and +1 on it not raising an
exception in 2.5 .

 - Josiah

Barry Warsaw <barry at python.org> wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Aug 4, 2006, at 11:43 AM, M.-A. Lemburg wrote:
> 
> > How about generating a warning instead and then go for the exception
> > in 2.6 ?
> 
>  From the perspective of wanting to avoid blog entries in 2007  
> railing against our gratuitous breakages in Python 2.5 but also  
> wanting to avoid perpetuating broken code, this solution seems the  
> most reasonable.
> 
> - -Barry
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.3 (Darwin)
> 
> iQCVAwUBRNNu8HEjvBPtnXfVAQL9MQP/SCuLPFwS0m5vWJ3+i2iVZVCg21eXKQte
> R8zoTngSx7unxfn5WQ7Bi8l9Ai1SkmZ7z0mOr6UbtRXmxM9+HwSFvpYpWazuaC4R
> AylYA1ZbfsnzplHZW/TPxZKZJG++qWK2+mIcdHa9MS6OoBb583BQ4oXN8gs6tT2P
> 9VoUL5OW4Gw=
> =IPeG
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jcarlson%40uci.edu


From exarkun at divmod.com  Fri Aug  4 20:24:48 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 4 Aug 2006 14:24:48 -0400
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <20060804112104.E6CD.JCARLSON@uci.edu>
Message-ID: <20060804182448.1717.1264031474.divmod.quotient.14372@ohm>

On Fri, 04 Aug 2006 11:23:10 -0700, Josiah Carlson <jcarlson at uci.edu> wrote:
>
>There's one problem with generating a warning for 2.5, and that is the
>same problem as generating a warning for possible packages that lack an
>__init__.py; users may start to get a bunch of warnings, and be unaware
>of how to suppress them.
>
>All in all though, I'm +0 on the warning, and +1 on it not raising an
>exception in 2.5 .
>

Um.  This warning would indicate a bug in the code which will lead to
actual misbehavior in a future version of Python.  The __init__.py
warning would have indicated a deployment configuration which didn't
actually cause any misbehavior.

They aren't the same case at all, unless you think that all warnings
should be classed this way (a position I do not think is completely
unreasonable, but since you singled out the package warning by way of
comparison, I assume this is not the argument you are trying to make).

Jean-Paul

From pc at gafol.net  Fri Aug  4 20:31:38 2006
From: pc at gafol.net (Paul Colomiets)
Date: Fri, 04 Aug 2006 21:31:38 +0300
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <eavofa$n2f$1@sea.gmane.org>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>
	<eavofa$n2f$1@sea.gmane.org>
Message-ID: <44D3928A.1090700@gafol.net>

Hi!

Terry Reedy wrote:
> The fundamental axiom of sets and hence of dict keys is that any 
> object/value either is or is not a member (at any given time for 'mutable' 
> set collections).  This requires that testing an object for possible 
> membership by equality give a clean True or False answer.
>   
Yes this makes sense. But returning to dictionaries for python newbies, 
it will be strange why this
 >>> d = { u'abc': 1, u'ab\xe8': 2}
 >>> d['abc']
 >1
works as expected, but this
 >>> d['ab\xe8']
raises an exception.

Another good argument pronounced by M.-A. Lemburg:
> What's making this particular case interesting is that
> the comparison is hidden in the dictionary implementation
> and only triggers if you get a hash collision, which makes
> the whole issue appear to be happening randomly.
>
> This whole thread aside: it's never recommended to mix strings
> and Unicode, unless you really have to.
...
 >How about generating a warning instead and then go for the exception
 >in 2.6 ?

Well it's not recomended to mix strings and unicode in the dictionaries 
but if we mix for example integer and float we have the same thing. It 
doesn't raise exception but still it is not expected behavior for me:
 >>> d = { 1.0: 10, 2.0: 20 }
then if i somewhere later do:
 >>> d[1] = 100
 >>> d[2] = 200
to have here all floats in d.keys(). May be this is not a best example. 
So if you generate a warning, it should be generated every time when 
there are keys of different types inserted into dict. May be python 
should check type of the key after collision and before testing for 
equality? So the 1 and 1.0 is different as u'a' and 'a' also different. 
It even can give some perfomance overhead I think.

--
Regards,
  Paul.

From nmm1 at cus.cam.ac.uk  Fri Aug  4 20:59:34 2006
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Fri, 04 Aug 2006 19:59:34 +0100
Subject: [Python-Dev] Dicts are broken ...
Message-ID: <E1G94tS-0002yz-GT@draco.cus.cam.ac.uk>

Michael Hudson <mwh at python.net> wrote:
> 
> I'd say it's more to do with __eq__.  It's a strange __eq__ method
> that raises an Exception, IMHO.

Not entirely.  Any type that supports invalid values (e.g. IEEE 754)
and is safe against losing the invalid state by accident needs to
raise an exception on A == B.  IEEE 754 is not safe.

> Please do realize that the motivation for this change was hours and
> hours of tortous debugging caused by a buggy __eq__ method making keys
> "impossibly" seem to not be in dictionaries.

Quite.  Been there - been caught by that.  It is a catastrophic (but
very common) misdesign to conflate failure and the answer "no".
There is a fundamental flaw of that nature in card-based banking,
that I pointed out was insoluble to the Grid people, and then got
caught by just a month later!


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From rasky at develer.com  Fri Aug  4 21:51:09 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Fri, 4 Aug 2006 21:51:09 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
	unicode	asdictionarykeys
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net><eavofa$n2f$1@sea.gmane.org>
	<44D3928A.1090700@gafol.net>
Message-ID: <176301c6b7ff$554d9890$8cb22997@bagio>

Paul Colomiets <pc at gafol.net> wrote:

> Well it's not recomended to mix strings and unicode in the
> dictionaries
> but if we mix for example integer and float we have the same thing. It
> doesn't raise exception but still it is not expected behavior for me:
>  >>> d = { 1.0: 10, 2.0: 20 }
> then if i somewhere later do:
>  >>> d[1] = 100
>  >>> d[2] = 200
> to have here all floats in d.keys(). May be this is not a best
> example.

There is a strong difference. Python is moving towards unifying number types in
a way (see the true division issue): the idea is that, all in all, user
shouldn't really care what type a number is, as long as he knows it's a number.
On the other hand, unicode and str are going to diverge more and more.

Giovanni Bajo


From bob at redivi.com  Fri Aug  4 22:23:20 2006
From: bob at redivi.com (Bob Ippolito)
Date: Fri, 4 Aug 2006 13:23:20 -0700
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
	unicode	asdictionarykeys
In-Reply-To: <176301c6b7ff$554d9890$8cb22997@bagio>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net><eavofa$n2f$1@sea.gmane.org>
	<44D3928A.1090700@gafol.net> <176301c6b7ff$554d9890$8cb22997@bagio>
Message-ID: <FBFEADC4-CE3A-4470-86E1-FAE5A9A5ABD0@redivi.com>


On Aug 4, 2006, at 12:51 PM, Giovanni Bajo wrote:

> Paul Colomiets <pc at gafol.net> wrote:
>
>> Well it's not recomended to mix strings and unicode in the
>> dictionaries
>> but if we mix for example integer and float we have the same  
>> thing. It
>> doesn't raise exception but still it is not expected behavior for me:
>>>>> d = { 1.0: 10, 2.0: 20 }
>> then if i somewhere later do:
>>>>> d[1] = 100
>>>>> d[2] = 200
>> to have here all floats in d.keys(). May be this is not a best
>> example.
>
> There is a strong difference. Python is moving towards unifying  
> number types in
> a way (see the true division issue): the idea is that, all in all,  
> user
> shouldn't really care what type a number is, as long as he knows  
> it's a number.
> On the other hand, unicode and str are going to diverge more and more.

Well, not really. True division makes int/int return float instead of  
an int. You really do have to care if you have an int or a float most  
of the time, they're very different semantically.

Unicode and str are eventually going to be the same thing (str would  
ideally end up becoming a synonym of unicode). The difference being  
that there will be some other type to contain bytes.

-bob


From pc at gafol.net  Fri Aug  4 22:32:42 2006
From: pc at gafol.net (Paul Colomiets)
Date: Fri, 04 Aug 2006 23:32:42 +0300
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
	unicode	asdictionarykeys
In-Reply-To: <176301c6b7ff$554d9890$8cb22997@bagio>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net><eavofa$n2f$1@sea.gmane.org>
	<44D3928A.1090700@gafol.net> <176301c6b7ff$554d9890$8cb22997@bagio>
Message-ID: <44D3AEEA.3080201@gafol.net>

Giovanni Bajo wrote:
> Paul Colomiets <pc at gafol.net> wrote:
>
>   
>> Well it's not recomended to mix strings and unicode in the
>> dictionaries
>> but if we mix for example integer and float we have the same thing. It
>> doesn't raise exception but still it is not expected behavior for me:
>>  >>> d = { 1.0: 10, 2.0: 20 }
>> then if i somewhere later do:
>>  >>> d[1] = 100
>>  >>> d[2] = 200
>> to have here all floats in d.keys(). May be this is not a best
>> example.
>>     
>
> There is a strong difference. Python is moving towards unifying number types in
> a way (see the true division issue): the idea is that, all in all, user
> shouldn't really care what type a number is, as long as he knows it's a number.
> On the other hand, unicode and str are going to diverge more and more.
>
> Giovanni Bajo
>
>   
It makes sense, but consider this example:

 >>> from decimal import Decimal
 >>> d = {}
 >>> d[Decimal(0)] = 1
 >>> d[0] = 2
 >>> d[Decimal("0.5")] = 3
 >>> d[0.5]  = 4
 >>> d.keys()
[Decimal("0"), 0.5, Decimal("0.5")]

I expect d.keys() to have 2 or 4 keys but don't 3, it's confusing. Don't 
you think that someday line "d[0.5] = 4" will raise exception? Or at 
least that it should raise if mixing str and unicode raises?

--
Regards,
  Paul.

From david.ascher at gmail.com  Fri Aug  4 23:20:14 2006
From: david.ascher at gmail.com (David Ascher)
Date: Fri, 4 Aug 2006 14:20:14 -0700
Subject: [Python-Dev] More tracker demos online
In-Reply-To: <200607261426.29143.richardjones@optushome.com.au>
References: <44C6748C.4060701@v.loewis.de> <ea6msv$hgf$1@sea.gmane.org>
	<200607261426.29143.richardjones@optushome.com.au>
Message-ID: <dd28fc2f0608041420j27320b48k5292d66af5fe74bc@mail.gmail.com>

> > ""Martin v. L?wis"" <martin at v.loewis.de> wrote in message
> > news:44C6748C.4060701 at v.loewis.de...
> >
> > > Currently, we have two running tracker demos online:
> > >
> > > Roundup:
> > > http://efod.se/python-tracker/
> > >
> > > Jira:
> > > http://jira.python.atlassian.com/secure/Dashboard.jspa


Is anyone looking at the Google Code Hosting tracker, just for fun? =)  (
code.google.com/hosting, although performance seems to be an issue for now)

--david
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060804/2c15f38c/attachment.html 

From rrr at ronadam.com  Sat Aug  5 00:26:02 2006
From: rrr at ronadam.com (Ron Adam)
Date: Fri, 04 Aug 2006 17:26:02 -0500
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <44D36B36.8030202@egenix.com>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>	<eavofa$n2f$1@sea.gmane.org>
	<44D36B36.8030202@egenix.com>
Message-ID: <44D3C97A.8000209@ronadam.com>

M.-A. Lemburg wrote:
> Terry Reedy wrote:
>> "Michael Hudson" <mwh at python.net> wrote in message 
>> news:2m3bccwopj.fsf at starship.python.net...
>>> Michael Chermside <mcherm at mcherm.com> writes:
>>>
>>>> I'm changing the subject line because I want to convince everyone that
>>>> the problem being discussed in the "unicode hell" thread has nothing
>>>> to do with unicode and strings. It's all about dicts.
>>> I'd say it's more to do with __eq__.  It's a strange __eq__ method
>>> that raises an Exception, IMHO.
>> I agree; a == b should always work, certainly unless explicitly programmed 
>> otherwise in Python for a particular class. 
> 
> ... which this is.
> 
>> So I think the proper solution 
>> is fix the buggy __eq__ method to return False instead.  If a byte string 
>> does not have a default (ascii) text interpretation, then it obviously is 
>> not equal to any particular unicode text.
>>
>> The fundamental axiom of sets and hence of dict keys is that any 
>> object/value either is or is not a member (at any given time for 'mutable' 
>> set collections).  This requires that testing an object for possible 
>> membership by equality give a clean True or False answer.
>>
>>> Please do realize that the motivation for this change was hours and
>>> hours of tortous debugging caused by a buggy __eq__ method making keys
>>> "impossibly" seem to not be in dictionaries.
>> So why not fix the buggy __eq__ method?
> 
> Because it's not buggy.
> 
> Python just doesn't know the encoding of the 8-bit string, so can't
> make any assumptions on it. As result, it raises an exception to inform
> the programmer.
> 
> It is well possible that the string uses an encoding where the
> Unicode string is indeed the equal to the string, assuming this
> encoding, e.g.

Isn't this a case where it should be up to the programmer to make sure 
the comparison makes sense in the context it is being used.  That is, if 
I'm comparing two different forms of data, it's up to me to convert them 
explicitly to the same form before comparing them?

In the case of comparing an 8 bit string and unicode, I would think they 
are always unequal.  But changing that now would probably (?) break way 
too much. (but it may also uncover quite a few potential or even real 
bugs as well.) ;-)

Cheers,
    Ron










From jcarlson at uci.edu  Sat Aug  5 00:46:21 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri, 04 Aug 2006 15:46:21 -0700
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
	unicode	asdictionarykeys
In-Reply-To: <20060804182448.1717.1264031474.divmod.quotient.14372@ohm>
References: <20060804112104.E6CD.JCARLSON@uci.edu>
	<20060804182448.1717.1264031474.divmod.quotient.14372@ohm>
Message-ID: <20060804153937.E6E3.JCARLSON@uci.edu>


Jean-Paul Calderone <exarkun at divmod.com> wrote:
> On Fri, 04 Aug 2006 11:23:10 -0700, Josiah Carlson <jcarlson at uci.edu> wrote:
> >There's one problem with generating a warning for 2.5, and that is the
> >same problem as generating a warning for possible packages that lack an
> >__init__.py; users may start to get a bunch of warnings, and be unaware
> >of how to suppress them.
> >
> >All in all though, I'm +0 on the warning, and +1 on it not raising an
> >exception in 2.5 .
> 
> Um.  This warning would indicate a bug in the code which will lead to
> actual misbehavior in a future version of Python.  The __init__.py
> warning would have indicated a deployment configuration which didn't
> actually cause any misbehavior.
> 
> They aren't the same case at all, unless you think that all warnings
> should be classed this way (a position I do not think is completely
> unreasonable, but since you singled out the package warning by way of
> comparison, I assume this is not the argument you are trying to make).

I see both as being a potential cause for a large number of warning
messages to people starting to use Python 2.5 (from 2.3 or 2.4) .

 - Josiah


From brett at python.org  Sat Aug  5 02:11:29 2006
From: brett at python.org (Brett Cannon)
Date: Fri, 4 Aug 2006 17:11:29 -0700
Subject: [Python-Dev] need an SSH key removed
Message-ID: <bbaeab100608041711j3e0e5bfdr3d54fccd9da755a4@mail.gmail.com>

My bcannon at google.com SSH key should be removed since my internship is now
over.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060804/5d1db915/attachment.html 

From tim.peters at gmail.com  Sat Aug  5 05:07:14 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 4 Aug 2006 23:07:14 -0400
Subject: [Python-Dev] need an SSH key removed
In-Reply-To: <bbaeab100608041711j3e0e5bfdr3d54fccd9da755a4@mail.gmail.com>
References: <bbaeab100608041711j3e0e5bfdr3d54fccd9da755a4@mail.gmail.com>
Message-ID: <1f7befae0608042007t3f47c7ecrd96eb76e028cc727@mail.gmail.com>

[Brett Cannon]
> My bcannon at google.com SSH key should be removed since my internship is now
> over.

Thank you for being conscientious.  While it feared death, your key
didn't complain about being deleted, and right before it vanished I
saw the most astonishing look of profound peace passing over its bits.
 You should rest assured tonight that yYv6r9zcrb and ZORyPRLc5Sf and
all the other little key segments have gone on to a better repository.

From aahz at pythoncraft.com  Sat Aug  5 06:32:44 2006
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 4 Aug 2006 21:32:44 -0700
Subject: [Python-Dev] need an SSH key removed
In-Reply-To: <1f7befae0608042007t3f47c7ecrd96eb76e028cc727@mail.gmail.com>
References: <bbaeab100608041711j3e0e5bfdr3d54fccd9da755a4@mail.gmail.com>
	<1f7befae0608042007t3f47c7ecrd96eb76e028cc727@mail.gmail.com>
Message-ID: <20060805043244.GA23050@panix.com>

On Fri, Aug 04, 2006, Tim Peters wrote:
> [Brett Cannon]
>>
>> My bcannon at google.com SSH key should be removed since my internship is now
>> over.
> 
> Thank you for being conscientious.  While it feared death, your key
> didn't complain about being deleted, and right before it vanished I
> saw the most astonishing look of profound peace passing over its bits.
> You should rest assured tonight that yYv6r9zcrb and ZORyPRLc5Sf and
> all the other little key segments have gone on to a better repository.

That means it's now an ex-key?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

C'mon, you knew *someone* was gonna say that

From rasky at develer.com  Sat Aug  5 13:17:18 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 5 Aug 2006 13:17:18 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str
	andunicode	asdictionarykeys
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net><eavofa$n2f$1@sea.gmane.org><44D3928A.1090700@gafol.net>
	<176301c6b7ff$554d9890$8cb22997@bagio>
	<FBFEADC4-CE3A-4470-86E1-FAE5A9A5ABD0@redivi.com>
Message-ID: <1c0801c6b880$b6f804f0$8cb22997@bagio>

Bob Ippolito <bob at redivi.com> wrote:

>>> Well it's not recomended to mix strings and unicode in the
>>> dictionaries
>>> but if we mix for example integer and float we have the same
>>> thing. It
>>> doesn't raise exception but still it is not expected behavior for
>>> me:
>>>>>> d = { 1.0: 10, 2.0: 20 }
>>> then if i somewhere later do:
>>>>>> d[1] = 100
>>>>>> d[2] = 200
>>> to have here all floats in d.keys(). May be this is not a best
>>> example.
>>
>> There is a strong difference. Python is moving towards unifying
>> number types in
>> a way (see the true division issue): the idea is that, all in all,
>> user
>> shouldn't really care what type a number is, as long as he knows
>> it's a number.
>> On the other hand, unicode and str are going to diverge more and
>> more.
>
> Well, not really. True division makes int/int return float instead of
> an int. You really do have to care if you have an int or a float most
> of the time, they're very different semantically.

Then I'd ask why Python goes through hoops to make sure that hash(1.0) ==
hash(1), in the first place.

Giovanni Bajo


From hfoffani at gmail.com  Sat Aug  5 13:52:19 2006
From: hfoffani at gmail.com (Hernan M Foffani)
Date: Sat, 5 Aug 2006 13:52:19 +0200
Subject: [Python-Dev] More tracker demos online
Message-ID: <11fab4bc0608050452l458a2073n250134484cf303e0@mail.gmail.com>

> > > > Currently, we have two running tracker demos online:
> > > >
> > > > Roundup:
> > > > http://efod.se/python-tracker/
> > > >
> > > > Jira:
> > > > http://jira.python.atlassian.com/secure/Dashboard.jspa
>
>
> Is anyone looking at the Google Code Hosting tracker, just for fun? =)  (
> code.google.com/hosting, although performance seems to be an issue for now)

It's propietary code, isn't it?
http://code.google.com/hosting/faq.html#itselfoss
(Is that why you said "just for fun"?)

Regards,
-H.

From bob at redivi.com  Sun Aug  6 00:11:52 2006
From: bob at redivi.com (Bob Ippolito)
Date: Sat, 5 Aug 2006 15:11:52 -0700
Subject: [Python-Dev] More tracker demos online
In-Reply-To: <11fab4bc0608050452l458a2073n250134484cf303e0@mail.gmail.com>
References: <11fab4bc0608050452l458a2073n250134484cf303e0@mail.gmail.com>
Message-ID: <85FDF5FA-853E-4000-8E95-0F46B56E6910@redivi.com>


On Aug 5, 2006, at 4:52 AM, Hernan M Foffani wrote:

>>>>> Currently, we have two running tracker demos online:
>>>>>
>>>>> Roundup:
>>>>> http://efod.se/python-tracker/
>>>>>
>>>>> Jira:
>>>>> http://jira.python.atlassian.com/secure/Dashboard.jspa
>>
>>
>> Is anyone looking at the Google Code Hosting tracker, just for  
>> fun? =)  (
>> code.google.com/hosting, although performance seems to be an issue  
>> for now)
>
> It's propietary code, isn't it?
> http://code.google.com/hosting/faq.html#itselfoss
> (Is that why you said "just for fun"?)

So is Jira...

-bob


From l.oluyede at gmail.com  Sun Aug  6 10:41:39 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Sun, 6 Aug 2006 10:41:39 +0200
Subject: [Python-Dev] BZ2File.writelines should raise more meaningful
	exceptions
Message-ID: <9eebf5740608060141k336e036n978e7a4f9f416ce4@mail.gmail.com>

In the BZ2File object of bz2 module the writelines() method does not
check its closed state before doing the actual work so its behavior
it's different from write()'s behavior. See:

>>> from bz2 import BZ2File
>>> f = BZ2File("foo", "w")
>>> f.close()
>>> f.closed
1
>>> f.write("foobar")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: I/O operation on closed file
>>> f.closed
1
>>> f.writelines(["foobar"])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: unknown IO error

It also doesn't check if it can write for real:

>>> from bz2 import BZ2File
>>> f = BZ2File("foo", "r")
>>> f.write("foobar")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: file is not ready for writing
>>> f.writelines(['foobar'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: wrong sequence of bz2 library commands used

The patch is attached. If you think it's ok to fix this I'll post it
to the bug tracker

-- 
Lawrence
http://www.oluyede.org/blog
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: bz2module.diff.txt
Url: http://mail.python.org/pipermail/python-dev/attachments/20060806/bd63dc42/attachment.txt 

From andymac at bullseye.apana.org.au  Sun Aug  6 14:09:09 2006
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Sun, 06 Aug 2006 23:09:09 +1100
Subject: [Python-Dev] test_mailbox on Cygwin
Message-ID: <44D5DBE5.4030304@bullseye.apana.org.au>

I just had a look at the Cygwin buildbot's test log, and noticed that
test_mailbox has failures that look very similar to those I addressed in
rev 50789 for the OS/2 EMX platform.

Hopefully this might give someone with access to Cygwin a starting point
in getting this test working on that platform.

-- 
-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac at pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia

From aahz at pythoncraft.com  Sun Aug  6 16:20:24 2006
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 6 Aug 2006 07:20:24 -0700
Subject: [Python-Dev] BZ2File.writelines should raise more meaningful
	exceptions
In-Reply-To: <9eebf5740608060141k336e036n978e7a4f9f416ce4@mail.gmail.com>
References: <9eebf5740608060141k336e036n978e7a4f9f416ce4@mail.gmail.com>
Message-ID: <20060806142024.GA25227@panix.com>

On Sun, Aug 06, 2006, Lawrence Oluyede wrote:
>
> The patch is attached. If you think it's ok to fix this I'll post it
> to the bug tracker

Always post patches -- that way they can't get lost.  *THEN* post to
python-dev with your analysis and explanation (which you presumably also
included with the patch), starting with a link to the patch.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it."  --Brian W. Kernighan

From l.oluyede at gmail.com  Sun Aug  6 21:33:12 2006
From: l.oluyede at gmail.com (Lawrence Oluyede)
Date: Sun, 6 Aug 2006 21:33:12 +0200
Subject: [Python-Dev] BZ2File.writelines should raise more meaningful
	exceptions
In-Reply-To: <20060806142024.GA25227@panix.com>
References: <9eebf5740608060141k336e036n978e7a4f9f416ce4@mail.gmail.com>
	<20060806142024.GA25227@panix.com>
Message-ID: <9eebf5740608061233g79926169m41a94d011eb29db7@mail.gmail.com>

> Always post patches -- that way they can't get lost.  *THEN* post to
> python-dev with your analysis and explanation (which you presumably also
> included with the patch), starting with a link to the patch.

Thanks for the hint. This is the link:
http://sourceforge.net/tracker/index.php?func=detail&aid=1535500&group_id=5470&atid=305470

-- 
Lawrence
http://www.oluyede.org/blog

From greg at electricrain.com  Sun Aug  6 21:41:35 2006
From: greg at electricrain.com (Gregory P. Smith)
Date: Sun, 6 Aug 2006 12:41:35 -0700
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
	1535502]
Message-ID: <20060806194135.GC9899@zot.electricrain.com>

Whoever knows how the windows build process works and controls the
python 2.5 windows release builds could you please make sure the
hashlib module gets built + linked with OpenSSL rather than falling
back to its much slower builtin implementations.

 http://sourceforge.net/tracker/index.php?func=detail&aid=1535502&group_id=5470&atid=105470

thanks,
greg


From brett at python.org  Sun Aug  6 23:39:17 2006
From: brett at python.org (Brett Cannon)
Date: Sun, 6 Aug 2006 14:39:17 -0700
Subject: [Python-Dev] More tracker demos online
In-Reply-To: <11fab4bc0608050452l458a2073n250134484cf303e0@mail.gmail.com>
References: <11fab4bc0608050452l458a2073n250134484cf303e0@mail.gmail.com>
Message-ID: <bbaeab100608061439g21770547v914b1b9f0a2685c@mail.gmail.com>

On 8/5/06, Hernan M Foffani <hfoffani at gmail.com> wrote:
>
> > > > > Currently, we have two running tracker demos online:
> > > > >
> > > > > Roundup:
> > > > > http://efod.se/python-tracker/
> > > > >
> > > > > Jira:
> > > > > http://jira.python.atlassian.com/secure/Dashboard.jspa
> >
> >
> > Is anyone looking at the Google Code Hosting tracker, just for fun?
> =)  (
> > code.google.com/hosting, although performance seems to be an issue for
> now)
>
> It's propietary code, isn't it?
> http://code.google.com/hosting/faq.html#itselfoss
> (Is that why you said "just for fun"?)


Yes, it's proprietary, but so what?  As Bob pointed out, JIRA is as well.
Read the call for trackers and you will see we explicitly said closed-source
was fine.

But Google Code Hosting is not being considered since it won't have data
imported by the deadline for accepting test trackers (it currently doesn't
have any support for mass issue uploading).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060806/69973255/attachment.htm 

From tim.peters at gmail.com  Mon Aug  7 06:00:37 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 7 Aug 2006 00:00:37 -0400
Subject: [Python-Dev] cgi.FieldStorage DOS (sf bug #1112549)
In-Reply-To: <D38E3187-946F-4649-9E6D-F366E1F7E52C@plope.com>
References: <D38E3187-946F-4649-9E6D-F366E1F7E52C@plope.com>
Message-ID: <1f7befae0608062100y601e18dtaad2490e50228acc@mail.gmail.com>

[Chris McDonough, on 7/28/06]
>  From the initial bugreport
> (http://sourceforge.net/tracker/index.php?
> func=detail&aid=1112549&group_id=5470&atid=105470)
>
> """
> Various parts of cgi.FieldStorage call its
> "read_lines_to_outerboundary", "read_lines" and
> "skip_lines" methods. These methods use the
> "readline" method of the file object that represents an
> input stream. The input stream is typically data
> supplied by an untrusted source (such as a user
> uploading a file from a web browser). The input data
> is not required by the RFC 822/1521/1522/1867
> specifications to contain any newline characters. For
> example, it is within the bounds of the specification
> to supply a a multipart/form-data input stream with a
> "file-data" part that consists of a 2GB string composed
> entirely of "x" characters (which happens to be
> something I did that led me to noticing this bug).
> """
>
> This bug has been around for about a year but I just worked up a
> patch yesterday that applies OK against current SVN.  It's attached
> to the issue.  Would someone be so kind as to check it in?  Guido has
> already reviewed it, I believe.

Are either of our 2.5 release managers/coordinators on the internal
Python security mailing list?  I am, but only the list admin can see
who's on that list.

Since this bug is thought to be a security hole (the "DOS" in the
subject line doesn't refer to your favorite operating system -- it's
the Denial-Of-Service flavor of DOS), it's important that someone with
sufficient power stare at this one and Pronounce on its fate for
2.5c1.  Here's a clicky thing:

    http://www.python.org/sf/1112549

From robinbryce at gmail.com  Mon Aug  7 13:01:31 2006
From: robinbryce at gmail.com (Robin Bryce)
Date: Mon, 7 Aug 2006 12:01:31 +0100
Subject: [Python-Dev] 2.5b3,
	commit r46372 regressed PEP 302 machinery (sf not letting me post)
Message-ID: <bcf87d920608070401p78367590g9b2be0ca3650f018@mail.gmail.com>

Hi,

Appologies for the lack of an sf#. I tried to submit this there but
couldn't. (sf is logging me out each time I visit a new page and it is
refusing my attempt to post anonymously).

Python 2.5b3 (trunk:51136M, Aug  7 2006, 10:48:15)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2


The need for speed patch commited in revision r46372 included a change
whose intent was to reduce the number of open calls. The `continue`
statement at line 1283 in import.c:r51136 has the effect of skipping
the builtin import machinery if the find_module method of a custom
importer returns None.

In Python 2.4.3, if find_module returned None the builtin machinery is
allowed to process the path tail.

In my particular case I am working on an importer that deals with kid
templates that may or may not exist as .py[c] files.

The short of it is that in Python 2.4.3 this produces a usable module
``__import__('foo.a/templateuri')`` wheras in 2.5b3 I get import
error. The python 2.4.3 implementation *allows* module paths that are
not seperated with '.' Python 2.5b3 does not allow this and it does
not look like this was an intentional change. I believe this point
about 'illeagal' module paths is actualy independent of the regresion
I am asserting. Detailed session logs are attatched (following the sf
guidance even though I'm posting to py-dev)

The 'use case' for the importer is: Robin wants to package a default
template file as normal python module and provide a custom importer
that allows users of his package to reference both: there own
templates and html files on the file system in arbitrary locations AND
the stock templates provided as python modules under the same name
space. He would like to leave normal imports to the standard
machinery.

Cheers,

Robin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bugreport.rst
Type: application/octet-stream
Size: 2594 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060807/c1a25e6d/attachment.obj 

From martin at v.loewis.de  Mon Aug  7 14:46:45 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 07 Aug 2006 14:46:45 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D31E64.3070407@egenix.com>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>	<44D2F653.5090305@egenix.com>	<44D3140A.2040600@canterbury.ac.nz>
	<44D31E64.3070407@egenix.com>
Message-ID: <44D73635.4000003@v.loewis.de>

M.-A. Lemburg schrieb:
>> There's no disputing that an exception should be raised
>> if the string *must* be interpretable as characters in
>> order to continue. But that's not true here if you allow
>> for the interpretation that they're simply objects of
>> different (duck) type and therefore unequal.
> 
> Hmm, given that interpretation, 1 == 1.0 would have to be
> False.

No, but 1 == 1.5 would have to be False (and actually is).
In that analogy, int relates to float as ascii-bytes to
Unicode: some values are shared between int and float (e.g.
1 and 1.0), other values are not shared (e.g. 1.5 has no
equivalent in int). An int equals a float only  if both
values originate from the shared subset.

Now, int is a (nearly) true subset of float, so there are
no ints with no float equivalent (actually, there are, but
Python ignores that).

> Note that you do have to interpret the string as characters
> if you compare it to Unicode and there's nothing wrong with
> that.

Consider this:
py> int(3+4j)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: can't convert complex to int; use int(abs(z))
py> 3 == 3+4j
False

So even though the conversion raises an exception, the
values are determined to be not equal. Again, because int
is a nearly true subset of complex, the conversion goes
the other way, but *if* it would use the complex->int
conversion, then the TypeError should be taken as
a guarantee that the objects don't compare equal.

Expanding this view to Unicode should mean that a unicode
string U equals a byte string B if
U.encode(system_encode) == B or B.decode(system_encoding) == U,
and that they don't equal otherwise (e.g. if the conversion
fails with a "not convertible" exception). Which of the
two conversions is selected is arbitrary; we should, of
course, continue to use the one we always used (for
"ascii", there is no difference between the two).

Regards,
Martin

From martin at v.loewis.de  Mon Aug  7 15:00:20 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 07 Aug 2006 15:00:20 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <44D36B36.8030202@egenix.com>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>	<eavofa$n2f$1@sea.gmane.org>
	<44D36B36.8030202@egenix.com>
Message-ID: <44D73964.3050402@v.loewis.de>

M.-A. Lemburg schrieb:
> Python just doesn't know the encoding of the 8-bit string, so can't
> make any assumptions on it. As result, it raises an exception to inform
> the programmer.

Oh, Python does make an assumption what the encoding is: it assumes
it is the system encoding (i.e. "ascii"). Then invoking the ascii
codec raises an exception, because the string clearly isn't ascii.

> It is well possible that the string uses an encoding where the
> Unicode string is indeed the equal to the string, assuming this
> encoding

So what? Python uses the system encoding for this operation.
What does it matter that the result would be different if it
had used a different encoding.

The strings are unequal under the system encoding; it's irrelevant
that they might be equal under a different encoding.

The same holds for the ASCII part (i.e. where you don't get an
exception):

py> u"foo" == "sbb"
False
py> u"foo".encode("rot13") == "sbb"
True

So the strings compare as unequal, even though they compare
equal if treated as rot13. That doesn't stop Python from considering
them unequal.

Regards,
Martin

From martin at v.loewis.de  Mon Aug  7 15:04:24 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 07 Aug 2006 15:04:24 +0200
Subject: [Python-Dev] Python 2.5b3 and AIX 4.3 - It Works
In-Reply-To: <loom.20060804T164808-435@post.gmane.org>
References: <loom.20060804T164808-435@post.gmane.org>
Message-ID: <44D73A58.2040502@v.loewis.de>

Michael Kent schrieb:
> Because of a requirement to remain compatible with AIX 4.3, I have been forced
> to stay with Python 2.3, because while 2.4 would compile under AIX 4.3, it would
> segfault immediately when run.
> 
> I'm happy to report that Python 2.5b3 compiles and runs fine under AIX 4.3, and
> passes most of its test suite.  However, here are a few test failures.  I
> realize AIX 4.3 is long obsolete, so is there any interest on the list for me to
> report which tests failed and how?

As Neal says: There would be interest in receiving patches. Interest in
receiving bug reports for systems we don't have access to is low.

Regards,
Martin

From martin at v.loewis.de  Mon Aug  7 15:16:22 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 07 Aug 2006 15:16:22 +0200
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
 1535502]
In-Reply-To: <20060806194135.GC9899@zot.electricrain.com>
References: <20060806194135.GC9899@zot.electricrain.com>
Message-ID: <44D73D26.4050803@v.loewis.de>

Gregory P. Smith schrieb:
> Whoever knows how the windows build process works and controls the
> python 2.5 windows release builds could you please make sure the
> hashlib module gets built + linked with OpenSSL rather than falling
> back to its much slower builtin implementations.

If the project files are changed in that direction, then my build will
pick that up automatically. I can't promise to change the files myself.

I'm somewhat worried about yet another size increase in pythonxy.dll
(actually, I'm personally not worried, but I anticipate complaints about
 such a change).

What should happen to the existing sha* modules?

I believe that the performance of the OpenSSL routines depends on
the way OpenSSL was built, e.g. whether the assembler implementations
are used or not. Somebody would have to check, but I doubt they are.

So in short: I'm very doubtful that such a change can still be made,
and if it is, that it will be "right". I'm accepting patches
regardless.

Regards,
Martin

From fuzzyman at voidspace.org.uk  Mon Aug  7 15:22:24 2006
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 07 Aug 2006 14:22:24 +0100
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D73635.4000003@v.loewis.de>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>	<44D2F653.5090305@egenix.com>	<44D3140A.2040600@canterbury.ac.nz>	<44D31E64.3070407@egenix.com>
	<44D73635.4000003@v.loewis.de>
Message-ID: <44D73E90.6070707@voidspace.org.uk>

Martin v. L?wis wrote:
> [snip..]
> Expanding this view to Unicode should mean that a unicode
> string U equals a byte string B if
> U.encode(system_encode) == B or B.decode(system_encoding) == U,
> and that they don't equal otherwise (e.g. if the conversion
> fails with a "not convertible" exception). Which of the
> two conversions is selected is arbitrary; we should, of
> course, continue to use the one we always used (for
> "ascii", there is no difference between the two).
>
>   
+1

This seems the most (only ?) logical solution.

Michael Foord

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>
>   


From tanzer at swing.co.at  Mon Aug  7 16:26:03 2006
From: tanzer at swing.co.at (tanzer at swing.co.at)
Date: Mon, 07 Aug 2006 16:26:03 +0200
Subject: [Python-Dev] 2.5 status
In-Reply-To: Your message of "Thu, 03 Aug 2006 22:49:22 PDT."
	<ee2a432c0608032249i5779606btb24a2324204239e3@mail.gmail.com>
Message-ID: <E1GA63P-0001nL-GS@swing.co.at>

"Neal Norwitz" <nnorwitz at gmail.com> wrote:

> Things are getting better, but we still have some really important
> outstanding issues.  PLEASE CONTINUE TESTING AS MUCH AS POSSIBLE.

I've run into a problem with a big application that I wasn't able to
reproduce with a small example. I've submitted a bug report to
Sourceforge (Id 1536021)*.

As Sourceforge seems to mangle the code I used to describe the
problem, I'll include it here once more:

    The code (exception handler added to demonstrate and work around
    the problem):

                try :
                    h = hash(p)
                except OverflowError, e:
                    print type(p), p, id(p), e
                    h = id(p) & 0x0FFFFFFF

    prints the following output:

    <type 'instancemethod'>
        <bound method Script_Category.is_applicable of
           <Script_Menu_Mgr.Script_Category object at 0xb6cb4f8c>>
           3066797028 long int too large to convert to int

    This happens with Python 2.5b3, but didn't happen with Python 2.4.3.

    I assume that the hash-function for function/methods returns the
    `id` of the function. The following code demonstrates the same
    problem with a Python class whose `__hash__` returns the `id` of
    the object:

    $ python2.4
        Python 2.4.3 (#1, Jun 30 2006, 10:02:59)
        [GCC 3.4.6 (Gentoo 3.4.6-r1, ssp-3.4.5-1.0, pie-8.7.9)] on linux2
        Type "help", "copyright", "credits" or "license" for more information.
        >>> class X(object):
        ...   def __hash__(self): return id(self)
        ...
        >>> hash (X())
        -1211078036
    $ python2.5
        Python 2.5b3 (r25b3:51041, Aug  7 2006, 15:35:35)
        [GCC 3.4.6 (Gentoo 3.4.6-r1, ssp-3.4.5-1.0, pie-8.7.9)] on linux2
        Type "help", "copyright", "credits" or "license" for more information.
        >>> class X(object):
        ...   def __hash__(self): return id(self)
        ...
        >>> hash (X())
        Traceback (most recent call last):
          File "<stdin>", line 1, in <module>
        OverflowError: long int too large to convert to int

(*) http://sourceforge.net/tracker/index.php?func=detail&aid=1536021&group_id=5470&atid=105470


-- 
Christian Tanzer                                    http://www.c-tanzer.at/


From martin at v.loewis.de  Mon Aug  7 16:55:24 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 07 Aug 2006 16:55:24 +0200
Subject: [Python-Dev] 2.5 status
In-Reply-To: <E1GA63P-0001nL-GS@swing.co.at>
References: <E1GA63P-0001nL-GS@swing.co.at>
Message-ID: <44D7545C.80205@v.loewis.de>

tanzer at swing.co.at schrieb:
>     The code (exception handler added to demonstrate and work around
>     the problem):
> 
>                 try :
>                     h = hash(p)
>                 except OverflowError, e:
>                     print type(p), p, id(p), e
>                     h = id(p) & 0x0FFFFFFF
> 
>     prints the following output:
> 
>     <type 'instancemethod'>
>         <bound method Script_Category.is_applicable of
>            <Script_Menu_Mgr.Script_Category object at 0xb6cb4f8c>>
>            3066797028 long int too large to convert to int
> 
>     This happens with Python 2.5b3, but didn't happen with Python 2.4.3.
> 
>     I assume that the hash-function for function/methods returns the
>     `id` of the function.

No (not really). Instead, it combines the hash of the target object
with the address of the function object. The hash function of the
method object, in itself, cannot raise this overflow error.

However, it involves hash(p.im_self). So if Script_Category.__hash__
is implemented as you show below, this error might occur.

>         >>> class X(object):
>         ...   def __hash__(self): return id(self)
>         ...
>         >>> hash (X())
>         Traceback (most recent call last):
>           File "<stdin>", line 1, in <module>
>         OverflowError: long int too large to convert to int

Yes, this comes from id() now always returning positive integers,
which might be a long if the object pointer is > MAXINT

I think both instance_hash and slot_tp_hash should be changed
to just truncate long ints to the range LONG_MIN..LONG_MAX

Notice that this error could have occurred already in 2.4,
on a 64-bit system where sizeof(void*) > sizeof(long) (i.e.
on Win64).

Regards,
Martin

From david.nospam.hopwood at blueyonder.co.uk  Mon Aug  7 16:57:15 2006
From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood)
Date: Mon, 07 Aug 2006 15:57:15 +0100
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D73E90.6070707@voidspace.org.uk>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>	<44D2F653.5090305@egenix.com>	<44D3140A.2040600@canterbury.ac.nz>	<44D31E64.3070407@egenix.com>	<44D73635.4000003@v.loewis.de>
	<44D73E90.6070707@voidspace.org.uk>
Message-ID: <44D754CB.8070607@blueyonder.co.uk>

Michael Foord wrote:
> Martin v. L?wis wrote:
> 
>>[snip..]
>>Expanding this view to Unicode should mean that a unicode
>>string U equals a byte string B if
>>U.encode(system_encode) == B or B.decode(system_encoding) == U,
>>and that they don't equal otherwise (e.g. if the conversion
>>fails with a "not convertible" exception).

I disagree. Unicode strings should always be considered distinct from
non-ASCII byte strings. Implicitly encoding or decoding in order to
perform a comparison is a bad idea; it is expensive and will often do
the wrong thing.

The programmer should explicitly encode the Unicode string or decode
the byte string before comparison (which one of these is correct is
application-dependent).

>>Which of the two conversions is selected is arbitrary; [...]

It would not be arbitrary. In the common case where the byte encoding
uses "precomposed" characters, using "U.encode(system_encoding) == B"
will tend to succeed in more cases than "B.decode(system_encoding) == U",
because alternative representations of the same abstract character in
Unicode will be mapped to the same precomposed character.

(Whether these are cases in which the comparison *should* succeed is,
as I said above, application-dependent.)

The special case of considering US-ASCII strings to compare equal to
the corresponding Unicode string, is more reasonable than this would be
for a general byte encoding, because:

 - it can be done with no (or only a trivial) conversion,
 - US-ASCII has no precomposed characters or combining marks, so it
   does not have multiple encodings for the same abstract character,
 - Unicode has a US-ASCII subset that uses exactly the same encoding
   model as US-ASCII (whereas in general, a byte encoding might use
   an arbitrarily different encoding model to Unicode, as for example
   is the case for ISCII).

>>we should, of course, continue to use the one we always used (for
>>"ascii", there is no difference between the two).
> 
> +1
> 
> This seems the most (only ?) logical solution.

No; always considering Unicode and non-ASCII byte strings to be distinct
is just as logical.

-- 
David Hopwood <david.nospam.hopwood at blueyonder.co.uk>




From ncoghlan at gmail.com  Mon Aug  7 17:05:52 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 08 Aug 2006 01:05:52 +1000
Subject: [Python-Dev] 2.5 status
In-Reply-To: <44D7545C.80205@v.loewis.de>
References: <E1GA63P-0001nL-GS@swing.co.at> <44D7545C.80205@v.loewis.de>
Message-ID: <44D756D0.3030703@gmail.com>

Martin v. L?wis wrote:
> tanzer at swing.co.at schrieb:
>>         >>> class X(object):
>>         ...   def __hash__(self): return id(self)
>>         ...
>>         >>> hash (X())
>>         Traceback (most recent call last):
>>           File "<stdin>", line 1, in <module>
>>         OverflowError: long int too large to convert to int
> 
> Yes, this comes from id() now always returning positive integers,
> which might be a long if the object pointer is > MAXINT
> 
> I think both instance_hash and slot_tp_hash should be changed
> to just truncate long ints to the range LONG_MIN..LONG_MAX

Couldn't they be changed to invoke long's own hash method when a long object 
is returned from __hash__?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From martin at v.loewis.de  Mon Aug  7 17:41:39 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 07 Aug 2006 17:41:39 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D754CB.8070607@blueyonder.co.uk>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>	<44D2F653.5090305@egenix.com>	<44D3140A.2040600@canterbury.ac.nz>	<44D31E64.3070407@egenix.com>	<44D73635.4000003@v.loewis.de>	<44D73E90.6070707@voidspace.org.uk>
	<44D754CB.8070607@blueyonder.co.uk>
Message-ID: <44D75F33.2020200@v.loewis.de>

David Hopwood schrieb:
> I disagree. Unicode strings should always be considered distinct from
> non-ASCII byte strings. Implicitly encoding or decoding in order to
> perform a comparison is a bad idea; it is expensive and will often do
> the wrong thing.

That's a pretty irrelevant position at this point; Python has had
the notion of a system encoding since Unicode was introduced,
and we are not going to remove that just before a release candidate
of Python 2.5.

The question at hand is not whether certain object should compare
unequal, but whether comparing them should raise an exception.

>>> Which of the two conversions is selected is arbitrary; [...]
>
> It would not be arbitrary. In the common case where the byte encoding
> uses "precomposed" characters, using "U.encode(system_encoding) == B"
> will tend to succeed in more cases than "B.decode(system_encoding) == U",
> because alternative representations of the same abstract character in
> Unicode will be mapped to the same precomposed character.

No, they won't (although they should, perhaps):

py> u'o\u0308'.encode("latin-1")
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u0308' in
position 1: ordinal not in range(256)

In addition, it's also possible to find encodings (e.g. iso-2022) where
different byte sequences decode to the same Unicode string.

Regards,
Martin


From arigo at tunes.org  Mon Aug  7 17:55:00 2006
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 7 Aug 2006 17:55:00 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
	keys
In-Reply-To: <44D23807.4050604@egenix.com>
References: <eat5ds$37j$1@sea.gmane.org> <44D21A99.2080801@brainbot.com>
	<44D22989.7000202@egenix.com> <44D22FA7.7030905@brainbot.com>
	<44D234C3.4010404@egenix.com>
	<Pine.WNT.4.64.0608031842340.2112@shaolin>
	<44D23807.4050604@egenix.com>
Message-ID: <20060807155500.GA11910@code0.codespeak.net>

Hi,

On Thu, Aug 03, 2006 at 07:53:11PM +0200, M.-A. Lemburg wrote:
> > I though I'd heard (from Guido here or on the py3k list) that it was only 
> > 1 < u'abc' that would raise an exception, and that 1 == u'abc' would still 
> > evaluate to False.  Did I misunderstand?
> 
> Could be that I'm wrong.

I also seem to remember that TypeErrors should only signal ordering
non-sense, not equality.  In this case, I'm on the opinion that unicode
objects and completely-unrelated strings of random bytes should
successfully compare as unequal, but I'm not enough of a unicode user to
be sure.


A bientot,

Armin.

From greg at electricrain.com  Mon Aug  7 18:50:27 2006
From: greg at electricrain.com (Gregory P. Smith)
Date: Mon, 7 Aug 2006 09:50:27 -0700
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
	1535502]
In-Reply-To: <44D73D26.4050803@v.loewis.de>
References: <20060806194135.GC9899@zot.electricrain.com>
	<44D73D26.4050803@v.loewis.de>
Message-ID: <20060807165027.GD9899@zot.electricrain.com>

On Mon, Aug 07, 2006 at 03:16:22PM +0200, "Martin v. L?wis" wrote:
> Gregory P. Smith schrieb:
> > Whoever knows how the windows build process works and controls the
> > python 2.5 windows release builds could you please make sure the
> > hashlib module gets built + linked with OpenSSL rather than falling
> > back to its much slower builtin implementations.
> 
> If the project files are changed in that direction, then my build will
> pick that up automatically. I can't promise to change the files myself.
> 
> I'm somewhat worried about yet another size increase in pythonxy.dll
> (actually, I'm personally not worried, but I anticipate complaints about
>  such a change).
> 
> What should happen to the existing sha* modules?

Actually, this change will either:

 (a) leave pythonxy.dll the same size.
   OR
 (b) *reduce* the size of pythonxy.dll.
 
hashlib's OpenSSL implementation on windows comes in the form of a
300k _hashlib.pyd library.  Build that and pythonxy.dll won't change.
If you want to reduce the pythonxy.dll size you can remove _md5, _sha,
_sha256 and _sha512 from the builtins list so long as _hashlib.pyd is
built.

There is OpenSSL library code duplication between the _hashlib (300k)
and _ssl (650k) modules the way things are linked today (static).  If
you wanted absolute smallest distro size and code reuse we'd need to
change things to use an OpenSSL.dll.  I'm not proposing that (though
it is a good idea).

> I believe that the performance of the OpenSSL routines depends on
> the way OpenSSL was built, e.g. whether the assembler implementations
> are used or not. Somebody would have to check, but I doubt they are.

That'd be unfortunate as that negatively impacts the socket _ssl
module as well.  OpenSSL should always be built with the assembler
implementations.


From fuzzyman at voidspace.org.uk  Mon Aug  7 19:07:29 2006
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 07 Aug 2006 18:07:29 +0100
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D754CB.8070607@blueyonder.co.uk>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>	<44D2F653.5090305@egenix.com>	<44D3140A.2040600@canterbury.ac.nz>	<44D31E64.3070407@egenix.com>	<44D73635.4000003@v.loewis.de>	<44D73E90.6070707@voidspace.org.uk>
	<44D754CB.8070607@blueyonder.co.uk>
Message-ID: <44D77351.80603@voidspace.org.uk>

David Hopwood wrote:[snip..]
>
>   
>>> we should, of course, continue to use the one we always used (for
>>> "ascii", there is no difference between the two).
>>>       
>> +1
>>
>> This seems the most (only ?) logical solution.
>>     
>
> No; always considering Unicode and non-ASCII byte strings to be distinct
> is just as logical.
>   
Except there has been an implicit promise in Python for years now that 
ascii byte-strings will compare equally to the unicode equivalent: lots 
of code assumes this. Breaking this is fine in principle - but for Py3K 
not Py 2.x.

That means Martin's solution is the best for the current problem. (IMHO 
of course...)

Michael
http://www.voidspace.org.uk/python/index.shtml



From anthony at interlink.com.au  Mon Aug  7 19:25:46 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue, 8 Aug 2006 03:25:46 +1000
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
	1535502]
In-Reply-To: <20060807165027.GD9899@zot.electricrain.com>
References: <20060806194135.GC9899@zot.electricrain.com>
	<44D73D26.4050803@v.loewis.de>
	<20060807165027.GD9899@zot.electricrain.com>
Message-ID: <200608080325.50041.anthony@interlink.com.au>

I'm nervous about this change being made at this stage of the release process. 
It seems to me to have a chance of causing breakages - admittedly a small 
chance, but one that's higher than I'd like.

I'd also like to make sure that the PCBuild8 directory is updated at the same 
time - with the recent disappearance of the previous free MS compiler 
version, I think this will become more important over the 18 months or so of 
Python 2.5's life.

Anthony

From greg at electricrain.com  Mon Aug  7 20:21:23 2006
From: greg at electricrain.com (Gregory P. Smith)
Date: Mon, 7 Aug 2006 11:21:23 -0700
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
	1535502]
In-Reply-To: <200608080325.50041.anthony@interlink.com.au>
References: <20060806194135.GC9899@zot.electricrain.com>
	<44D73D26.4050803@v.loewis.de>
	<20060807165027.GD9899@zot.electricrain.com>
	<200608080325.50041.anthony@interlink.com.au>
Message-ID: <20060807182123.GE9899@zot.electricrain.com>

On Tue, Aug 08, 2006 at 03:25:46AM +1000, Anthony Baxter wrote:
> I'm nervous about this change being made at this stage of the release process. 
> It seems to me to have a chance of causing breakages - admittedly a small 
> chance, but one that's higher than I'd like.

Sigh.  Half the reason I did the hashlib work was to get much faster
optimized versions of the hash algorithms into python.  I'll be
disappointed if that doesn't happen.

hashlib passes its test suite with our without openssl.  If I make the
windows project file updates to simply build and include _hashlib.pyd
in the windows installer what harm is that going to cause?  IMHO the
windows python 2.5 build as it is is missing a feature by not
including this.

> I'd also like to make sure that the PCBuild8 directory is updated at the same 
> time - with the recent disappearance of the previous free MS compiler 
> version, I think this will become more important over the 18 months or so of 
> Python 2.5's life.

agreed.

frustrated..
-greg


From david.nospam.hopwood at blueyonder.co.uk  Mon Aug  7 23:34:10 2006
From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood)
Date: Mon, 07 Aug 2006 22:34:10 +0100
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D77351.80603@voidspace.org.uk>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>	<44D2F653.5090305@egenix.com>	<44D3140A.2040600@canterbury.ac.nz>	<44D31E64.3070407@egenix.com>	<44D73635.4000003@v.loewis.de>	<44D73E90.6070707@voidspace.org.uk>
	<44D754CB.8070607@blueyonder.co.uk>
	<44D77351.80603@voidspace.org.uk>
Message-ID: <44D7B1D2.2000105@blueyonder.co.uk>

Michael Foord wrote:
> David Hopwood wrote:[snip..]
> 
>>>> we should, of course, continue to use the one we always used (for
>>>> "ascii", there is no difference between the two).
>>>
>>> +1
>>>
>>> This seems the most (only ?) logical solution.
>>
>> No; always considering Unicode and non-ASCII byte strings to be distinct
>> is just as logical.
> 
> Except there has been an implicit promise in Python for years now that
> ascii byte-strings will compare equally to the unicode equivalent: lots
> of code assumes this.

I think you must have misread my comment:

  No; always considering Unicode and *non-ASCII* byte strings to be distinct
  is just as logical.

This says nothing about comparing Unicode and ASCII byte strings.

-- 
David Hopwood <david.nospam.hopwood at blueyonder.co.uk>



From rrr at ronadam.com  Mon Aug  7 23:37:42 2006
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 07 Aug 2006 16:37:42 -0500
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D77351.80603@voidspace.org.uk>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>	<44D2F653.5090305@egenix.com>	<44D3140A.2040600@canterbury.ac.nz>	<44D31E64.3070407@egenix.com>	<44D73635.4000003@v.loewis.de>	<44D73E90.6070707@voidspace.org.uk>	<44D754CB.8070607@blueyonder.co.uk>
	<44D77351.80603@voidspace.org.uk>
Message-ID: <44D7B2A6.70108@ronadam.com>

Michael Foord wrote:
> David Hopwood wrote:[snip..]
>>   
>>>> we should, of course, continue to use the one we always used (for
>>>> "ascii", there is no difference between the two).
>>>>       
>>> +1
>>>
>>> This seems the most (only ?) logical solution.
>>>     
>> No; always considering Unicode and non-ASCII byte strings to be distinct
>> is just as logical.

Yes, that's true.  (But can't be done prior to P3k of course.) Consider 
the comparison of ...

    [3] == (3,)   ->  False

These are not the same thing even though it may be trivial to treat them 
as being equivalent.  So how smart should a equivalence comparison be? 
I think testing for interchangeability and/or taking into account 
context is going down a very difficult road.  Which is what the string 
to Unicode comparison does by making an assumption that the string type 
is in the default encoding, which it may not be.

Purity in this would insist that comparing floats and integers always 
return False, but there is little ambiguity when it comes to whether 
numerical values are equivalent or not.  The rules for their comparisons 
are fairly well established.  So numerical equivalence can be the 
exception when comparing values of differing types and its the expected 
behavior as well as the established practice in programming.


> Except there has been an implicit promise in Python for years now that 
> ascii byte-strings will compare equally to the unicode equivalent: lots 
> of code assumes this. Breaking this is fine in principle - but for Py3K 
> not Py 2.x.

Also True.  And I hope that a bytes to Unicode comparison in Py3k will 
always returns False just like [3] == (3,) always returns False.


> That means Martin's solution is the best for the current problem. (IMHO 
> of course...)

I think (IMHO) in this particular case, maintaining "backwards 
compatibility" should take precedence (until Py3k) and be the stated 
reason for the continued behavior in the documents as well.  And so 
Unicode to String comparisons should be the second exception to not 
doing data form conversions when comparing two objects.  At least for 
pre-Py3k.

Are there other cases where different types of objects compare equal? 
(Not including those where the user writes or overrides a method to get 
that functionality of course.)


Cheers,
    Ron



From martin at v.loewis.de  Tue Aug  8 01:27:24 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 01:27:24 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D7B1D2.2000105@blueyonder.co.uk>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>	<44D2F653.5090305@egenix.com>	<44D3140A.2040600@canterbury.ac.nz>	<44D31E64.3070407@egenix.com>	<44D73635.4000003@v.loewis.de>	<44D73E90.6070707@voidspace.org.uk>	<44D754CB.8070607@blueyonder.co.uk>	<44D77351.80603@voidspace.org.uk>
	<44D7B1D2.2000105@blueyonder.co.uk>
Message-ID: <44D7CC5C.5080508@v.loewis.de>

David Hopwood schrieb:
> Michael Foord wrote:
>> David Hopwood wrote:[snip..]
>>
>>>>> we should, of course, continue to use the one we always used (for
>>>>> "ascii", there is no difference between the two).
>>>> +1
>>>>
>>>> This seems the most (only ?) logical solution.
>>> No; always considering Unicode and non-ASCII byte strings to be distinct
>>> is just as logical.
> 
> I think you must have misread my comment:

Indeed. The misunderstanding originates from your sentence starting with
"no", when, in fact, you seem to be supporting the proposal I made.

Regards,
Martin

From martin at v.loewis.de  Tue Aug  8 01:46:13 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 01:46:13 +0200
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
 1535502]
In-Reply-To: <20060807165027.GD9899@zot.electricrain.com>
References: <20060806194135.GC9899@zot.electricrain.com>
	<44D73D26.4050803@v.loewis.de>
	<20060807165027.GD9899@zot.electricrain.com>
Message-ID: <44D7D0C5.2050204@v.loewis.de>

Gregory P. Smith schrieb:
> hashlib's OpenSSL implementation on windows comes in the form of a
> 300k _hashlib.pyd library.

What do you mean by "comes"? I can't find any _hashlib.vcproj file
inside the PCbuild directory.

>> I believe that the performance of the OpenSSL routines depends on
>> the way OpenSSL was built, e.g. whether the assembler implementations
>> are used or not. Somebody would have to check, but I doubt they are.
> 
> That'd be unfortunate as that negatively impacts the socket _ssl
> module as well.  OpenSSL should always be built with the assembler
> implementations.

But Visual Studio doesn't ship with an assembler! So how could I
build it?

Regards,
Martin

From greg at electricrain.com  Tue Aug  8 02:21:07 2006
From: greg at electricrain.com (Gregory P. Smith)
Date: Mon, 7 Aug 2006 17:21:07 -0700
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
	1535502]
In-Reply-To: <44D7D0C5.2050204@v.loewis.de>
References: <20060806194135.GC9899@zot.electricrain.com>
	<44D73D26.4050803@v.loewis.de>
	<20060807165027.GD9899@zot.electricrain.com>
	<44D7D0C5.2050204@v.loewis.de>
Message-ID: <20060808002107.GG9899@zot.electricrain.com>

On Tue, Aug 08, 2006 at 01:46:13AM +0200, "Martin v. L?wis" wrote:
> Gregory P. Smith schrieb:
> > hashlib's OpenSSL implementation on windows comes in the form of a
> > 300k _hashlib.pyd library.
> 
> What do you mean by "comes"? I can't find any _hashlib.vcproj file
> inside the PCbuild directory.

I'll see about creating one later today when I'm reunited with my laptop.

> >> I believe that the performance of the OpenSSL routines depends on
> >> the way OpenSSL was built, e.g. whether the assembler implementations
> >> are used or not. Somebody would have to check, but I doubt they are.
> > 
> > That'd be unfortunate as that negatively impacts the socket _ssl
> > module as well.  OpenSSL should always be built with the assembler
> > implementations.
> 
> But Visual Studio doesn't ship with an assembler! So how could I
> build it?

yes it does.  Visual Studio comes with MASM (ml) and OpenSSL ships
with build scripts to use it.  See openssl's INSTALL.W32 file.  Also,
a free assembler (NASM) is available that OpenSSL is also capable of
building with if for some reason you don't have masm installed.

Looking into how the windows python build process works (honestly
something i've not looked at since 2.0) it appears that
PCbuild/build_ssl.py handles the compilation of OpenSSL..

I haven't tested this yet (I'll try it tonight) but I believe this
patch is all thats needed to enable the openssl assembly build:

--- build_ssl.py        (revision 51136)
+++ build_ssl.py        (working copy)
@@ -98,6 +98,8 @@
         if not cmd: break
         if cmd.strip()[:5].lower() == "nmake":
             continue
+        if re.match("set OPTS=no-asm", cmd):
+            continue
         temp_bat.write(cmd)
     in_bat.close()
     temp_bat.close()

Alternatively just modifying build_ssl.py to run "ms\do_masm.bat"
before it runs "nmake -f ms\32.mak" should work.

-g


From martin at v.loewis.de  Tue Aug  8 02:23:02 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 02:23:02 +0200
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
 1535502]
In-Reply-To: <20060807182123.GE9899@zot.electricrain.com>
References: <20060806194135.GC9899@zot.electricrain.com>	<44D73D26.4050803@v.loewis.de>	<20060807165027.GD9899@zot.electricrain.com>	<200608080325.50041.anthony@interlink.com.au>
	<20060807182123.GE9899@zot.electricrain.com>
Message-ID: <44D7D966.3000407@v.loewis.de>

Gregory P. Smith schrieb:
> Sigh.  Half the reason I did the hashlib work was to get much faster
> optimized versions of the hash algorithms into python.  I'll be
> disappointed if that doesn't happen.

Sad as it sounds, it appears you just did half of the work, then
(omitting the Windows build process).

> hashlib passes its test suite with our without openssl.  If I make the
> windows project file updates to simply build and include _hashlib.pyd
> in the windows installer what harm is that going to cause?

None, if you do it correctly (where correctly includes AMD64 and IA64
builds), add text to PCbuild/readme.txt, and edit msi.py properly.

That is, assuming hashlib then still works correctly on Windows (which
is hard to tell).

> IMHO the
> windows python 2.5 build as it is is missing a feature by not
> including this.

Wrong incantation :-) We are in feature freeze now, so adding a feature
is a big no-no. You should have argued this is a bug <0.5 wink>.

Regards,
Martin

From martin at v.loewis.de  Tue Aug  8 02:29:48 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 02:29:48 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <20060807155500.GA11910@code0.codespeak.net>
References: <eat5ds$37j$1@sea.gmane.org>
	<44D21A99.2080801@brainbot.com>	<44D22989.7000202@egenix.com>
	<44D22FA7.7030905@brainbot.com>	<44D234C3.4010404@egenix.com>	<Pine.WNT.4.64.0608031842340.2112@shaolin>	<44D23807.4050604@egenix.com>
	<20060807155500.GA11910@code0.codespeak.net>
Message-ID: <44D7DAFC.6060308@v.loewis.de>

Armin Rigo schrieb:
> I also seem to remember that TypeErrors should only signal ordering
> non-sense, not equality.  In this case, I'm on the opinion that unicode
> objects and completely-unrelated strings of random bytes should
> successfully compare as unequal, but I'm not enough of a unicode user to
> be sure.

I believe this was the original intent for raising TypeErrors here in
the first place: string-vs-unicode comparison predates rich comparisons,
and there is no way to implement __cmp__ meaningfully if the strings
don't convert successfully under the system encoding (if they are
inequal, you wouldn't be able to tell which one is smaller).

With rich comparisons available, I see no reason to keep raising that
exception.

As for unicode users: As others have said, they should avoid mixing
unicode and ascii strings. We provide a fallback for a limited case
(ascii); beyond that, Python assumes that non-ascii strings represent
uninterpreted bytes, not characters.

Regards,
Martin

From greg at electricrain.com  Tue Aug  8 02:36:23 2006
From: greg at electricrain.com (Gregory P. Smith)
Date: Mon, 7 Aug 2006 17:36:23 -0700
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
	1535502]
In-Reply-To: <44D7D966.3000407@v.loewis.de>
References: <20060806194135.GC9899@zot.electricrain.com>
	<44D73D26.4050803@v.loewis.de>
	<20060807165027.GD9899@zot.electricrain.com>
	<200608080325.50041.anthony@interlink.com.au>
	<20060807182123.GE9899@zot.electricrain.com>
	<44D7D966.3000407@v.loewis.de>
Message-ID: <20060808003623.GH9899@zot.electricrain.com>

On Tue, Aug 08, 2006 at 02:23:02AM +0200, "Martin v. L?wis" wrote:
> Gregory P. Smith schrieb:
> > Sigh.  Half the reason I did the hashlib work was to get much faster
> > optimized versions of the hash algorithms into python.  I'll be
> > disappointed if that doesn't happen.
> 
> Sad as it sounds, it appears you just did half of the work, then
> (omitting the Windows build process).

I had no access to a windows build environment at the time (as many
python developers do not).  Apparently I neglected to bribe someone
who did to do it after I checked the module in. ;)

So is it worth my time doing this in a hurry for 2.5 or do other
people really just not care if python for windows uses a slow OpenSSL?

Widely deployed popular applications use python for both large scale
hashing and ssl communications.

If no, can this go in 2.5.1?  Its not an API change.

-g


From martin at v.loewis.de  Tue Aug  8 08:26:08 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 08:26:08 +0200
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
 1535502]
In-Reply-To: <20060808003623.GH9899@zot.electricrain.com>
References: <20060806194135.GC9899@zot.electricrain.com>
	<44D73D26.4050803@v.loewis.de>
	<20060807165027.GD9899@zot.electricrain.com>
	<200608080325.50041.anthony@interlink.com.au>
	<20060807182123.GE9899@zot.electricrain.com>
	<44D7D966.3000407@v.loewis.de>
	<20060808003623.GH9899@zot.electricrain.com>
Message-ID: <44D82E80.5070905@v.loewis.de>

Gregory P. Smith schrieb:
> So is it worth my time doing this in a hurry for 2.5 or do other
> people really just not care if python for windows uses a slow OpenSSL?

As I said: I would accept patches. If you arrange for a separate
_hashlib.pyd file, most of my concerns would go away. So please
do produce a patch, so we have something to review.

> Widely deployed popular applications use python for both large scale
> hashing and ssl communications.

Yet, nobody has worried about performance in all these years to notice
that the assembler code isn't being used. So it can't be that bad.
For SSL specifically, the usage of hashing is minimal, as the actual
communication uses symmetric encryption.

Regards,
Martin

From mal at egenix.com  Tue Aug  8 09:25:44 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 08 Aug 2006 09:25:44 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <44D73964.3050402@v.loewis.de>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>	<eavofa$n2f$1@sea.gmane.org>	<44D36B36.8030202@egenix.com>	<44D73964.3050402@v.loewis.de>
Message-ID: <44D83C78.4070008@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg schrieb:
>> Python just doesn't know the encoding of the 8-bit string, so can't
>> make any assumptions on it. As result, it raises an exception to inform
>> the programmer.
> 
> Oh, Python does make an assumption what the encoding is: it assumes
> it is the system encoding (i.e. "ascii"). Then invoking the ascii
> codec raises an exception, because the string clearly isn't ascii.

Right, and as consequence, Python raises an exception to let the
programmer correct the problem.

The subsequent solution to the problem may result in the
string being decoded into Unicode and the two resulting Unicode
objects being unequal, or it may also result in them being equal.
Python doesn't have this knowledge, so always returning false
is clearly wrong.

Hiding programmer errors is not making life easier in the
long run, so I'm -1 on having the equality comparison return
False.

Instead we should generate a warning in Python 2.5 and introduce
the exception in Python 2.6.

>> Note that you do have to interpret the string as characters
>> > if you compare it to Unicode and there's nothing wrong with
>> > that.
> 
> Consider this:
> py> int(3+4j)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: can't convert complex to int; use int(abs(z))
> py> 3 == 3+4j
> False
>
> So even though the conversion raises an exception, the
> values are determined to be not equal. Again, because int
> is a nearly true subset of complex, the conversion goes
> the other way, but *if* it would use the complex->int
> conversion, then the TypeError should be taken as
> a guarantee that the objects don't compare equal.

In the above example, you clearly know that the two are
unequal due to the relationship between complex numbers
having an imaginary part and integers..

The same is true for the overflow case:

>>> 2**10000 == 1.23
False
>>> float(2**10000)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
OverflowError: long int too large to convert to float

(Note that in Python 2.3 this used to raise an exception as well.)

However, this is not the case for 8-bit string vs. Unicode,
since you cannot use such extra knowledge if you find that ASCII
encoding assumption obviously doesn't match the string
in question.

> Expanding this view to Unicode should mean that a unicode
> string U equals a byte string B if
> U.encode(system_encode) == B or B.decode(system_encoding) == U,
> and that they don't equal otherwise 

Agreed.

Note that Python always coerces to the "bigger" type. As a result,
the second option is what is actually implemented in Python.

> (e.g. if the conversion
> fails with a "not convertible" exception). 

I disagree with this part.

Failure to decode a string doesn't imply inequality. It implies
that the programmer needs to step in and correct the problem by
making an explicit and conscious decision.

The alternative would be to decide that equal comparisons should never
be allowed to raise exceptions and instead have the equal comparison
return False. In which case, we'd have the revert the dict patch
altogether and instead silence all exceptions that
are generated during the equal comparison (not only in the dict
implementation), replacing them with a False return value.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 08 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From martin at v.loewis.de  Tue Aug  8 09:56:53 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 09:56:53 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <44D83C78.4070008@egenix.com>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>	<eavofa$n2f$1@sea.gmane.org>	<44D36B36.8030202@egenix.com>	<44D73964.3050402@v.loewis.de>
	<44D83C78.4070008@egenix.com>
Message-ID: <44D843C5.1060907@v.loewis.de>

M.-A. Lemburg schrieb:
> Hiding programmer errors is not making life easier in the
> long run, so I'm -1 on having the equality comparison return
> False.

There is no error to hide here. The objects are inequal, period.

> Instead we should generate a warning in Python 2.5 and introduce
> the exception in Python 2.6.

A warning about what? That you can't put byte string and Unicode
strings into the same dictionary (as keys)? Next we start not allowing
to put numbers and strings into the same dictionary, because there
is no conversion defined between them?

> In the above example, you clearly know that the two are
> unequal due to the relationship between complex numbers
> having an imaginary part and integers..

Right. And so I do when the byte string does not convert to
Unicode.

> However, this is not the case for 8-bit string vs. Unicode,
> since you cannot use such extra knowledge if you find that ASCII
> encoding assumption obviously doesn't match the string
> in question.

It's not the question "Could there be a conversion under which
they are equal?" If you ask that question, then

py> "3"==3
False

should raise an exception, because there exists a conversion under
which these objects are equal:

py> int("3")==3
True

It's just that, under the conversion Python applies, the byte
string and the Unicode string are not equal.

> Note that Python always coerces to the "bigger" type. As a result,
> the second option is what is actually implemented in Python.
[which is decode-to-unicode]

It might be debatable which of the types is the "bigger" type. It's
not that byte strings are a true subset of Unicode strings, under
some conversion, since there are byte strings which have no Unicode
equivalent (because they are not characters, and don't convert under
the encoding), and there are Unicode strings that have no byte string
equivalent.

For example, if the system encoding is UTF-8, then byte string is
the bigger type (all Unicode strings convert to byte strings, but
not all byte strings convert to Unicode strings).

However, this is a red herring: Python has, for whatever reason,
chosen to convert byte->unicode, and nobody is questioning that
choice.

> I disagree with this part.
> 
> Failure to decode a string doesn't imply inequality.

If the failure is "these bytes don't have a meaningful character
interpretation", then the bytes are *clearly* not equal to
some character string.

> It implies
> that the programmer needs to step in and correct the problem by
> making an explicit and conscious decision.

There is no problem to correct. The strings *are* inequal.

> The alternative would be to decide that equal comparisons should never
> be allowed to raise exceptions and instead have the equal comparison
> return False.

There are many reasons why comparison could raise an exception.
It could be out of memory, it could be that there is an
internal/programming error in the codec being used, it could be
that the codec is not found (likewise for other comparisons).

However, if the codec is working properly, and clearly determines
that the byte string has no character string equivalent, then
it can't be equal to some character (unicode) string.

Regards,
Martin

From greg at electricrain.com  Tue Aug  8 09:59:33 2006
From: greg at electricrain.com (Gregory P. Smith)
Date: Tue, 8 Aug 2006 00:59:33 -0700
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
	1535502]
In-Reply-To: <20060808003623.GH9899@zot.electricrain.com>
References: <20060806194135.GC9899@zot.electricrain.com>
	<44D73D26.4050803@v.loewis.de>
	<20060807165027.GD9899@zot.electricrain.com>
	<200608080325.50041.anthony@interlink.com.au>
	<20060807182123.GE9899@zot.electricrain.com>
	<44D7D966.3000407@v.loewis.de>
	<20060808003623.GH9899@zot.electricrain.com>
Message-ID: <20060808075933.GI9899@zot.electricrain.com>


I have supplied a patch that does everything needed to both make the
windows build process build OpenSSL with x86 assembly optimizations on
Win32 and to build the _hashlib.pyd module.  It works for me.

The only thing the patch doesn't do is add _hashlib.pyd to the .msi
windows installer because I want to go to bed and haven't looked into
how that works.

 http://sourceforge.net/tracker/index.php?func=detail&aid=1535502&group_id=5470&atid=105470

I know Anthony suggested "no" earlier (which is why I haven't
committed it) but I really think that should be reconsidered.
This doesn't change python at all, only fixes a build process.

Ship with optimized code by default!  help delay the heat death of the
universe from cpu fans. :)

either way, enjoy.

> So is it worth my time doing this in a hurry for 2.5 or do other
> people really just not care if python for windows uses a slow OpenSSL?
> 
> Widely deployed popular applications use python for both large scale
> hashing and ssl communications.
> 
> If no, can this go in 2.5.1?  Its not an API change.
> 
> -g

From ralf at brainbot.com  Tue Aug  8 10:12:59 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Tue, 08 Aug 2006 10:12:59 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
	unicode asdictionarykeys
In-Reply-To: <44D843C5.1060907@v.loewis.de>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>	<eavofa$n2f$1@sea.gmane.org>	<44D36B36.8030202@egenix.com>	<44D73964.3050402@v.loewis.de>	<44D83C78.4070008@egenix.com>
	<44D843C5.1060907@v.loewis.de>
Message-ID: <eb9h22$kk6$1@sea.gmane.org>

Martin v. L?wis wrote:
> M.-A. Lemburg schrieb:
>> Hiding programmer errors is not making life easier in the
>> long run, so I'm -1 on having the equality comparison return
>> False.
> 
> There is no error to hide here. The objects are inequal, period.

And in the case of dicts it hides errors randomly...

> 
>> Instead we should generate a warning in Python 2.5 and introduce
>> the exception in Python 2.6.
> 
> A warning about what? That you can't put byte string and Unicode
> strings into the same dictionary (as keys)? Next we start not allowing
> to put numbers and strings into the same dictionary, because there
> is no conversion defined between them?

A warning that an exception has been ignored while adding a key to a 
dict, I guess. I'd see keep those dict changes, this is where real 
programmer errors are hidden.

>> I disagree with this part.
>>
>> Failure to decode a string doesn't imply inequality.
> 
> If the failure is "these bytes don't have a meaningful character
> interpretation", then the bytes are *clearly* not equal to
> some character string.

One could also think of a "magic encoding", which decodes non-ascii 
strings to None, making them compare unequal to any unicode string.


From greg at electricrain.com  Tue Aug  8 10:43:42 2006
From: greg at electricrain.com (Gregory P. Smith)
Date: Tue, 8 Aug 2006 01:43:42 -0700
Subject: [Python-Dev] windows 2.5 build: use OpenSSL for hashlib [bug
	1535502]
In-Reply-To: <44D82E80.5070905@v.loewis.de>
References: <20060806194135.GC9899@zot.electricrain.com>
	<44D73D26.4050803@v.loewis.de>
	<20060807165027.GD9899@zot.electricrain.com>
	<200608080325.50041.anthony@interlink.com.au>
	<20060807182123.GE9899@zot.electricrain.com>
	<44D7D966.3000407@v.loewis.de>
	<20060808003623.GH9899@zot.electricrain.com>
	<44D82E80.5070905@v.loewis.de>
Message-ID: <20060808084342.GJ9899@zot.electricrain.com>

On Tue, Aug 08, 2006 at 08:26:08AM +0200, "Martin v. L?wis" wrote:
> Gregory P. Smith schrieb:
> > Widely deployed popular applications use python for both large scale
> > hashing and ssl communications.
> 
> Yet, nobody has worried about performance in all these years to notice
> that the assembler code isn't being used. So it can't be that bad.
> For SSL specifically, the usage of hashing is minimal, as the actual
> communication uses symmetric encryption.

OpenSSL uses x86 asm to speed up other things besides hashes.  For SSL
sockets each new connection requires an RSA or DSA public key
operation for the symmetric key exchange.  Huge speedup there:

2ghz pentium-m with C ssl:
                   sign    verify    sign/s verify/s
 rsa 1024 bits 0.007576s 0.000368s    132.0   2715.4
                   sign    verify    sign/s verify/s
 dsa 1024 bits 0.003655s 0.004409s    273.6    226.8

2ghz pentium-m with x86 asm ssl:
                   sign    verify    sign/s verify/s
 rsa 1024 bits 0.003410s 0.000171s    293.3   5843.5
                   sign    verify    sign/s verify/s
 dsa 1024 bits 0.001632s 0.001987s    612.9    503.4

[data comes from running "out32\openssl speed" in openssl 0.9.8b]


From ralf at brainbot.com  Tue Aug  8 12:08:11 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Tue, 08 Aug 2006 12:08:11 +0200
Subject: [Python-Dev] free(): invalid pointer
Message-ID: <eb9nq2$a0e$1@sea.gmane.org>

Hi all,

I've got another error porting our apps. It's a django app
and stops with (when pressing CTRL-C):

*** glibc detected *** free(): invalid pointer: 0xb684c650 ***



with MALLOC_CHECK=1 and gdb I get the following backtrace:

Program received signal SIGINT, Interrupt.
[Switching to Thread -1209690432 (LWP 10036)]
0xffffe410 in __kernel_vsyscall ()
(gdb) *** glibc detected *** free(): invalid pointer: 0xb66a22a8 ***

(gdb) bt
#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7fbd463 in __waitpid_nocancel () from 
/lib/tls/i686/cmov/libpthread.so.0
#2  0x080ed524 in posix_waitpid (self=0x0, args=0xfffffe00)
     at ./Modules/posixmodule.c:5502
#3  0x080bec34 in PyEval_EvalFrameEx (f=0x825a3b4, throwflag=0)
     at Python/ceval.c:3565
#4  0x080be459 in PyEval_EvalFrameEx (f=0x81bd60c, throwflag=0)
     at Python/ceval.c:3650
#5  0x080be459 in PyEval_EvalFrameEx (f=0x82354f4, throwflag=0)
     at Python/ceval.c:3650
#6  0x080be459 in PyEval_EvalFrameEx (f=0x81dbb94, throwflag=0)
     at Python/ceval.c:3650
#7  0x080bff75 in PyEval_EvalCodeEx (co=0xb7c91c38, globals=0xb7a3713c,
     locals=0x0, args=0x822fcd0, argcount=1, kws=0x822fcd4, kwcount=0,
     defs=0xb7a33058, defcount=2, closure=0x0) at Python/ceval.c:2832
#8  0x080bd771 in PyEval_EvalFrameEx (f=0x822fb6c, throwflag=0)
     at Python/ceval.c:3661
#9  0x080bff75 in PyEval_EvalCodeEx (co=0xb7dfeba8, globals=0xb7dfd714,
     locals=0x0, args=0x820165c, argcount=2, kws=0x8201664, kwcount=0,
     defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2832
#10 0x080bd771 in PyEval_EvalFrameEx (f=0x82014cc, throwflag=0)
     at Python/ceval.c:3661
#11 0x080bff75 in PyEval_EvalCodeEx (co=0xb7dfede8, globals=0xb7dfd714,
     locals=0x0, args=0x81ec784, argcount=1, kws=0x81ec788, kwcount=0,
     defs=0xb7cfcdb8, defcount=1, closure=0x0) at Python/ceval.c:2832
#12 0x080bd771 in PyEval_EvalFrameEx (f=0x81ec634, throwflag=0)
     at Python/ceval.c:3661
#13 0x080be459 in PyEval_EvalFrameEx (f=0x81ae054, throwflag=0)
     at Python/ceval.c:3650
#14 0x080bff75 in PyEval_EvalCodeEx (co=0xb7de2770, globals=0xb7e319bc,
     locals=0xb7e319bc, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0,
     defcount=0, closure=0x0) at Python/ceval.c:2832
---Type <return> to continue, or q <return> to quit---
#15 0x080c00f6 in PyEval_EvalCode (co=0xfffffe00, globals=0xfffffe00,
     locals=0xfffffe00) at Python/ceval.c:494
#16 0x080de682 in PyRun_FileExFlags (fp=0x815f008,
     filename=0xbfdf18b1 "manage.py", start=-512, globals=0xfffffe00,
     locals=0xfffffe00, closeit=1, flags=0xbfdefbb8) at 
Python/pythonrun.c:1255
#17 0x080de9f3 in PyRun_SimpleFileExFlags (fp=<value optimized out>,
     filename=0xbfdf18b1 "manage.py", closeit=1, flags=0xbfdefbb8)
     at Python/pythonrun.c:861
#18 0x08056a69 in Py_Main (argc=2, argv=0xbfdefc54) at Modules/main.c:496
#19 0xb7e6fea2 in __libc_start_main () from /lib/tls/i686/cmov/libc.so.6
#20 0x08055fa1 in _start () at ../sysdeps/i386/elf/start.S:119


I'm using Python 2.5b3 (trunk:51066M, Aug  3 2006, 16:55:04).

Sorry for not using the bugtracker (sf sucks). Did you guys already 
settle on a new one?

- Ralf


From arigo at tunes.org  Tue Aug  8 12:26:08 2006
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 8 Aug 2006 12:26:08 +0200
Subject: [Python-Dev] returning longs from __hash__()
Message-ID: <20060808102608.GA9098@code0.codespeak.net>

Hi all,

The 2.5 change of id() to return positive ints-or-longs is likely to
cause quite some breakage in user programs that erroneously implemented
custom __hash__() functions returning a value based on an id().  This
was discussed a few times already but it showed up again as a bug report
(#1536021).  Of course it has always been documented that id() is not
directly suitable as the return value of a custom __hash__(), but so far
it worked on 32-bit machines so people have been doing it all the time.

Maybe the user should just be able to return any integer value from a
custom __hash__() without having to worry about not exceeding
sys.maxint.

After all the returned value has no real meaning.  If a long is returned
we could take its hash again, and use that number internally.


A bientot,

Armin

From ralf at brainbot.com  Tue Aug  8 15:02:07 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Tue, 08 Aug 2006 15:02:07 +0200
Subject: [Python-Dev] free(): invalid pointer
In-Reply-To: <eb9nq2$a0e$1@sea.gmane.org>
References: <eb9nq2$a0e$1@sea.gmane.org>
Message-ID: <44D88B4F.2000200@brainbot.com>

Ralf Schmitt wrote:
> 
> 
> Sorry for not using the bugtracker (sf sucks). Did you guys already 
> settle on a new one?
> 

And sorry for bothering this list. It seems like this problem is related 
to the python cdb module.

- Ralf


From bart at r8.cg.nu  Tue Aug  8 15:39:32 2006
From: bart at r8.cg.nu (Bart Thate)
Date: Tue, 8 Aug 2006 15:39:32 +0200 (CEST)
Subject: [Python-Dev] should i put this on the bug tracker ?
Message-ID: <20060808153512.R59309@r8.cg.nu>


hello python-dev,

the following code hangs on FreeBSD 6.1-STABLE,
Python 2.5b3 (r25b3:51041, Aug 5 2006, 20:46:57)

$ cat pythontest
#!/usr/local/bin/python2.5

import os, thread, time, sys

def reboot():
    print 'reboot'
    os.execl(sys.argv[0], sys.argv[0])

thread.start_new_thread(reboot, ())

while 1:
    time.sleep(1)


works fine on python2.4

is this a bug or is it something i "should not do" ?

Bart
-------------- next part --------------
#!/usr/local/bin/python2.5

import os, thread, time, sys

def reboot():
    print 'reboot'
    os.execl(sys.argv[0], sys.argv[0])

thread.start_new_thread(reboot, ())

while 1:
    time.sleep(1)

From hyeshik at gmail.com  Tue Aug  8 16:36:52 2006
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Tue, 8 Aug 2006 23:36:52 +0900
Subject: [Python-Dev] should i put this on the bug tracker ?
In-Reply-To: <20060808153512.R59309@r8.cg.nu>
References: <20060808153512.R59309@r8.cg.nu>
Message-ID: <4f0b69dc0608080736t51656bb3s878b0dced9610b2a@mail.gmail.com>

On 8/8/06, Bart Thate <bart at r8.cg.nu> wrote:
> hello python-dev,
>
> the following code hangs on FreeBSD 6.1-STABLE,
> Python 2.5b3 (r25b3:51041, Aug 5 2006, 20:46:57)
>

Python 2.5 now uses system scope threads in FreeBSD just like
in other platforms.  So python may behave different for corner cases.

[snip]
>
> works fine on python2.4
>
> is this a bug or is it something i "should not do" ?

In my machine (FreeBSD 6.1), 2.4 and 2.5 work same.
What was the problem on your running?  Did you install
it from the port?

BTW, os.exec() from a sub-thread looks not so safe.


Hye-Shik

From bart at r8.cg.nu  Tue Aug  8 16:43:03 2006
From: bart at r8.cg.nu (Bart Thate)
Date: Tue, 8 Aug 2006 16:43:03 +0200 (CEST)
Subject: [Python-Dev] should i put this on the bug tracker ?
In-Reply-To: <4f0b69dc0608080736t51656bb3s878b0dced9610b2a@mail.gmail.com>
References: <20060808153512.R59309@r8.cg.nu>
	<4f0b69dc0608080736t51656bb3s878b0dced9610b2a@mail.gmail.com>
Message-ID: <20060808164128.T90747@r8.cg.nu>



On Tue, 8 Aug 2006, Hye-Shik Chang wrote:

> In my machine (FreeBSD 6.1), 2.4 and 2.5 work same.
> What was the problem on your running?  Did you install
> it from the port?

i installed it from the python-devel port

Bart

From bart at r8.cg.nu  Tue Aug  8 17:00:52 2006
From: bart at r8.cg.nu (Bart Thate)
Date: Tue, 8 Aug 2006 17:00:52 +0200 (CEST)
Subject: [Python-Dev] should i put this on the bug tracker ?
In-Reply-To: <4f0b69dc0608080736t51656bb3s878b0dced9610b2a@mail.gmail.com>
References: <20060808153512.R59309@r8.cg.nu>
	<4f0b69dc0608080736t51656bb3s878b0dced9610b2a@mail.gmail.com>
Message-ID: <20060808165712.Y90747@r8.cg.nu>



On Tue, 8 Aug 2006, Hye-Shik Chang wrote:

sorry i should reply a little better ;]

> On 8/8/06, Bart Thate <bart at r8.cg.nu> wrote:
> > hello python-dev,
> >
> > the following code hangs on FreeBSD 6.1-STABLE,
> > Python 2.5b3 (r25b3:51041, Aug 5 2006, 20:46:57)
> >
>
> Python 2.5 now uses system scope threads in FreeBSD just like
> in other platforms.  So python may behave different for corner cases.
>

is there a way i can compile python with the old settings ?

> [snip]
> >
> > works fine on python2.4
> >
> > is this a bug or is it something i "should not do" ?
>
> In my machine (FreeBSD 6.1), 2.4 and 2.5 work same.
> What was the problem on your running?  Did you install
> it from the port?

in python2.5 i cannot start threads after the os.execl,
python2.4 lets me reboot my app and has no problem with
starting threads after that.


> BTW, os.exec() from a sub-thread looks not so safe.

i know it has problems but since it was working fine
i thought it was ok.

>
>
> Hye-Shik

Bart

From mal at egenix.com  Tue Aug  8 17:08:18 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 08 Aug 2006 17:08:18 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <44D843C5.1060907@v.loewis.de>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>	<eavofa$n2f$1@sea.gmane.org>	<44D36B36.8030202@egenix.com>	<44D73964.3050402@v.loewis.de>	<44D83C78.4070008@egenix.com>
	<44D843C5.1060907@v.loewis.de>
Message-ID: <44D8A8E2.5090002@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg schrieb:
>> Failure to decode a string doesn't imply inequality.
> 
> If the failure is "these bytes don't have a meaningful character
> interpretation", then the bytes are *clearly* not equal to
> some character string.
>
>> It implies
>> that the programmer needs to step in and correct the problem by
>> making an explicit and conscious decision.
> 
> There is no problem to correct. The strings *are* inequal.

If the programmer writes:

x = '???'
y = u'???'
...
if x == y:
    do_something()

then he clearly has had the intention to compare two character
strings.

Now, if what you were saying were true, then the above would
simply continue to work without raising an exception, possibly
causing the application to return wrong results.

With the exception, the programmer will have a chance to correct
the problem (in this case, probably a forgotten u-prefix) and also
be safe in not having the application produce wrong data -
something that's usually hard to detect, debug and, more
importantly, can have effects which are a lot worse than
a failing application.

Note that we are not discussing changing the behavior of the
__eq__ comparison between strings and Unicode, since this has
always been to raise exceptions in case the automatic propagation
fails.

The discussion is about silencing exceptions in the dict lookup
mechanism - something which used to happen and now no longer
is done.

Since this behavior is an implementation detail of the
dictionary implementation, users perceive this change as random
exceptions occurring in their application.

While these exceptions do hint at programming errors (the main
reason for no longer silencing them), the particular case in
the dict implementation requires some extra thought.

I've suggested to go about this in a slightly more user-friendly
way, namely by giving a warning instead of raising an exception
in Python 2.5 and then going for the exception in Python 2.6.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 08 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From exarkun at divmod.com  Tue Aug  8 17:11:43 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 8 Aug 2006 11:11:43 -0400
Subject: [Python-Dev] Release manager pronouncement needed: PEP 302 Fix
In-Reply-To: <5.1.1.6.0.20060728173543.01b7c120@sparrow.telecommunity.com>
Message-ID: <20060808151143.1717.322851754.divmod.quotient.17504@ohm>

On Fri, 28 Jul 2006 18:00:36 -0400, "Phillip J. Eby" <pje at telecommunity.com> wrote:
>At 10:55 PM 7/28/2006 +0200, Martin v. L?wis wrote:
>>Phillip J. Eby wrote:
>> > The issue is that a proper fix that caches existence requires adding new
>> > types to import.c and thus might appear to be more of a feature.  I was
>> > therefore reluctant to embark upon the work without some assurance that it
>> > wouldn't be rejected as adding a last-minute feature.
>>
>>So do you have a patch, or are going to write one?
>
>Yes, it's checked in as r50916.
>
>It ultimately turned out to be simpler than I thought; only one new type
>(imp.NullImporter) was required.
>

Is this going to be the final state of PEP 302 support in Python 2.5?  I
don't particularly care how this ends up, but I'd like to know what has
been decided on (PEP 302 doesn't seem to have been updated yet) so I can
fix Twisted's test suite (which cannot even be run with Python 2.5b3
right now).

Jean-Paul

From mal at egenix.com  Tue Aug  8 17:33:46 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 08 Aug 2006 17:33:46 +0200
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <20060807155500.GA11910@code0.codespeak.net>
References: <eat5ds$37j$1@sea.gmane.org>
	<44D21A99.2080801@brainbot.com>	<44D22989.7000202@egenix.com>
	<44D22FA7.7030905@brainbot.com>	<44D234C3.4010404@egenix.com>	<Pine.WNT.4.64.0608031842340.2112@shaolin>	<44D23807.4050604@egenix.com>
	<20060807155500.GA11910@code0.codespeak.net>
Message-ID: <44D8AEDA.70305@egenix.com>

Armin Rigo wrote:
> Hi,
> 
> On Thu, Aug 03, 2006 at 07:53:11PM +0200, M.-A. Lemburg wrote:
>>> I though I'd heard (from Guido here or on the py3k list) that it was only 
>>> 1 < u'abc' that would raise an exception, and that 1 == u'abc' would still 
>>> evaluate to False.  Did I misunderstand?
>> Could be that I'm wrong.
> 
> I also seem to remember that TypeErrors should only signal ordering
> non-sense, not equality.  In this case, I'm on the opinion that unicode
> objects and completely-unrelated strings of random bytes should
> successfully compare as unequal, but I'm not enough of a unicode user to
> be sure.

Agreed - for Py3k where strings no longer exist and Unicode is
the only text type.

In Python 2.x the situation is a little different, since strings
are still very often used as container for text data.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 08 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From pje at telecommunity.com  Tue Aug  8 18:04:16 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 08 Aug 2006 12:04:16 -0400
Subject: [Python-Dev] Release manager pronouncement needed: PEP 302 Fix
In-Reply-To: <20060808151143.1717.322851754.divmod.quotient.17504@ohm>
References: <5.1.1.6.0.20060728173543.01b7c120@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060808120140.03a0b480@sparrow.telecommunity.com>

At 11:11 AM 8/8/2006 -0400, Jean-Paul Calderone wrote:
>On Fri, 28 Jul 2006 18:00:36 -0400, "Phillip J. Eby" 
><pje at telecommunity.com> wrote:
>>At 10:55 PM 7/28/2006 +0200, Martin v. L??wis wrote:
>>>Phillip J. Eby wrote:
>>> > The issue is that a proper fix that caches existence requires adding new
>>> > types to import.c and thus might appear to be more of a feature.  I was
>>> > therefore reluctant to embark upon the work without some assurance 
>>> that it
>>> > wouldn't be rejected as adding a last-minute feature.
>>>
>>>So do you have a patch, or are going to write one?
>>
>>Yes, it's checked in as r50916.
>>
>>It ultimately turned out to be simpler than I thought; only one new type
>>(imp.NullImporter) was required.
>
>Is this going to be the final state of PEP 302 support in Python 2.5?  I
>don't particularly care how this ends up, but I'd like to know what has
>been decided on (PEP 302 doesn't seem to have been updated yet) so I can
>fix Twisted's test suite (which cannot even be run with Python 2.5b3
>right now).

PEP 302 doesn't need to be changed, since Python now conforms to it 
again.  That is, every object in sys.path_importer_cache is either an 
importer or None.  It's just that there is an additional type of importer 
there that didn't occur before.  If you have code that expects only certain 
types of importers to be there, that code was already broken.


From exarkun at divmod.com  Tue Aug  8 18:18:10 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 8 Aug 2006 12:18:10 -0400
Subject: [Python-Dev] Release manager pronouncement needed: PEP 302 Fix
In-Reply-To: <5.1.1.6.0.20060808120140.03a0b480@sparrow.telecommunity.com>
Message-ID: <20060808161810.1717.717581672.divmod.quotient.17543@ohm>

On Tue, 08 Aug 2006 12:04:16 -0400, "Phillip J. Eby" <pje at telecommunity.com> wrote:
>
>PEP 302 doesn't need to be changed, since Python now conforms to it again. 
>That is, every object in sys.path_importer_cache is either an importer or 
>None.  It's just that there is an additional type of importer there that 
>didn't occur before.  If you have code that expects only certain types of 
>importers to be there, that code was already broken.
>

Thanks.

Jean-Paul

From david.nospam.hopwood at blueyonder.co.uk  Tue Aug  8 18:42:14 2006
From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood)
Date: Tue, 08 Aug 2006 17:42:14 +0100
Subject: [Python-Dev] unicode hell/mixing str and unicode as dictionary
 keys
In-Reply-To: <44D7CC5C.5080508@v.loewis.de>
References: <2773CAC687FD5F4689F526998C7E4E5F018774F9@au3010avexu1.global.avaya.com>	<44D2F653.5090305@egenix.com>	<44D3140A.2040600@canterbury.ac.nz>	<44D31E64.3070407@egenix.com>	<44D73635.4000003@v.loewis.de>	<44D73E90.6070707@voidspace.org.uk>	<44D754CB.8070607@blueyonder.co.uk>	<44D77351.80603@voidspace.org.uk>
	<44D7B1D2.2000105@blueyonder.co.uk> <44D7CC5C.5080508@v.loewis.de>
Message-ID: <44D8BEE6.70902@blueyonder.co.uk>

Martin v. L?wis wrote:
> David Hopwood schrieb:
>>Michael Foord wrote:
>>>David Hopwood wrote:[snip..]
>>>
>>>>>>we should, of course, continue to use the one we always used (for
>>>>>>"ascii", there is no difference between the two).
>>>>>
>>>>>+1
>>>>>
>>>>>This seems the most (only ?) logical solution.
>>>>
>>>>No; always considering Unicode and non-ASCII byte strings to be distinct
>>>>is just as logical.
>>
>>I think you must have misread my comment:
> 
> Indeed. The misunderstanding originates from your sentence starting with
> "no", when, in fact, you seem to be supporting the proposal I made.

I had misunderstood what the existing Python behaviour is. I now think
the current behaviour (which uses "B.decode(system_encoding) == U") is
definitely a bad idea, especially in cases where the system encoding is
not US-ASCII, but I agree that it can't be changed for 2.5.

-- 
David Hopwood <david.nospam.hopwood at blueyonder.co.uk>




From jimjjewett at gmail.com  Tue Aug  8 19:09:47 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 8 Aug 2006 13:09:47 -0400
Subject: [Python-Dev] openSSL and windows binaries - license
Message-ID: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>

The OpenSSL library implements some algorithms that are patented.  The
source code should be fine to (re)distribute, but but there may be a
slight legal risk with distributing a binary.

Note that http://www.openssl.org/support/faq.html#LEGAL1 says that we
can avoid building the problem sections with

    ./config no-idea no-mdc2 no-rc5

As best I could tell from

(search for %disabled in)
    http://svn.python.org/view/external/openssl-0.9.8a/Configure
(or search for OPTIONS in)
    http://svn.python.org/view/external/openssl-0.9.8a/Makefile

python just takes the OpenSSL defaults, which excludes RC5 and MDC2
but does build IDEA.  The documentation does not promise any of these
three, and it doesn't look like they're used internally or advertised,
but they are available if built.  It might be safer to explicitly
exclude IDEA from the binary distribution.

(Well, unless the PSF actually has an appropriate license, which is possible.

     http://svn.python.org/view/external/openssl-0.9.8a/README

says where to get them.)

-jJ

From martin at v.loewis.de  Tue Aug  8 21:00:26 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 21:00:26 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <44D8A8E2.5090002@egenix.com>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>	<eavofa$n2f$1@sea.gmane.org>	<44D36B36.8030202@egenix.com>	<44D73964.3050402@v.loewis.de>	<44D83C78.4070008@egenix.com>
	<44D843C5.1060907@v.loewis.de> <44D8A8E2.5090002@egenix.com>
Message-ID: <44D8DF4A.70408@v.loewis.de>

M.-A. Lemburg schrieb:
> If the programmer writes:
> 
> x = '???'
> y = u'???'
> ...
> if x == y:
>     do_something()
> 
> then he clearly has had the intention to compare two character
> strings.

Programmers make all kinds of mistakes when comparing objects,
assuming that things ought to be equal that actually aren't:

py> 1.6/math.pi*math.pi == 1.6
False
py> if 10*10 is 100:
...   print "yes"
... else:
...   print "no"
...
no

> Now, if what you were saying were true, then the above would
> simply continue to work without raising an exception, possibly
> causing the application to return wrong results.

That correct. It is a programming mistake, hence you get a wrong
result. However, you cannot assume that every comparison between
a string and a Unicode object is always a programming mistake.
You must not raise exceptions just because of a *potential*
programming mistake; that's what PyChecker is there for.

> Note that we are not discussing changing the behavior of the
> __eq__ comparison between strings and Unicode, since this has
> always been to raise exceptions in case the automatic propagation
> fails.

Not sure what you are discussing: This is *precisely* what I'm
discussing. Making that change would solve this problem.

> The discussion is about silencing exceptions in the dict lookup
> mechanism - something which used to happen and now no longer
> is done.

No, that's not what the discussion is about. The discussion
is about the backwards incompatibility in Python 2.5 wrt.
Python 2.4. There are several ways to solve that; silencing
the exception is just one way.

I think it is the wrong way, as I think that
string-unicode-comparison should have a consistent behaviour
no matter where the comparison occurs.

> Since this behavior is an implementation detail of the
> dictionary implementation, users perceive this change as random
> exceptions occurring in their application.

That key comparison occurs is *not* an implementation detail.
It is a necessary and documented aspect of the dictionary
lookup.

> I've suggested to go about this in a slightly more user-friendly
> way, namely by giving a warning instead of raising an exception
> in Python 2.5 and then going for the exception in Python 2.6.

Yes, and I have suggested to make it even more user-friendly
by defining string-unicode-__eq__ in a sensible manner. It
is more user-friendly, because it doesn't show the inconsistency
Michael Hudson documented in

http://mail.python.org/pipermail/python-dev/2006-August/067981.html

Regards,
Martin


From martin at v.loewis.de  Tue Aug  8 21:08:56 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 21:08:56 +0200
Subject: [Python-Dev] returning longs from __hash__()
In-Reply-To: <20060808102608.GA9098@code0.codespeak.net>
References: <20060808102608.GA9098@code0.codespeak.net>
Message-ID: <44D8E148.5050701@v.loewis.de>

Armin Rigo schrieb:
> Maybe the user should just be able to return any integer value from a
> custom __hash__() without having to worry about not exceeding
> sys.maxint.
> 
> After all the returned value has no real meaning.  If a long is returned
> we could take its hash again, and use that number internally.

Nick Coghlan already suggested that, when __hash__ returns a long int,
the tp_hash of long int should be used to compute the true hash value.

Could you see any problems with that approach? If not, and if I don't
hear other objections, I'd like to go ahead and fix it that way.

Regards,
Martin

From martin at v.loewis.de  Tue Aug  8 21:18:36 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 21:18:36 +0200
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
Message-ID: <44D8E38C.8000505@v.loewis.de>

Jim Jewett schrieb:
> The OpenSSL library implements some algorithms that are patented.  The
> source code should be fine to (re)distribute, but but there may be a
> slight legal risk with distributing a binary.

I don't want to change the build process in that way (i.e. dropping a
feature) just before a release. I personally don't think there is a risk
distributing the code (if there was, distribution of OpenSSL would also
be a risk); anybody /using/ a patented algorithm would violate the
patent. I am not a lawyer, so if you want to know for sure, ask one.

If you think it's necessary, a note could be added to the readme. Would
you like to create a patch?

Regards,
Martin

From skip at pobox.com  Tue Aug  8 21:33:58 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 8 Aug 2006 14:33:58 -0500
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode asdictionarykeys
In-Reply-To: <44D8DF4A.70408@v.loewis.de>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>
	<2m3bccwopj.fsf@starship.python.net> <eavofa$n2f$1@sea.gmane.org>
	<44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de>
	<44D83C78.4070008@egenix.com> <44D843C5.1060907@v.loewis.de>
	<44D8A8E2.5090002@egenix.com> <44D8DF4A.70408@v.loewis.de>
Message-ID: <17624.59174.240858.504775@montanaro.dyndns.org>


    Martin> Programmers make all kinds of mistakes when comparing objects,
    Martin> assuming that things ought to be equal that actually aren't:

    py> 1.6/math.pi*math.pi == 1.6
    False

By extension, perhaps Computer Science departments should begin offering
Unicode Analysis as an advanced undergraduate class. ;-)

(Sorry, couldn't resist...)

Skip

From theller at python.net  Tue Aug  8 21:46:24 2006
From: theller at python.net (Thomas Heller)
Date: Tue, 08 Aug 2006 21:46:24 +0200
Subject: [Python-Dev] Releasemanager, please approve #1532975
In-Reply-To: <eapm2p$mnm$1@sea.gmane.org>
References: <eapm2p$mnm$1@sea.gmane.org>
Message-ID: <ebapmd$fu$1@sea.gmane.org>

Thomas Heller schrieb:
> Approval requested for patch:
> http://python.org/sf/1532975
> 

What does the silence mean?  Should I go ahead and commit this patch?

Thanks,
Thomas


From martin at v.loewis.de  Tue Aug  8 21:53:48 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 21:53:48 +0200
Subject: [Python-Dev] Releasemanager, please approve #1532975
In-Reply-To: <ebapmd$fu$1@sea.gmane.org>
References: <eapm2p$mnm$1@sea.gmane.org> <ebapmd$fu$1@sea.gmane.org>
Message-ID: <44D8EBCC.2000306@v.loewis.de>

Thomas Heller schrieb:
> Thomas Heller schrieb:
>> Approval requested for patch:
>> http://python.org/sf/1532975
>>
> 
> What does the silence mean?  Should I go ahead and commit this patch?

If it's not there already, you should add it to the PEP. If you think
it is "release-critical" (i.e. it would be really embarrassing for
Python 2.5 if that change wasn't made), you should put it into the
appropriate section.

Regards,
Martin

From tim.peters at gmail.com  Tue Aug  8 22:20:01 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 8 Aug 2006 16:20:01 -0400
Subject: [Python-Dev] returning longs from __hash__()
In-Reply-To: <44D8E148.5050701@v.loewis.de>
References: <20060808102608.GA9098@code0.codespeak.net>
	<44D8E148.5050701@v.loewis.de>
Message-ID: <1f7befae0608081320j61bacd63l8ea8d3a9f32f0424@mail.gmail.com>

[Armin]
>> Maybe the user should just be able to return any integer value from a
>> custom __hash__() without having to worry about not exceeding
>> sys.maxint.
>>
>> After all the returned value has no real meaning.  If a long is returned
>> we could take its hash again, and use that number internally.

[Martin]
> Nick Coghlan already suggested that, when __hash__ returns a long int,
> the tp_hash of long int should be used to compute the true hash value.
>
> Could you see any problems with that approach? If not, and if I don't
> hear other objections, I'd like to go ahead and fix it that way.

It sounds fine to me, except I'm not immediately clear on which code
needs to be changed.  The internal _Py_HashPointer() already does
exactly this (return the hash of a Python long) when PyObject_Hash()
decides to hash an address on a SIZEOF_LONG < SIZEOF_VOID_P box ...
but on a SIZEOF_LONG == SIZEOF_VOID_P box, _Py_HashPointer() may still
return a negative C long.  I /hope/ that a class that decides to add

    def __hash__(self):
         return id(self)

will end up using the same hash code internally as when that
supposedly do-nothing-different definition doesn't exist.

Note that a while back I changed several custom __hash__ methods in
Python's test suite to stop returning id(self) (as a result of tests
failing after the "make id() non-negative" change).  That's why we
haven't seen such complaints from the buildbots recently.  I expect
that few Python programmers realize it was never legit for __hash__()
to return id(self), and that it's not worth forcing them to learn that
now ;-)

From jimjjewett at gmail.com  Tue Aug  8 22:54:44 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 8 Aug 2006 16:54:44 -0400
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <44D8E38C.8000505@v.loewis.de>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
	<44D8E38C.8000505@v.loewis.de>
Message-ID: <fb6fbf560608081354o62df4a6dy8c79aef6d0b317a4@mail.gmail.com>

On 8/8/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Jim Jewett schrieb:
> > The OpenSSL library implements some algorithms that are patented.  The
> > source code should be fine to (re)distribute, but but there may be a
> > slight legal risk with distributing a binary.

> I don't want to change the build process in that way (i.e. dropping a
> feature) just before a release.

OK, but this does argue against making the fast version available by
default on windows.  :{

The 2.5c1 windows binary does not ship with _hashlib, so IDEA is only
available if someone else has compiled it.

The problem is that if the binary is recompiled as part of the python
build (as is typical on unix), then a default build will make IDEA
available.

> I personally don't think there is a risk
> distributing the code (if there was, distribution of OpenSSL would also
> be a risk); anybody /using/ a patented algorithm would violate the

There is at least some legal precedent for treating source code
differently from binaries.  (http://jya.com/bernstein-9th.htm)  The
source code is a communication or description (which would not violate
the patent), while the binary is an implementation (which would).

Note that openssl.org do not distribute binaries themselves, and at
least some vendors (such as Red Hat) have excluded the questionable
algorithms from their binaries.

> If you think it's necessary, a note could be added to the readme. Would
> you like to create a patch?

I believe the openSSL readme
    http://svn.python.org/view/external/openssl-0.9.8a/README
is sufficient for a source release.

But for a binary release, I think that IDEA should be added to the
Configure exclude.
http://svn.python.org/view/external/openssl-0.9.8a/Configure

    # All of the following is disabled by default (RC5 was enabled
before 0.9.8):

    my %disabled = ( # "what"         => "comment"
                 "gmp"		  => "default",
+                 "idea"		  => "default",
                 "mdc2"           => "default",
                 "rc5"            => "default",
                 "shared"         => "default",
                 "zlib"           => "default",
                 "zlib-dynamic"   => "default"
               );

-jJ

From martin at v.loewis.de  Tue Aug  8 23:14:56 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 08 Aug 2006 23:14:56 +0200
Subject: [Python-Dev] returning longs from __hash__()
In-Reply-To: <1f7befae0608081320j61bacd63l8ea8d3a9f32f0424@mail.gmail.com>
References: <20060808102608.GA9098@code0.codespeak.net>	
	<44D8E148.5050701@v.loewis.de>
	<1f7befae0608081320j61bacd63l8ea8d3a9f32f0424@mail.gmail.com>
Message-ID: <44D8FED0.9060108@v.loewis.de>

Tim Peters schrieb:
> It sounds fine to me, except I'm not immediately clear on which code
> needs to be changed.

My change would essentially be the code below, in instance_hash and
slot_tp_hash; I have yet to add test cases and check for documentation
changes.

Regards,
Martin

Index: Objects/typeobject.c
===================================================================
--- Objects/typeobject.c        (Revision 51155)
+++ Objects/typeobject.c        (Arbeitskopie)
@@ -4559,7 +4559,10 @@
                Py_DECREF(func);
                if (res == NULL)
                        return -1;
-               h = PyInt_AsLong(res);
+               if (PyLong_Check(res))
+                       h = res->ob_type->tp_hash(res);
+               else
+                       h = PyInt_AsLong(res);
                Py_DECREF(res);
        }
        else {
Index: Objects/classobject.c
===================================================================
--- Objects/classobject.c       (Revision 51155)
+++ Objects/classobject.c       (Arbeitskopie)
@@ -934,11 +934,9 @@
        Py_DECREF(func);
        if (res == NULL)
                return -1;
-       if (PyInt_Check(res)) {
-               outcome = PyInt_AsLong(res);
-               if (outcome == -1)
-                       outcome = -2;
-       }
+       if (PyInt_Check(res) || PyLong_Check(res))
+               /* This already converts a -1 result to -2. */
+               outcome = res->ob_type->tp_hash(res);
        else {
                PyErr_SetString(PyExc_TypeError,
                                "__hash__() should return an int");

From greg at electricrain.com  Wed Aug  9 00:07:48 2006
From: greg at electricrain.com (Gregory P. Smith)
Date: Tue, 8 Aug 2006 15:07:48 -0700
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <fb6fbf560608081354o62df4a6dy8c79aef6d0b317a4@mail.gmail.com>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
	<44D8E38C.8000505@v.loewis.de>
	<fb6fbf560608081354o62df4a6dy8c79aef6d0b317a4@mail.gmail.com>
Message-ID: <20060808220748.GN9899@zot.electricrain.com>

On Tue, Aug 08, 2006 at 04:54:44PM -0400, Jim Jewett wrote:
> On 8/8/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > Jim Jewett schrieb:
> > > The OpenSSL library implements some algorithms that are patented.  The
> > > source code should be fine to (re)distribute, but but there may be a
> > > slight legal risk with distributing a binary.
> 
> > I don't want to change the build process in that way (i.e. dropping a
> > feature) just before a release.
> 
> OK, but this does argue against making the fast version available by
> default on windows.  :{

disabling/enabling a cipher in openssl that isn't commonly used and
isn't even directly exposed via any API to a python user hardly sounds
like dropping a feature to me.  it'll make your _ssl.pyd smaller if
anything at all.  (any sane SSL connection will negotiate AES or 3DES
as its cipher; IDEA isn't required)

If the release manager declares, "absolutely no changes to the windows
build process!"  Then clearly none of the changes I submitted will
make it in and neither would removing any hint of IDEA in 2.5 as
they're both too late.

> The 2.5c1 windows binary does not ship with _hashlib, so IDEA is only
> available if someone else has compiled it.

IDEA is a cipher not a hash algorithm.  it won't appear in _hashlib.
the code is probably already linked and present in _ssl.pyd even if
the ssl protocol itself doesn't allow that as a cipher.

> But for a binary release, I think that IDEA should be added to the
> Configure exclude.
> http://svn.python.org/view/external/openssl-0.9.8a/Configure
> 
>     # All of the following is disabled by default (RC5 was enabled
> before 0.9.8):
> 
>     my %disabled = ( # "what"         => "comment"
>                  "gmp"		  => "default",
> +                 "idea"		  => "default",
>                  "mdc2"           => "default",
>                  "rc5"            => "default",
>                  "shared"         => "default",
>                  "zlib"           => "default",
>                  "zlib-dynamic"   => "default"
>                );

yeah i'd just do that if you're worried about the code being in the
binary causing a problem.


From tim.peters at gmail.com  Wed Aug  9 03:09:11 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 8 Aug 2006 21:09:11 -0400
Subject: [Python-Dev] 2.5 status
In-Reply-To: <eav54q$jcp$1@sea.gmane.org>
References: <ee2a432c0608032249i5779606btb24a2324204239e3@mail.gmail.com>
	<eav54q$jcp$1@sea.gmane.org>
Message-ID: <1f7befae0608081809v3cd63b20l313b9d6b5cd4eef@mail.gmail.com>

[Georg Brandl, on
     http://python.org/sf/1523610 - PyArg_ParseTupleAndKeywords
potential core dump
]
> This one's almost fixed if we can decide what to do with "levels".
> I wrote some time ago:
>
> """
> With respect to this bug (which is about stack issues in Python/getargs.c
> involving misuse of the "levels" array), I think that we can drop the
> "levels" thing completely. It's only there to tell the user which exact item
> passed as part of a tuple argument cannot be accepted (and only if that
> function is implemented in C code). As tuple arguments
> are very rare "argument x" should be enough to tell the user that
> something's wrong with that tuple.
> """

More, the problem that remains is purely "a head bug":  nobody ever
bumped into it, and the only way to provoke it is to write C (calling,
e.g., PyArg_ParseTupleAndKeywords) nesting tuple codes in an argument
descriptor string to an absurd depth.  This is far from serious --
heck, it's far from even interesting <0.5 wink>.

I suggest closing this bug, getting it out of PEP 356, and opening a
new low-priority bug report against it.  Fine by me if the `levels`
convolutions went away entirely, but at this stage in the 2.5 release
process I expect that's a bad idea:  there is no /actual/ real-life
bug here, and getting rid of `levels` seems far more likely (because
of the amount of code touched) to introduce a new error than that
someone will nest tuple argument descriptors 32 deep.  If someone's
actually afraid of that, fine, s/32/320/g in getargs.c instead :-)

From greg.ewing at canterbury.ac.nz  Wed Aug  9 03:43:32 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 09 Aug 2006 13:43:32 +1200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <44D83C78.4070008@egenix.com>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>
	<2m3bccwopj.fsf@starship.python.net> <eavofa$n2f$1@sea.gmane.org>
	<44D36B36.8030202@egenix.com> <44D73964.3050402@v.loewis.de>
	<44D83C78.4070008@egenix.com>
Message-ID: <44D93DC4.4030403@canterbury.ac.nz>

M.-A. Lemburg wrote:

> Hiding programmer errors is not making life easier in the
> long run, so I'm -1 on having the equality comparison return
> False.

I don't see how this is greatly different from, e.g.

   [1, 2] == (1, 2)

returning False. Comparing things of different types
may or may not indicate a bug in the code as well,
but we don't seem to worry that it doesn't raise an
exception.

--
Greg

From greg.ewing at canterbury.ac.nz  Wed Aug  9 05:02:21 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 09 Aug 2006 15:02:21 +1200
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <44D8E38C.8000505@v.loewis.de>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
	<44D8E38C.8000505@v.loewis.de>
Message-ID: <44D9503D.1040809@canterbury.ac.nz>

Martin v. L?wis wrote:
> I personally don't think there is a risk
> distributing the code (if there was, distribution of OpenSSL would also
> be a risk); anybody /using/ a patented algorithm would violate the
> patent.

If distributing the source doesn't violate the patent,
and distributing a binary doesn't violate the patent,
then what *would* constitute a violation of a software
patent?

Writing new code using the algorithm? Compiling
something which uses it?

--
Greg

From kbk at shore.net  Wed Aug  9 07:34:21 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed, 9 Aug 2006 01:34:21 -0400 (EDT)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200608090534.k795YLcP005740@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  402 open ( +6) /  3360 closed ( +6) /  3762 total (+12)
Bugs    :  861 open ( -3) /  6114 closed (+27) /  6975 total (+24)
RFE     :  228 open ( +2) /   234 closed ( +0) /   462 total ( +2)

New / Reopened Patches
______________________

Replace the ctypes internal '_as_parameter_' mechanism  (2006-08-02)
       http://python.org/sf/1532975  opened by  Thomas Heller

Remove mentions of "PyUnit" from unittest docs  (2006-08-02)
CLOSED http://python.org/sf/1533336  opened by  Collin Winter

Allow thread(ing) tests to pass without setting stack size  (2006-08-02)
       http://python.org/sf/1533520  opened by  Matt Fleming

Let timeit accept functions  (2006-08-03)
       http://python.org/sf/1533909  opened by  Erik Demaine

Add notes on locale module changes to whatsnew25.tex  (2006-08-03)
       http://python.org/sf/1534027  opened by  Iain Lowe

Typo in weakref error message  (2006-08-03)
CLOSED http://python.org/sf/1534048  opened by  Christopher Tur Lesniewski-Laas

Fix code generation bug in 'compiler' package  (2006-08-03)
CLOSED http://python.org/sf/1534084  opened by  Neil Schemenauer

Cleanup/error-correction for unittest's docs  (2006-08-05)
CLOSED http://python.org/sf/1534922  opened by  Collin Winter

writelines() in bz2 module does not raise check for errors  (2006-08-06)
       http://python.org/sf/1535500  opened by  Lawrence Oluyede

CGIHTTPServer doesn't handle path names with embeded space  (2006-08-06)
       http://python.org/sf/1535504  opened by  Hartmut Goebel

NNTPS support in nntplib  (2006-08-06)
       http://python.org/sf/1535659  opened by  Aurojit Panda

trace.py on win32 has problems with lowercase drive names  (2006-08-07)
       http://python.org/sf/1536071  opened by  Adam Groszer

Build ctypes on OpenBSD x86_64  (2006-08-08)
       http://python.org/sf/1536908  opened by  Thomas Heller

Patches Closed
______________

New ver. of 1102879: Fix for 926423: socket timeouts  (2006-07-07)
       http://python.org/sf/1519025  closed by  nnorwitz

Remove mentions of "PyUnit" from unittest docs  (2006-08-02)
       http://python.org/sf/1533336  deleted by  collinwinter

Typo in weakref error message  (2006-08-03)
       http://python.org/sf/1534048  closed by  fdrake

Fix code generation bug in 'compiler' package  (2006-08-03)
       http://python.org/sf/1534084  closed by  nascheme

Cleanup/error-correction for unittest's docs  (2006-08-05)
       http://python.org/sf/1534922  closed by  gbrandl

New / Reopened Bugs
___________________

NetBSD build with --with-pydebug causes SIGSEGV  (2006-08-02)
       http://python.org/sf/1533105  opened by  Matt Fleming

the csv module writes files that Excel sees as SYLK files  (2006-08-01)
CLOSED http://python.org/sf/1532483  reopened by  madewokherd

Installed but not listed *.pyo break bdist_rpm  (2006-08-02)
       http://python.org/sf/1533164  opened by  Shmyrev Nick

CTypes _as_parameter_ not working as documented  (2006-08-02)
       http://python.org/sf/1533481  opened by  Shane Holloway

long -> Py_ssize_t (C/API 1.2.1)  (2006-08-02)
       http://python.org/sf/1533486  opened by  Jim Jewett

C/API sec 10 is clipped  (2006-08-02)
       http://python.org/sf/1533491  opened by  Jim Jewett

Tools/modulator does not exist (ext 1.4)  (2006-08-02)
       http://python.org/sf/1533493  opened by  Jim Jewett

botched html for index subheadings  (2004-05-26)
CLOSED http://python.org/sf/960860  reopened by  jimjjewett

__name__ doesn't show up in dir() of class  (2006-08-03)
       http://python.org/sf/1534014  opened by  Tim Chase

getlines() in linecache.py raises TypeError  (2006-08-04)
CLOSED http://python.org/sf/1534517  opened by  Stefan Behnel

Python 2.5 svn crash in _elementtree.c  (2006-08-04)
       http://python.org/sf/1534630  opened by  Barry A. Warsaw

Win32 debug version of _msi creates _msi.pyd, not _msi_d.pyd  (2006-08-04)
       http://python.org/sf/1534738  opened by  John Ehresman

sys.path gets munged with certain directory structures  (2006-08-04)
       http://python.org/sf/1534764  opened by  Gustavo Tabares

logging's fileConfig causes KeyError on shutdown  (2006-08-04)
       http://python.org/sf/1534765  opened by  mdbeachy

Identical floats print inconsistently  (2006-08-04)
CLOSED http://python.org/sf/1534769  opened by  Marc W. Abel

termios.c in qnx4.25  (2005-09-19)
       http://python.org/sf/1295179  reopened by  gbrandl

can't staticaly build modules md5 and sha  (2006-08-06)
CLOSED http://python.org/sf/1535081  opened by  kbob_ru

python segfaults when reading from closed stdin  (2006-08-05)
CLOSED http://python.org/sf/1535165  opened by  Patrick Mezard

typo in test_bz2.py  (2006-08-06)
CLOSED http://python.org/sf/1535182  opened by  Lawrence Oluyede

Python 2.5 windows builds should link hashlib with OpenSSL  (2006-08-06)
       http://python.org/sf/1535502  opened by  Gregory P. Smith

hash(method) sometimes raises OverflowError  (2006-08-07)
       http://python.org/sf/1536021  opened by  Christian Tanzer

Missing builtin help for with and as  (2006-08-08)
       http://python.org/sf/1536059  opened by  Nick Coghlan

"make install" doesn't install to /usr/lib64 on x86_64 boxes  (2006-08-07)
       http://python.org/sf/1536339  opened by  Bernhard Rosenkraenzer

Micro fix for Reference Manual  (2006-08-08)
CLOSED http://python.org/sf/1536660  opened by  Marcelo de Gomensoro Malheiros

buffer comparison emits a RuntimeWarning  (2006-08-08)
CLOSED http://python.org/sf/1536786  opened by  Thomas Heller

distutils.sysconfig.get_config_h_filename gets useless file  (2006-08-08)
       http://python.org/sf/1536825  opened by  Parzival Herzog

Typo, types.TypeType should read types.StringType  (2006-08-08)
CLOSED http://python.org/sf/1536828  opened by  Reillyeon

Bugs Closed
___________

socket timeouts + Ctrl-C don't play nice  (2004-03-30)
       http://python.org/sf/926423  closed by  nnorwitz

mailbox.PortableUnixMailbox fails to parse 'From ' in body  (2006-04-13)
       http://python.org/sf/1470212  closed by  akuchling

Build fails on MacOSX with missing symbol  (2006-07-31)
       http://python.org/sf/1531662  closed by  gdm

ctypes build fails on Solaris 10  (2006-07-28)
       http://python.org/sf/1530448  closed by  theller

the csv module writes files that Excel sees as SYLK files  (2006-08-01)
       http://python.org/sf/1532483  closed by  montanaro

incorrect behaviour of PyUnicode_EncodeMBCS?  (2006-08-01)
       http://python.org/sf/1532726  closed by  nnorwitz

format_exception raises if str(exception) raises  (2006-07-30)
       http://python.org/sf/1531405  closed by  nnorwitz

botched html for index subheadings  (2004-05-26)
       http://python.org/sf/960860  closed by  jimjjewett

MACOSX_DEPLOYMENT_TARGET checked incorrectly  (2005-04-30)
       http://python.org/sf/1193190  closed by  etrepum

[AST] Failing tests  (2005-04-27)
       http://python.org/sf/1191458  closed by  nnorwitz

Bugs of the new AST compiler  (2005-10-21)
       http://python.org/sf/1333982  closed by  nnorwitz

test_ossaudiodev timing failure  (2003-08-04)
       http://python.org/sf/783242  closed by  akuchling

repr.repr not always safe  (2003-01-12)
       http://python.org/sf/666958  closed by  akuchling

ftplib has incomplete transfer when sending files in Windows  (2004-09-22)
       http://python.org/sf/1032875  closed by  akuchling

datetime.strftime %s  (2005-07-07)
       http://python.org/sf/1234123  closed by  akuchling

__weaklist__ in typeobject.c  (2006-07-29)
       http://python.org/sf/1531003  closed by  fdrake

getlines() in linecache.py raises TypeError  (2006-08-04)
       http://python.org/sf/1534517  closed by  scoder

Identical floats print inconsistently  (2006-08-04)
       http://python.org/sf/1534769  closed by  gbrandl

zipfile -- too many files?  (2006-07-09)
       http://python.org/sf/1519452  closed by  gbrandl

can't staticaly build modules md5 and sha  (2006-08-05)
       http://python.org/sf/1535081  closed by  gbrandl

python segfaults when reading from closed stdin  (2006-08-05)
       http://python.org/sf/1535165  closed by  gbrandl

typo in test_bz2.py  (2006-08-05)
       http://python.org/sf/1535182  closed by  gbrandl

TAB SETTINGS DONT WORK (win)  (2005-10-27)
       http://python.org/sf/1339883  closed by  kbk

non-uniform behavior in  'startswith' / 'endswith'  (2006-07-10)
       http://python.org/sf/1520176  closed by  sf-robot

Micro fix for Reference Manual  (2006-08-08)
       http://python.org/sf/1536660  closed by  gbrandl

buffer comparison emits a RuntimeWarning  (2006-08-08)
       http://python.org/sf/1536786  closed by  theller

Typo, types.TypeType should read types.StringType  (2006-08-08)
       http://python.org/sf/1536828  closed by  gbrandl

New / Reopened RFE
__________________

IndexError: Add bad index to msg  (2006-08-04)
       http://python.org/sf/1534607  opened by  Michael Kleehammer

Print identical floats consistently  (2006-08-05)
       http://python.org/sf/1534942  opened by  Marc W. Abel

RFE Closed
__________

feature requests for logging lib  (2006-04-22)
       http://python.org/sf/1474577  closed by  sf-robot


From martin at v.loewis.de  Wed Aug  9 08:42:43 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 09 Aug 2006 08:42:43 +0200
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <44D9503D.1040809@canterbury.ac.nz>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
	<44D8E38C.8000505@v.loewis.de> <44D9503D.1040809@canterbury.ac.nz>
Message-ID: <44D983E3.701@v.loewis.de>

Greg Ewing schrieb:
> If distributing the source doesn't violate the patent,
> and distributing a binary doesn't violate the patent,
> then what *would* constitute a violation of a software
> patent?

IANAL, but AFAICT, the rights controlled by patent law
are the right to make, to use, to sell, to offer to sell,
and to import.

In the context of an encryption algorithm, the right to
use would be the most prominent one; you wouldn't be
allowed to use the algorithm unless you have a patent
license. In general, the right to sell and to offer to
sell would be relevant for software as well, but not
so for free software (I assume).

Regards,
Martin

From g.brandl at gmx.net  Wed Aug  9 09:10:30 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 09 Aug 2006 09:10:30 +0200
Subject: [Python-Dev] 2.5 status
In-Reply-To: <1f7befae0608081809v3cd63b20l313b9d6b5cd4eef@mail.gmail.com>
References: <ee2a432c0608032249i5779606btb24a2324204239e3@mail.gmail.com>	<eav54q$jcp$1@sea.gmane.org>
	<1f7befae0608081809v3cd63b20l313b9d6b5cd4eef@mail.gmail.com>
Message-ID: <ebc1p6$82n$1@sea.gmane.org>

Tim Peters wrote:
> [Georg Brandl, on
>      http://python.org/sf/1523610 - PyArg_ParseTupleAndKeywords
> potential core dump
> ]
>> This one's almost fixed if we can decide what to do with "levels".
>> I wrote some time ago:
>>
>> """
>> With respect to this bug (which is about stack issues in Python/getargs.c
>> involving misuse of the "levels" array), I think that we can drop the
>> "levels" thing completely. It's only there to tell the user which exact item
>> passed as part of a tuple argument cannot be accepted (and only if that
>> function is implemented in C code). As tuple arguments
>> are very rare "argument x" should be enough to tell the user that
>> something's wrong with that tuple.
>> """
> 
> More, the problem that remains is purely "a head bug":  nobody ever
> bumped into it, and the only way to provoke it is to write C (calling,
> e.g., PyArg_ParseTupleAndKeywords) nesting tuple codes in an argument
> descriptor string to an absurd depth.  This is far from serious --
> heck, it's far from even interesting <0.5 wink>.

I have now found a different solution: as the format code is checked for '('
anyway and the nesting level is tracked, I introduced an upper bound for
it (rev. 51158). I believe that fixes the problem.

Georg


From martin at v.loewis.de  Wed Aug  9 09:16:19 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 09 Aug 2006 09:16:19 +0200
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <20060808220748.GN9899@zot.electricrain.com>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>	<44D8E38C.8000505@v.loewis.de>	<fb6fbf560608081354o62df4a6dy8c79aef6d0b317a4@mail.gmail.com>
	<20060808220748.GN9899@zot.electricrain.com>
Message-ID: <44D98BC3.5090802@v.loewis.de>

Gregory P. Smith schrieb:
> disabling/enabling a cipher in openssl that isn't commonly used and
> isn't even directly exposed via any API to a python user hardly sounds
> like dropping a feature to me.

Strictly speaking, it is dropping a feature: a connection that can get
established with 2.5b3 might not get established with 2.5c1, assuming
a server that requires some IDEA-based cipher.

> (any sane SSL connection will negotiate AES or 3DES
> as its cipher; IDEA isn't required)

Ok, I'll believe you that there isn't a practical risk associated with
dropping that algorithm; I'll see what I can do.

> If the release manager declares, "absolutely no changes to the windows
> build process!"

He didn't declare that: bug fixes are always possible. He declared
"absolutely no new features without explicit permission".

> Then clearly none of the changes I submitted will
> make it in and neither would removing any hint of IDEA in 2.5 as
> they're both too late.

Those changes still need permission, as they constitute a new feature.

> yeah i'd just do that if you're worried about the code being in the
> binary causing a problem.

See, my own worries come from the "if". Should I risk breaking
somebody's application just because somebody else is worried about
breaking patents? You haven't indicated whether you also worry:
do you?

Regards,
Martin

From greg at electricrain.com  Wed Aug  9 09:56:21 2006
From: greg at electricrain.com (Gregory P. Smith)
Date: Wed, 9 Aug 2006 00:56:21 -0700
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <44D98BC3.5090802@v.loewis.de>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
	<44D8E38C.8000505@v.loewis.de>
	<fb6fbf560608081354o62df4a6dy8c79aef6d0b317a4@mail.gmail.com>
	<20060808220748.GN9899@zot.electricrain.com>
	<44D98BC3.5090802@v.loewis.de>
Message-ID: <20060809075621.GO9899@zot.electricrain.com>

> Strictly speaking, it is dropping a feature: a connection that can get
> established with 2.5b3 might not get established with 2.5c1, assuming
> a server that requires some IDEA-based cipher.
> 
> > (any sane SSL connection will negotiate AES or 3DES
> > as its cipher; IDEA isn't required)
> 
> Ok, I'll believe you that there isn't a practical risk associated with
> dropping that algorithm; I'll see what I can do.

yep.  nothing uses it.  Plus the RFC for TLS (SSL) lists the minimum
supported cipher that all implementations need as 3DES.

> > yeah i'd just do that if you're worried about the code being in the
> > binary causing a problem.
> 
> See, my own worries come from the "if". Should I risk breaking
> somebody's application just because somebody else is worried about
> breaking patents? You haven't indicated whether you also worry:
> do you?

I'm not worried about breaking applications at all in this case.  Many
linux distributions ship with their OpenSSL (used by pretty much all
software on the system including python) without IDEA support.

-greg


From robinbryce at gmail.com  Wed Aug  9 10:09:50 2006
From: robinbryce at gmail.com (Robin Bryce)
Date: Wed, 9 Aug 2006 09:09:50 +0100
Subject: [Python-Dev] 2.5b3,
	commit r46372 regressed PEP 302 machinery (sf not letting me post)
In-Reply-To: <bcf87d920608070401p78367590g9b2be0ca3650f018@mail.gmail.com>
References: <bcf87d920608070401p78367590g9b2be0ca3650f018@mail.gmail.com>
Message-ID: <bcf87d920608090109i30e5912av9793fa8b285f8e76@mail.gmail.com>

This has an sf number now #1537167, and hopefully a clearer
explanation of what I think the problem is. This is not a duplicate of
the earlier "PEP 302 Fix" thread.

Thanks,
Robin


On 07/08/06, Robin Bryce <robinbryce at gmail.com> wrote:
> Hi,
>
> Appologies for the lack of an sf#. I tried to submit this there but
> couldn't. (sf is logging me out each time I visit a new page and it is
> refusing my attempt to post anonymously).
>
> Python 2.5b3 (trunk:51136M, Aug  7 2006, 10:48:15)
> [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
>
>
> The need for speed patch commited in revision r46372 included a change
> whose intent was to reduce the number of open calls. The `continue`
> statement at line 1283 in import.c:r51136 has the effect of skipping
> the builtin import machinery if the find_module method of a custom
> importer returns None.
>
> In Python 2.4.3, if find_module returned None the builtin machinery is
> allowed to process the path tail.
>
> In my particular case I am working on an importer that deals with kid
> templates that may or may not exist as .py[c] files.
>
> The short of it is that in Python 2.4.3 this produces a usable module
> ``__import__('foo.a/templateuri')`` wheras in 2.5b3 I get import
> error. The python 2.4.3 implementation *allows* module paths that are
> not seperated with '.' Python 2.5b3 does not allow this and it does
> not look like this was an intentional change. I believe this point
> about 'illeagal' module paths is actualy independent of the regresion
> I am asserting. Detailed session logs are attatched (following the sf
> guidance even though I'm posting to py-dev)
>
> The 'use case' for the importer is: Robin wants to package a default
> template file as normal python module and provide a custom importer
> that allows users of his package to reference both: there own
> templates and html files on the file system in arbitrary locations AND
> the stock templates provided as python modules under the same name
> space. He would like to leave normal imports to the standard
> machinery.
>
> Cheers,
>
> Robin
>
>
>

From arigo at tunes.org  Wed Aug  9 11:47:25 2006
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 9 Aug 2006 11:47:25 +0200
Subject: [Python-Dev] returning longs from __hash__()
In-Reply-To: <44D8FED0.9060108@v.loewis.de>
References: <20060808102608.GA9098@code0.codespeak.net>
	<44D8E148.5050701@v.loewis.de>
	<1f7befae0608081320j61bacd63l8ea8d3a9f32f0424@mail.gmail.com>
	<44D8FED0.9060108@v.loewis.de>
Message-ID: <20060809094725.GA16199@code0.codespeak.net>

Hi Martin,

On Tue, Aug 08, 2006 at 11:14:56PM +0200, "Martin v. L?wis" wrote:
> -               h = PyInt_AsLong(res);
> +               if (PyLong_Check(res))
> +                       h = res->ob_type->tp_hash(res);

This bug will keep showing up forever :-)  It's unsafe against a user
subclassing 'long' and overriding __hash__ of that subclass to return
the object itself -- it would cause an infinite C recursion.

Instead, we need either a new API like PyLong_Hash(), or call
'PyLong_Type.tp_hash(res)'.

Same in the other change.


A bientot,

Armin

From g.brandl at gmx.net  Wed Aug  9 16:40:33 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 09 Aug 2006 16:40:33 +0200
Subject: [Python-Dev] Is this a bug?
Message-ID: <ebcs51$tgo$1@sea.gmane.org>

Is this considered a bug? Sure, deleting modules from sys.modules
isn't quite common, but it happened to me on one occasion.

Python 2.4.3 (#1, Jul 29 2006, 10:52:20)
 >>> import logging
 >>> import sys
 >>> del logging
 >>> del sys.modules['logging']
 >>> ^D
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
   File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs
     func(*targs, **kargs)
   File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown
     for h in _handlerList[:]: # was _handlers.keys():
TypeError: unsubscriptable object
Error in sys.exitfunc:
Traceback (most recent call last):
   File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs
     func(*targs, **kargs)
   File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown
     for h in _handlerList[:]: # was _handlers.keys():
TypeError: unsubscriptable object

Obviously, _handlerList (as a global) is already cleaned up, which is why
the subscript fails.

Georg


From mattjfleming at googlemail.com  Wed Aug  9 17:19:33 2006
From: mattjfleming at googlemail.com (Matt Fleming)
Date: Wed, 9 Aug 2006 15:19:33 +0000
Subject: [Python-Dev] Is this a bug?
In-Reply-To: <ebcs51$tgo$1@sea.gmane.org>
References: <ebcs51$tgo$1@sea.gmane.org>
Message-ID: <5ff4a1e50608090819i3dad63dfh3eb1dfd3ebcf2230@mail.gmail.com>

On 09/08/06, Georg Brandl <g.brandl at gmx.net> wrote:
> Is this considered a bug? Sure, deleting modules from sys.modules
> isn't quite common, but it happened to me on one occasion.
>
> Python 2.4.3 (#1, Jul 29 2006, 10:52:20)
>  >>> import logging
>  >>> import sys
>  >>> del logging
>  >>> del sys.modules['logging']
>  >>> ^D
> Error in atexit._run_exitfuncs:
> Traceback (most recent call last):
>    File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs
>      func(*targs, **kargs)
>    File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown
>      for h in _handlerList[:]: # was _handlers.keys():
> TypeError: unsubscriptable object
> Error in sys.exitfunc:
> Traceback (most recent call last):
>    File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs
>      func(*targs, **kargs)
>    File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown
>      for h in _handlerList[:]: # was _handlers.keys():
> TypeError: unsubscriptable object
>
> Obviously, _handlerList (as a global) is already cleaned up, which is why
> the subscript fails.
>
> Georg
>
> _______________________________________________


Could it be considered a bug in the atexit module (or is that what you
meant)? Seeing as there's no _decent_ way to recover from this error,
perhaps it could just slip silently passed?

-- 
Matt

http://mattssanctuary.blogspot.com

From jcarlson at uci.edu  Wed Aug  9 17:59:30 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 09 Aug 2006 08:59:30 -0700
Subject: [Python-Dev] Is this a bug?
In-Reply-To: <ebcs51$tgo$1@sea.gmane.org>
References: <ebcs51$tgo$1@sea.gmane.org>
Message-ID: <20060809084110.E71E.JCARLSON@uci.edu>


Georg Brandl <g.brandl at gmx.net> wrote:
> 
> Is this considered a bug? Sure, deleting modules from sys.modules
> isn't quite common, but it happened to me on one occasion.
> 
> Python 2.4.3 (#1, Jul 29 2006, 10:52:20)
>  >>> import logging
>  >>> import sys
>  >>> del logging
>  >>> del sys.modules['logging']
>  >>> ^D
> Error in atexit._run_exitfuncs:
> Traceback (most recent call last):
>    File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs
>      func(*targs, **kargs)
>    File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown
>      for h in _handlerList[:]: # was _handlers.keys():
> TypeError: unsubscriptable object
> Error in sys.exitfunc:
> Traceback (most recent call last):
>    File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs
>      func(*targs, **kargs)
>    File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown
>      for h in _handlerList[:]: # was _handlers.keys():
> TypeError: unsubscriptable object
> 
> Obviously, _handlerList (as a global) is already cleaned up, which is why
> the subscript fails.

Interestingly enough, I recently ran into a series of errors relating to
manipulating sys.modules and using imp.load_module...

The imp.load_module and the 3 other simple variants involved importing a
module 'x.y' without the package 'x' having been imported first.  This
resulted in all C-modules being re-initialized (builtins, 3rd party, etc.),
causing things like sys.stdout wrappers to be removed, and in the case
of wx, segfaults galore (especially during shutdown).  Python 2.3 and
2.4 performed these imports silently, while 2.5 complains "SystemError:
Parent module 'x' not loaded", which is actually a useful message, and
helped me fix it.


I have a use-case for replacing sys.modules['__main__'] with a different
module, and initial attempts to do:
    sys.modules['__main__'] = other_module
    other_module.main()

Resulted in:
    AttributeError: 'NoneType' object has no attribute 'main'

This is the case for Python 2.3, 2.4, and 2.5 beta.  prefixing the above
two operations with:
    sys.modules['_oldmain'] = sys.modules['__main__']

Is sufficient to prevent Python from tearing down everything after the
sys.modules['__main__'] reassignment.  Not a big deal, but a sys.modules
manipulation that has a gotcha.

 - Josiah


From skip at pobox.com  Wed Aug  9 18:06:25 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 9 Aug 2006 11:06:25 -0500
Subject: [Python-Dev] Is this a bug?
In-Reply-To: <5ff4a1e50608090819i3dad63dfh3eb1dfd3ebcf2230@mail.gmail.com>
References: <ebcs51$tgo$1@sea.gmane.org>
	<5ff4a1e50608090819i3dad63dfh3eb1dfd3ebcf2230@mail.gmail.com>
Message-ID: <17626.2049.928034.555633@montanaro.dyndns.org>


    Matt> Could it be considered a bug in the atexit module (or is that what
    Matt> you meant)?

Or a bug in the logging package?

Skip

From jimjjewett at gmail.com  Wed Aug  9 18:30:17 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 9 Aug 2006 12:30:17 -0400
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <44D9503D.1040809@canterbury.ac.nz>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
	<44D8E38C.8000505@v.loewis.de> <44D9503D.1040809@canterbury.ac.nz>
Message-ID: <fb6fbf560608090930n18f86098gc10b6947ad79c4a6@mail.gmail.com>

On 8/8/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> If distributing the source doesn't violate the patent,
> and distributing a binary doesn't violate the patent,
> then what *would* constitute a violation of a software
> patent?

> Writing new code using the algorithm? Compiling
> something which uses it?

Any of the above, depending on the particular judge.

The difference is that there is now some precedent saying that source
(can be) an expression, rather than an implementation.  There is no
such precedent suggesting protection for anything executable.  This
still doesn't mean that (writing or distributing) source code is
automatically safe -- but in practice, it means lawsuits would be
filed first against those using or distributing a binary.

Since the PSF does not have deep pockets, plaintiffs would prefer not
to involve them.  If the actual defendants had made an affirmative
choice to violate (by changing their own build instructions), other
BigCorps might not blame python for the risk.

"Martin v. L?wis"

> IANAL, but AFAICT, the rights controlled by patent law
> are the right to make, to use, to sell, to offer to sell,
> and to import. ... In general, the right to sell and to offer to
> sell would be relevant for software as well, but not
> so for free software (I assume).

Giving it away interferes with the patent owner's ability to sell it,
and is therefore considered interference with the exclusive right to
sell.

-jJ

From ndbecker2 at gmail.com  Wed Aug  9 18:35:13 2006
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 09 Aug 2006 12:35:13 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
Message-ID: <ebd2sp$nnv$1@sea.gmane.org>

class X (object):
    pass

X() += 2

> SyntaxError: can't assign to function call

Suppose I actually had defined __iadd__ for class X.  Python says this
syntax is invalid.  I wish is wasn't.

Here's where I might use it.  Suppose I have a container class.  Suppose I
could make a slice of this container, like so:

c = Container()
s = Slice (c, 0, 10)

where "Slice" is a class.  Then:

s += 2

Would perform iadd on the slice of the container.  A convenient syntax would
be:

Slice (c, 0, 10) += 2

Does anyone else think this would be a good addition to Python?



From jcarlson at uci.edu  Wed Aug  9 19:13:53 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 09 Aug 2006 10:13:53 -0700
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ebd2sp$nnv$1@sea.gmane.org>
References: <ebd2sp$nnv$1@sea.gmane.org>
Message-ID: <20060809101243.E721.JCARLSON@uci.edu>


Neal Becker <ndbecker2 at gmail.com> wrote:
> 
> class X (object):
>     pass
> 
> X() += 2
> 
> > SyntaxError: can't assign to function call
[snip]
> Does anyone else think this would be a good addition to Python?

No.  += implies assignment.  As the syntax error states, "can't assign
to function call".

-1

 - Josiah


From tjreedy at udel.edu  Wed Aug  9 19:41:32 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 9 Aug 2006 13:41:32 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
References: <ebd2sp$nnv$1@sea.gmane.org>
Message-ID: <ebd6od$6m4$1@sea.gmane.org>


"Neal Becker" <ndbecker2 at gmail.com> wrote in message 
news:ebd2sp$nnv$1 at sea.gmane.org...
> class X (object):
>    pass
>
> X() += 2
>
>> SyntaxError: can't assign to function call
>
> Suppose I actually had defined __iadd__ for class X.  Python says this
> syntax is invalid.  I wish is wasn't.

If you translate to x() = x() +2, with x called just once, it does not make 
sense.  You can bind to names and slots in compound objects, but not to 
objects.

> Here's where I might use it.  Suppose I have a container class.  Suppose 
> I
> could make a slice of this container, like so:
>
> c = Container()
> s = Slice (c, 0, 10)
>
> where "Slice" is a class.  Then:
>
> s += 2
>
> Would perform iadd on the slice of the container.  A convenient syntax 
> would
> be:
>
> Slice (c, 0, 10) += 2

If this were made to work, the unbound object you are making an 
'assignment' to, purely for side effect, would disappear.  So if you don't 
want a separate object to be named and kept around, give your container a 
sliceadd method:
c.sliceadd((0,10), 2)

>
> Does anyone else think this would be a good addition to Python?

I don't.  Augmented assigment is confusing enough to many.

Terry Jan Reedy 




From ndbecker2 at gmail.com  Wed Aug  9 20:10:15 2006
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 09 Aug 2006 14:10:15 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
References: <ebd2sp$nnv$1@sea.gmane.org> <ebd6od$6m4$1@sea.gmane.org>
Message-ID: <ebd8f0$csd$1@sea.gmane.org>

Terry Reedy wrote:

> 
> "Neal Becker" <ndbecker2 at gmail.com> wrote in message
> news:ebd2sp$nnv$1 at sea.gmane.org...
>> class X (object):
>>    pass
>>
>> X() += 2
>>
>>> SyntaxError: can't assign to function call
>>
>> Suppose I actually had defined __iadd__ for class X.  Python says this
>> syntax is invalid.  I wish is wasn't.
> 
> If you translate to x() = x() +2, with x called just once, it does not
> make
> sense.  You can bind to names and slots in compound objects, but not to
> objects.
> 
>> Here's where I might use it.  Suppose I have a container class.  Suppose
>> I
>> could make a slice of this container, like so:
>>
>> c = Container()
>> s = Slice (c, 0, 10)
>>
>> where "Slice" is a class.  Then:
>>
>> s += 2
>>
>> Would perform iadd on the slice of the container.  A convenient syntax
>> would
>> be:
>>
>> Slice (c, 0, 10) += 2
> 
> If this were made to work, the unbound object you are making an
> 'assignment' to, purely for side effect, would disappear.  So if you don't
> want a separate object to be named and kept around, give your container a
> sliceadd method:
> c.sliceadd((0,10), 2)
> 
>>
>> Does anyone else think this would be a good addition to Python?
> 
> I don't.  Augmented assigment is confusing enough to many.
> 

I guess I confused things with the augmented assignment.  It's not a
question specific to augmented assignment.  There are 2 questions:

1) Should assignment to a temporary object be allowed?
2) Should the syntax for creation of a temporary object be a constructor
call?


From mwh at python.net  Wed Aug  9 22:19:40 2006
From: mwh at python.net (Michael Hudson)
Date: Wed, 09 Aug 2006 21:19:40 +0100
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ebd8f0$csd$1@sea.gmane.org> (Neal Becker's message of "Wed, 09
	Aug 2006 14:10:15 -0400")
References: <ebd2sp$nnv$1@sea.gmane.org> <ebd6od$6m4$1@sea.gmane.org>
	<ebd8f0$csd$1@sea.gmane.org>
Message-ID: <2m4pwlvezn.fsf@starship.python.net>

Neal Becker <ndbecker2 at gmail.com> writes:

> 1) Should assignment to a temporary object be allowed?

The question doesn't make sense: in Python, you assign to a name,
an attribute or a subscript, and that's it.

Cheers,
mwh

-- 
  <exarkun> I think there's a rather large difference between a stale
            twinkie and a kernel swap daemon
                                                -- from Twisted.Quotes

From g.brandl at gmx.net  Wed Aug  9 22:33:21 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 09 Aug 2006 22:33:21 +0200
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ebd6od$6m4$1@sea.gmane.org>
References: <ebd2sp$nnv$1@sea.gmane.org> <ebd6od$6m4$1@sea.gmane.org>
Message-ID: <ebdgqh$8vh$1@sea.gmane.org>

Terry Reedy wrote:
> "Neal Becker" <ndbecker2 at gmail.com> wrote in message 
> news:ebd2sp$nnv$1 at sea.gmane.org...
>> class X (object):
>>    pass
>>
>> X() += 2
>>
>>> SyntaxError: can't assign to function call
>>
>> Suppose I actually had defined __iadd__ for class X.  Python says this
>> syntax is invalid.  I wish is wasn't.
> 
> If you translate to x() = x() +2, with x called just once, it does not make 
> sense.  You can bind to names and slots in compound objects, but not to 
> objects.

This is similar to

x = ([1], 2)
x[0] += [2]

which doesn't currently work either, though it could.

Georg


From jjl at pobox.com  Wed Aug  9 22:40:25 2006
From: jjl at pobox.com (John J Lee)
Date: Wed, 9 Aug 2006 20:40:25 +0000 (UTC)
Subject: [Python-Dev] httplib and bad response chunking
In-Reply-To: <20060731020902.GA2950@cthulhu.gerg.ca>
References: <20060726023213.GA3106@cthulhu.gerg.ca>
	<Pine.LNX.4.64.0607282251320.8644@localhost>
	<20060731020902.GA2950@cthulhu.gerg.ca>
Message-ID: <Pine.LNX.4.64.0608021800030.8393@localhost>

On Sun, 30 Jul 2006, Greg Ward wrote:
[...]
> Did you look at the crude attempt at testing for this bug that I hacked
> into test_httplib.py?  I posted it to bug #1486335 here:
>
>  http://sourceforge.net/tracker/download.php?group_id=5470&atid=105470&file_id=186245&aid=1486335
>
> The idea is simple: put various chunked responses into strings and then
> feed those strings to HTTPConnection.  The trouble is that StringIO does
> not behave the same as a real socket: where HTTPResponse fails one way
> reading from a real socket (eg. infinite loop), it fails differently (or
> not at all) reading from a StringIO.  Makes testing with the FakeSocket
> class in test_httplib.py problematic.

There are always ways around unit testing problems, but...


> Maybe the right way to test httplib is to spawn a server process
> (thread?) to listen on some random port, feed various HTTP responses at
> HTTPConnection/HTTPResponse, and see what happens.  I'm not sure how to
> do that portably, though.  Well, I'll see if I can whip up a Unix-y
> solution and see if anyone knows how to make it portable.

I think adding this kind of functional test is extremely valuable, given 
that we don't seem to have any for httplib at present 
(Lib/test/test_httplib.py does not send any network packets).  Maybe you 
could add a basic smoke test that does a simple successful GET, while 
you're going about adding your chunked encoding regression test(s)?

Oh, wait: there is a functional test that uses the network, but it's at 
the end of httplib.py rather than being part of the test suite, and it 
follows the "Guru Watches Output" pattern :-)

I tried to add a test for urllib2 recently that relied on a python.org URL 
being set up, but the python.org guys are pretty overworked already and 
haven't had time to enable that yet -- so I think that simply from that 
point of view your run-your-own-server approach is better.  Why does it 
need to be unix-y, though?  We have SimpleHTTPServer.  We're not trying to 
test the OS, so I don't see a problem with using loopback and a single 
process -- and that would keep test run times down.  Um, except that MS 
OSes seem a bit odd re the loopback interface -- ISTR that, at least back 
with NT4, you just didn't get a loopback i/face unless you had an ethernet 
driver installed (back then, I was on dialup).

More unit tests would also be valuable, of course.


John


From guido at python.org  Wed Aug  9 22:41:07 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Aug 2006 13:41:07 -0700
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ebdgqh$8vh$1@sea.gmane.org>
References: <ebd2sp$nnv$1@sea.gmane.org> <ebd6od$6m4$1@sea.gmane.org>
	<ebdgqh$8vh$1@sea.gmane.org>
Message-ID: <ca471dc20608091341g6196ff9flc9e120f25d411f43@mail.gmail.com>

On 8/9/06, Georg Brandl <g.brandl at gmx.net> wrote:
> Terry Reedy wrote:
> > "Neal Becker" <ndbecker2 at gmail.com> wrote in message
> > news:ebd2sp$nnv$1 at sea.gmane.org...
> >> class X (object):
> >>    pass
> >>
> >> X() += 2
> >>
> >>> SyntaxError: can't assign to function call
> >>
> >> Suppose I actually had defined __iadd__ for class X.  Python says this
> >> syntax is invalid.  I wish is wasn't.
> >
> > If you translate to x() = x() +2, with x called just once, it does not make
> > sense.  You can bind to names and slots in compound objects, but not to
> > objects.
>
> This is similar to
>
> x = ([1], 2)
> x[0] += [2]
>
> which doesn't currently work either, though it could.

No it couldn't. You can't assign to x[0]. L += R is defined as L =
L.__iadd__(R) so L must be a valid assignment target.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Wed Aug  9 22:55:46 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 09 Aug 2006 22:55:46 +0200
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ca471dc20608091341g6196ff9flc9e120f25d411f43@mail.gmail.com>
References: <ebd2sp$nnv$1@sea.gmane.org>
	<ebd6od$6m4$1@sea.gmane.org>	<ebdgqh$8vh$1@sea.gmane.org>
	<ca471dc20608091341g6196ff9flc9e120f25d411f43@mail.gmail.com>
Message-ID: <ebdi4j$e48$1@sea.gmane.org>

Guido van Rossum wrote:

>> This is similar to
>>
>> x = ([1], 2)
>> x[0] += [2]
>>
>> which doesn't currently work either, though it could.
> 
> No it couldn't. You can't assign to x[0]. L += R is defined as L =
> L.__iadd__(R) so L must be a valid assignment target.
> 

Thanks for making that clear. I actually had filed a bug about that
behavior (http://python.org/sf/1436226), and from your comment to my
proposed patch I inferred that the report was valid, only the patch
wasn't.

Cheers,
Georg


From guido at python.org  Wed Aug  9 23:30:04 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Aug 2006 14:30:04 -0700
Subject: [Python-Dev] Dict suppressing exceptions
Message-ID: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>

I've been happily ignoring python-dev for the last three weeks or so,
and Neal just pointed me to some thorny issues that are close to
resolution but not quite yet resolved, yet need to be before beta 3 on
August 18 (Friday next week).

Here's my take on the dict-suppressing-exceptions issue (I'll send out
separate messages for each issue where Neal has asked me to weigh in).

It wasn't my idea to stop ignoring exceptions in dict lookups; I would
gladly have put this off until Py3k, where the main problem
(str-unicode __eq__ raising UnicodeError) will go away.

But since people are adamant that they want this in sooner, I suggest
that to minimize breakage we could make an exception for str-unicode
comparisons.

I came up with the following code to reproduce the issue; this prints
0 in 2.2, False in 2.3 and 2.4, but raises UnicodeDecodeError in 2.5
(head):

  a = {u"1": 1}
  x = hash(u"1")
  class C(str):
      def __hash__(s): return x
  print C("\xff") in a

The following patch makes this print False in 2.5 again.

Notes about the patch:

- this also fixes an out-of-date comment that should be fixed even if
the rest of the idea is rejected (lookdict_string() can return NULL
when it calls lookdict)

- the exception could be narrowed even further by only suppressing the
exception when startkey and key are both either str or unicode
instances.

What do people think?

--- Objects/dictobject.c	(revision 51180)
+++ Objects/dictobject.c	(working copy)
@@ -230,7 +230,8 @@
 lookdict() is general-purpose, and may return NULL if (and only if) a
 comparison raises an exception (this was new in Python 2.5).
 lookdict_string() below is specialized to string keys, comparison of which can
-never raise an exception; that function can never return NULL.  For both, when
+never raise an exception; that function can never return NULL (except when it
+decides to replace itself with the more general lookdict()).  For both, when
 the key isn't found a dictentry* is returned for which the me_value field is
 NULL; this is the slot in the dict at which the key would have been found, and
 the caller can (if it wishes) add the <key, value> pair to the returned
@@ -259,8 +260,13 @@
 		if (ep->me_hash == hash) {
 			startkey = ep->me_key;
 			cmp = PyObject_RichCompareBool(startkey, key, Py_EQ);
-			if (cmp < 0)
-				return NULL;
+			if (cmp < 0) {
+				if (PyErr_Occurred()==PyExc_UnicodeDecodeError) {
+					PyErr_Clear();
+				}
+				else
+					return NULL;
+                        }
 			if (ep0 == mp->ma_table && ep->me_key == startkey) {
 				if (cmp > 0)
 					return ep;
@@ -289,8 +295,13 @@
 		if (ep->me_hash == hash && ep->me_key != dummy) {
 			startkey = ep->me_key;
 			cmp = PyObject_RichCompareBool(startkey, key, Py_EQ);
-			if (cmp < 0)
-				return NULL;
+			if (cmp < 0) {
+				if (PyErr_Occurred()==PyExc_UnicodeDecodeError) {
+					PyErr_Clear();
+				}
+				else
+					return NULL;
+                        }
 			if (ep0 == mp->ma_table && ep->me_key == startkey) {
 				if (cmp > 0)
 					return ep;


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Aug  9 23:46:09 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Aug 2006 14:46:09 -0700
Subject: [Python-Dev] __index__ clipping
Message-ID: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>

Here's another issue where Neal thought it would be useful if I
weighed in. I'm not quite sure of the current status, but perhaps the
following would work?

- Called from Python, (10**10).__index__() should return 10000000000L,
not raise an exception or return sys.maxint.

- The nb_index slot is changed to return an object; it should return a
Python int or long without clipping (same as __index__() called from
Python).

- There should be three C APIs (perhaps fewer if some of them are never needed):

  - One to call the nb_index slot, or raise an exception if it's not
set, returning the object

  - One to return the clipped value without an exception
    (can still return an exception if something else went wrong)

  - One to return the value if it fits, raise OverflowError if it doesn't

I know this is quite the deviation from the current code but it seems
the most rational and future-proof approach.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Aug  9 23:58:39 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Aug 2006 14:58:39 -0700
Subject: [Python-Dev] cgi DoS attack
Message-ID: <ca471dc20608091458i4b17540cnd0c8e674ab1846d6@mail.gmail.com>

This is regarding #1112549. I think this can go in. It should also be
backported to 2.4 and to 2.3 (if we ever release another one of that).
I reviewed the code and added some minor comments to the SF tracker.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Thu Aug 10 02:46:54 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 9 Aug 2006 20:46:54 -0400
Subject: [Python-Dev] Dict suppressing exceptions
Message-ID: <fb6fbf560608091746w2dadbe73n4184ce2426757474@mail.gmail.com>

> It wasn't my idea to stop ignoring exceptions in dict lookups; I would
> gladly have put this off until Py3k, where the main problem
> (str-unicode __eq__ raising UnicodeError) will go away.

> But since people are adamant that they want this in sooner,

Is this true for dictionaries specifically?

Would there really be strong objections to continuing to swallow
any Exception (not BaseException) raised by __eq__ ?

Writing an __eq__ method that doesn't handle generic objects is
sloppy, but it does happen, and I would hate to wrap all dictionaries
of objects.

-jJ

From greg.ewing at canterbury.ac.nz  Thu Aug 10 03:34:59 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 10 Aug 2006 13:34:59 +1200
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <44D983E3.701@v.loewis.de>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
	<44D8E38C.8000505@v.loewis.de> <44D9503D.1040809@canterbury.ac.nz>
	<44D983E3.701@v.loewis.de>
Message-ID: <44DA8D43.4040009@canterbury.ac.nz>

Martin v. L?wis wrote:

> In the context of an encryption algorithm, the right to
> use would be the most prominent one; you wouldn't be
> allowed to use the algorithm unless you have a patent
> license.

But what does "use" *mean* in relation to an
algorithm?

--
Greg

From oliphant.travis at ieee.org  Thu Aug 10 04:46:11 2006
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Wed, 09 Aug 2006 20:46:11 -0600
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>
Message-ID: <ebe6lk$1tk$1@sea.gmane.org>

Guido van Rossum wrote:
> Here's another issue where Neal thought it would be useful if I
> weighed in. I'm not quite sure of the current status, but perhaps the
> following would work?
> 
> - Called from Python, (10**10).__index__() should return 10000000000L,
> not raise an exception or return sys.maxint.
> 
> - The nb_index slot is changed to return an object; it should return a
> Python int or long without clipping (same as __index__() called from
> Python).

I don't like this approach because nb_int and nb_long are already 
available to convert an object into an int or a long, so the only value 
of nb_index is to make sure that floats aren't able to be converted this 
way.

The value of nb_index is that it provides a "small-overhead" means to 
quickly convert any object that allows it into a Py_ssize_t value which 
is needed in many internal situations (mostly indexing related).

Rather than assume a Python integer is the only candidate for conversion 
to a Py_ssize_t value,  nb_index allows other objects to be used in that 
fashion.  Having another level of indirection where one object is first 
converted to a Python int or Python long (whichever is needed) and then 
to a Py_ssize_t value seems like un-necessary waste to me and will 
needlessly slow certain indexing operations down.

It seems like Nick's recent patches solved the problems that were 
identified.

-Travis













> 
> - There should be three C APIs (perhaps fewer if some of them are never needed):
> 
>   - One to call the nb_index slot, or raise an exception if it's not
> set, returning the object
> 
>   - One to return the clipped value without an exception
>     (can still return an exception if something else went wrong)
> 
>   - One to return the value if it fits, raise OverflowError if it doesn't
> 
> I know this is quite the deviation from the current code but it seems
> the most rational and future-proof approach.
> 


From guido at python.org  Thu Aug 10 06:08:54 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 9 Aug 2006 21:08:54 -0700
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <ebe6lk$1tk$1@sea.gmane.org>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>
	<ebe6lk$1tk$1@sea.gmane.org>
Message-ID: <ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>

On 8/9/06, Travis E. Oliphant <oliphant.travis at ieee.org> wrote:
> Guido van Rossum wrote:
> > Here's another issue where Neal thought it would be useful if I
> > weighed in. I'm not quite sure of the current status, but perhaps the
> > following would work?
> >
> > - Called from Python, (10**10).__index__() should return 10000000000L,
> > not raise an exception or return sys.maxint.
> >
> > - The nb_index slot is changed to return an object; it should return a
> > Python int or long without clipping (same as __index__() called from
> > Python).
>
> I don't like this approach because nb_int and nb_long are already
> available to convert an object into an int or a long, so the only value
> of nb_index is to make sure that floats aren't able to be converted this
> way.
>
> The value of nb_index is that it provides a "small-overhead" means to
> quickly convert any object that allows it into a Py_ssize_t value which
> is needed in many internal situations (mostly indexing related).
>
> Rather than assume a Python integer is the only candidate for conversion
> to a Py_ssize_t value,  nb_index allows other objects to be used in that
> fashion.  Having another level of indirection where one object is first
> converted to a Python int or Python long (whichever is needed) and then
> to a Py_ssize_t value seems like un-necessary waste to me and will
> needlessly slow certain indexing operations down.

Not if the object is implemented in Python, since then the value is
necessarily returned as an int or long object anyway.

Is performance of __index__ really that important?

What do you think (10**10).__index__() should return (when called from Python)?

> It seems like Nick's recent patches solved the problems that were
> identified.

Nick, can you summarize how your patches differ from my proposal?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nnorwitz at gmail.com  Thu Aug 10 06:53:19 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 9 Aug 2006 21:53:19 -0700
Subject: [Python-Dev] Is this a bug?
In-Reply-To: <20060809084110.E71E.JCARLSON@uci.edu>
References: <ebcs51$tgo$1@sea.gmane.org> <20060809084110.E71E.JCARLSON@uci.edu>
Message-ID: <ee2a432c0608092153g4e391a0rf754b01d719cb2d8@mail.gmail.com>

On 8/9/06, Josiah Carlson <jcarlson at uci.edu> wrote:
> 2.4 performed these imports silently, while 2.5 complains "SystemError:
> Parent module 'x' not loaded", which is actually a useful message, and
> helped me fix it.

Can you make a small, self-contained test case?  The SystemError
should be a normal exception and might indicate a deeper problem.

Thanks,
n

From steven.bethard at gmail.com  Thu Aug 10 07:21:47 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 9 Aug 2006 23:21:47 -0600
Subject: [Python-Dev] DRAFT: python-dev summary for 2006-07-01 to 2006-07-15
Message-ID: <d11dcfba0608092221m6d880600ya8c5178f9b76fc7e@mail.gmail.com>

Here's the summary for the first half of July. Corrections and
comments are greatly apprecieated. Particularly for the section on
restricted execution (though my understanding is that the current
plans are somewhat different than what was discussed during the first
half of July).

=============
Announcements
=============

-------------------
Python 2.5 schedule
-------------------

Python continues to make progress towards Python 2.5 final. See `PEP
356`_ for more details and the full schedule.

.. _PEP 356: http://www.python.org/dev/peps/pep-0356/

Contributing threads:

- `TRUNK FREEZE for 2.5b2, tomorrow Tuesday 11th, 00:00 UTC
<http://mail.python.org/pipermail/python-dev/2006-July/067189.html>`__
- `Subject: RELEASED Python 2.5 (beta 2)
<http://mail.python.org/pipermail/python-dev/2006-July/067255.html>`__
- `TRUNK is UNFROZEN.
<http://mail.python.org/pipermail/python-dev/2006-July/067259.html>`__

--------------------------
ctypes in the Python trunk
--------------------------

Martin v. L?wis successfully imported the ctypes history into the
Python trunk, so ctypes now shares its repository with the rest of
Python. Thanks everyone for your hard work!

Contributing threads:

- `Moving the ctypes repository to python.org
<http://mail.python.org/pipermail/python-dev/2006-July/066962.html>`__
- `Subversion outage Friday 15:00 GMT
<http://mail.python.org/pipermail/python-dev/2006-July/067101.html>`__
- `SVN write access is back
<http://mail.python.org/pipermail/python-dev/2006-July/067125.html>`__
- `Extended Subversion outage: Friday 16:40 GMT
<http://mail.python.org/pipermail/python-dev/2006-July/067127.html>`__


=========
Summaries
=========

-------------------------
Rebinding non-local names
-------------------------

A bug in the Python trunk that allowed augmented assignment to rebind
names in outer scopes initiated a new discussion about how to allow
Python code to rebind non-local names, e.g.::

    def counter(n=0):
        def increment(i=1):
            n += i # change the n in counter
            return n
        return increment

Guido definitely didn't like the idea of introducing a new rebinding
operator, e.g. ``:=`` or ``.=``. Phillip J. Eby suggested that a dot
prefix to the name could indicate rebinding, e.g.::

    def counter(n=0):
        def increment(i=1):
            .n += i
            return .n
        return increment

Guido felt uncomfortable with this proposal because it would mean that
in referencing a value the only difference between ``.NAME`` and
``NAME`` was whether or not the name was assigned to at another place
in the same function. This ambiguity could be avoided by requiring all
references to names in outer scopes to be prefixed by the dot, but
that seemed like too large of a change for Python.

There was also some discussion of trying to reuse the global keyword,
but most folks felt that ``outer`` or ``nonlocal`` was a better name::

    def counter(n=0):
        def increment(i=1):
            nonlocal n
            n += i
            return n
        return increment

Guido requested a PEP on the subject, but at the time of this summary,
none was available.

Contributing threads:

- `2.5 and beyond
<http://mail.python.org/pipermail/python-dev/2006-July/066857.html>`__
- `Lexical scoping in Python 3k
<http://mail.python.org/pipermail/python-dev/2006-July/066862.html>`__
- `Explicit Lexical Scoping (pre-PEP?)
<http://mail.python.org/pipermail/python-dev/2006-July/066978.html>`__

--------------------------------------
Python and backwards incompatibilities
--------------------------------------

A `complaint by Greg Black`_ about Python introducing backward
incompatibilities started a long discussion about how Python
developers could stay in better touch with existing user code. Greg
Black's code had relied on undocumented behavior in time.strftime()
that had allowed zeros to be passed in as default values. When that
behavior changed in Python 2.4, Greg's code (along with a number of
other people's code) was broken. Python core didn't have any unittests
for the behavior, so no tests started failing when the change was
made.

Glyph suggested adding a mechanism so that user projects could submit
buildslaves to run their test suites when changes are made to Python.
The goal here would be to identify any de facto coding standards that
weren't tested for in Python's test suite. If a change to Python
caused major projects like Twisted or Zope to start failing
dramatically, that change should be reconsidered even if it didn't
break any documented behavior. People generally thought this was a
great idea, particularly since it might catch some of these changes
earlier than alpha or beta releases.

There was also some concern that the Python 2.5 release cycle had been
sped up too much and users hadn't had enough time to report errors.
However, adding more beta releases (which seemed to be the major
suggestion) also adds more work for release managers and requires more
schedule coordination so that all the release managers can be
available.

People also had the mistaken impression that the trunk of Python was
not so stable, particularly because they had assumed that non-green
buildbots meant an unusable Python. In general, non-green buildbots
typically mean that some part of the test suite is failing, not that
the interpreter could not be built. A.M. Kuchling suggested adding
some of the `checkin policies discussion`_ to the release
announcements to make this more obvious.

.. _complaint by Greg Black:
http://www.gbch.net/gjb/blog/software/discuss/python-sucks.html
.. _checkin policies discussion: http://www.python.org/dev/tools/

Contributing threads:

- `User's complaints
<http://mail.python.org/pipermail/python-dev/2006-July/066993.html>`__
- `changing time.strftime() to accept 0s (was: User's complaints)
<http://mail.python.org/pipermail/python-dev/2006-July/067297.html>`__
- `Community buildbots (was Re: User's complaints)
<http://mail.python.org/pipermail/python-dev/2006-July/067366.html>`__
- `Community buildbots
<http://mail.python.org/pipermail/python-dev/2006-July/067367.html>`__

------------------------------
Restricted execution in Python
------------------------------

Discussion continued on Brett Cannon's PhD thesis aimed at adding
restricted execution to Python. Instead of trying to cripple objects
like ``file``, a number of people suggested making ``file()`` and
``open()`` return different objects in restricted mode. This would
require separating out some capabilities, so that, for example, a
file-like object could be returned that didn't support writing.
Michael Chermside suggested implementing a new type in C that stores
its data privately, and requires the user to provide access-checking
functions in order to make the private data visible. That way you
could implement new access restrictions from pure Python code, by
simply creating appropriate instances of the new type. Ka-Ping Yee
asked about protecting one piece of Python code from another, but
Guido and Brett suggested that the virtual machine probably couldn't
provide such barriers.

Brett's current progress is available in the bcannon-sandboxing branch.

Contributing threads:

- `For sandboxing: alternative to crippling file()
<http://mail.python.org/pipermail/python-dev/2006-July/066858.html>`__
- `doc for new restricted execution design for Python
<http://mail.python.org/pipermail/python-dev/2006-July/066881.html>`__
- `branch for sandox work created: bcannon-sandboxing
<http://mail.python.org/pipermail/python-dev/2006-July/067046.html>`__
- `In defense of Capabilities [was: doc for new restricted execution
design for Python]
<http://mail.python.org/pipermail/python-dev/2006-July/067048.html>`__
- `what can we do to hide the 'file' type?
<http://mail.python.org/pipermail/python-dev/2006-July/067051.html>`__
- `Restricted execution: what's the threat model?
<http://mail.python.org/pipermail/python-dev/2006-July/067072.html>`__
- `Capabilities / Restricted Execution
<http://mail.python.org/pipermail/python-dev/2006-July/067240.html>`__
- `second draft of sandboxing design doc
<http://mail.python.org/pipermail/python-dev/2006-July/067291.html>`__

---------------------------------
Getting the current thread frames
---------------------------------

Tim Peters snuck in a late feature, ``sys._current_frames()`` which
returns a dict mapping each thread's id to that thread's current
Python frame. Some people wanted to expose ``head_mutex`` instead to
avoid introducing a new function, but that may not exist, depending on
the build type. Given that Zope had been trying to emulate something
like this for a while, it's impossible to do correctly in an extension
module because the appropriate internals are not available, and it's
essential for debugging deadlocks, the feature was eventually approved
and checked in.

Contributing thread:

- `"Missing" 2.5 feature
<http://mail.python.org/pipermail/python-dev/2006-July/067160.html>`__

--------------------
Time-out in URL Open
--------------------

Facundo Batista asked about adding a timeout argument to
``urllib2.urlopen()``.  People generally liked the idea, but since it
would be a new feature, it would have to wait until Python 2.6.  There
was an `existing patch` modifying httplib, ftplib, telnetllib, poplib
and smtplib in this way, but it needed updating for Python 2.6.

.. _existing patch: http://bugs.python.org/723312

Contributing thread:

- `Time-out in URL Open
<http://mail.python.org/pipermail/python-dev/2006-July/066965.html>`__

-------------------------------------------
zlib module build failure on Mac OSX 10.4.7
-------------------------------------------

Skip Montanaro had some trouble building the zlib module from Python
trunk on Mac OSX. Turns out he had an old static libz.a on the path,
and that was getting found instead of the newer dynamic library.
Ronald Oussoren checked in a patch to configure that should correctly
set HAVE_ZLIB_COPY even if there is an old static library sitting
around. At the same time, he suggested that maybe
'-Wl,-search_path_first' should be added to the default LDFLAGS on OSX
so that the OSX linker doesn't look for a dylib anywhere on the path
before looking for a static library.

Contributing thread:

- `zlib module build failure on Mac OSX 10.4.7
<http://mail.python.org/pipermail/python-dev/2006-July/066880.html>`__

--------------------------------
Command line args in Python 3000
--------------------------------

Greg Ewing suggested that ``sys.argv`` could be split into the program
name (``sys.argv[0]``) and the arguments (``sys.argv[1:]``). People
liked this idea, and pointed out that it would be handy when there's
no real program name, e.g. when exec is used with an inode number as
is possible on some Unices.

This was also accompanied by a discussion about splitting sys into
more coherent subsets, e.g. immutable and mutable objects. Guido asked
for people to hold off on this discussion until Brett had a clearer
idea what pieces would need to be split off for restricted execution.

Contributing thread:

- `Handling of sys.args (Re: User's complaints)
<http://mail.python.org/pipermail/python-dev/2006-July/067339.html>`__

--------------------------
Import semantics in Jython
--------------------------

Jython had been following the Java convention that sub-packages were
imported when their parent package was imported. Guido had suggested
that at least Python packages in Jython should import with the
standard Python semantics. During this fortnight, Guido checked back
in with the Jython folks to see if they'd made their mind up about
this. Frank Wierzbicki, the new maintainer for Jython, said that he
won't have a chance to fix this until after Jython 2.3 is out, but
both he and Samuele Pedroni suggested that it was a change that needed
to be made.

Contributing thread:

- `Import semantics
<http://mail.python.org/pipermail/python-dev/2006-July/066999.html>`__

-----------------------------------------------
ImportWarnings for directories without __init__
-----------------------------------------------

After the substantial discussion last fortnight on whether or not
warnings should be issued if a directory was not imported because it
was missing an __init__.py file, Anthony Baxter suggested that the
best course of action was to suppress the ImportWarning by default,
and allow users to unsuppress it in their sitecustomize.py file.

Contributing threads:

- `ImportWarning flood
<http://mail.python.org/pipermail/python-dev/2006-July/066879.html>`__
- `ImportWarning decision
<http://mail.python.org/pipermail/python-dev/2006-July/067058.html>`__

-----------------------------------
Putting doctest code into footnotes
-----------------------------------

Benji York posted a `patch for doctest`_ that teaches doctest about
ReST-style footnotes so that you can write documentation like::

    After initializing the system [#init]_ it is possible to retrieve
    status information:

        >>> system.status()
        'good to go'

    [snip some of the doctest]

    .. [#init] Initialize the system:
        >>> system = System()
        >>> system.init()

People liked the idea, and Benji promised to update the patch for the
2.5 version of doctest.

.. _patch for doctest: http://tinyurl.com/nekam

Contributing thread:

- `Doctest and Footnotes
<http://mail.python.org/pipermail/python-dev/2006-July/067223.html>`__

-------------------------------
Using urllib.quote with unicode
-------------------------------

Stefan Rank pointed out that ``urllib.quote()`` fails with a strange
KeyError on unicode stings. He suggested either raising a TypeError or
automatically encoding to UTF-8 as suggested in
http://www.w3.org/International/O-URL-code.html. John J Lee pointed
questioned whether this was generally accepted as the Right Thing,
given `RFC 2617`_, `RFC 2616`_ and the recent discussion about a
`uriparse module`_. In the end, people seemed to agree that the safest
thing would be to raise an exception.

.. _RFC 2617: http://www.ietf.org/rfc/rfc2617.txt
.. _RFC 2616: http://www.ietf.org/rfc/rfc2616.txt
.. _uriparse module:
http://www.python.org/dev/summary/2006-06-01_2006-06-15/#rfc-3986-uniform-resource-identifiers-uris

Contributing thread:

- `urllib.quote and unicode bug resuscitation attempt
<http://mail.python.org/pipermail/python-dev/2006-July/067248.html>`__

-----------------------------------
Borderline cases for ints and longs
-----------------------------------

Neil Schemenauer asked if -2147483648 (``-sys.maxint - 1``)should be
an int or a long.  In Python 2.4 it was an int, but in the trunk, it
was a long. Tim Peters explained that -2147483648 is not actually an
int literal -- it is a long literal with a unary minus.  Nonetheless,
practicality beat purity, and Neal Norwitz and Neil Schemenauer put
together a fix to make it an int again. You can still defeat the patch
with something like ``eval('-(%s)' % str(-sys.maxint - 1)[1:])``, but
no one seemed too worried about that.

Contributing thread:

- `Unary minus bug
<http://mail.python.org/pipermail/python-dev/2006-July/067176.html>`__

---------------------------------
Adding list.get() and tuple.get()
---------------------------------

Russell E. Owen asked about adding a ``.get()`` method to lists and
tuples that would return a default value if the index was out of
range. Most people seemed to think that wanting such a method was a
bad code smell and Raymond Hettinger suggested that it could be
replaced in Python 2.5 with a simple ``seq[i] if len(seq) > i else
default``.

Contributing thread:

- `get for lists and tuples?
<http://mail.python.org/pipermail/python-dev/2006-July/067263.html>`__

-----------------------------
Adding a __dir__ magic method
-----------------------------

Tomer Filiba suggested adding a ``__dir__()`` magic method that would
be called by ``dir()``. The default one on ``object`` would do the
normal search through ``__dict__`` and the superclasses, and
subclasses of ``object`` could override this to add attributes that
are harder to find, e.g. pseudo-attributed implemented through
``__getattr__``. Everyone liked the idea and Guido said it could be
added for Python 2.6.

Contributing thread:

- `introducing __dir__?
<http://mail.python.org/pipermail/python-dev/2006-July/067095.html>`__

-----------------------------------------------------
pydoc support for attributes defined with PyGetSetDef
-----------------------------------------------------

Barry Warsaw pointed out that pydoc couldn't handle attributes defined
with PyGetSetDef because there was no corresponding type in the types
module. He found a similar problem with "member_descriptor" types like
``datetime.timedelta.days``. The latter was particularly a problem
because the datetime module was not importable in the types module
because the types module is imported before the platform-specific
extension module directory is on sys.path. He suggested a `fix for
pydoc`_ that would introduce a _types module coded in C that could
make the appropriate types available.

.. _fix for pydoc: http://bugs.python.org/1520294

Contributing thread:

- `Support for PyGetSetDefs in pydoc
<http://mail.python.org/pipermail/python-dev/2006-July/067219.html>`__


================
Deferred Threads
================
- `Python Style Sheets ? Re: User's complaints
<http://mail.python.org/pipermail/python-dev/2006-July/067432.html>`__


==================
Previous Summaries
==================
- `Cleanup of test harness for Python
<http://mail.python.org/pipermail/python-dev/2006-July/066898.html>`__
- `PEP 328 and PEP 338, redux
<http://mail.python.org/pipermail/python-dev/2006-July/066904.html>`__
- `Empty Subscript PEP on Wiki - keep or toss?
<http://mail.python.org/pipermail/python-dev/2006-July/066922.html>`__
- `More Switch: Explicit freezing
<http://mail.python.org/pipermail/python-dev/2006-July/066925.html>`__
- `Proposal to eliminate PySet_Fini
<http://mail.python.org/pipermail/python-dev/2006-July/066952.html>`__
- `Switch and static, redux
<http://mail.python.org/pipermail/python-dev/2006-July/067117.html>`__


===============
Skipped Threads
===============
- `traceback regression
<http://mail.python.org/pipermail/python-dev/2006-July/066859.html>`__
- `sys.settrace() in Python 2.3 vs. 2.4
<http://mail.python.org/pipermail/python-dev/2006-July/066899.html>`__
- `Bug in stringobject?
<http://mail.python.org/pipermail/python-dev/2006-July/066901.html>`__
- `weakattr <http://mail.python.org/pipermail/python-dev/2006-July/066905.html>`__
- `how long to wait for expat to incorporate a fix to prevent a
crasher? <http://mail.python.org/pipermail/python-dev/2006-July/066932.html>`__
- `LOAD_CONST POP_TOP
<http://mail.python.org/pipermail/python-dev/2006-July/066937.html>`__
- `Another 2.5 bug candidate?
<http://mail.python.org/pipermail/python-dev/2006-July/066940.html>`__
- `DRAFT: python-dev summary for 2006-06-01 to 2006-06-15
<http://mail.python.org/pipermail/python-dev/2006-July/066948.html>`__
- `Proposed beta 2 changes (Q for Anthony/Neal)
<http://mail.python.org/pipermail/python-dev/2006-July/066982.html>`__
- `2.5b1 Windows install
<http://mail.python.org/pipermail/python-dev/2006-July/066994.html>`__
- `Patch for commands.py to provide callback
<http://mail.python.org/pipermail/python-dev/2006-July/067019.html>`__
- `import screwiness
<http://mail.python.org/pipermail/python-dev/2006-July/067062.html>`__
- `zipfile.ZipFile('foo.zip', 'a'): file not found -> create?
<http://mail.python.org/pipermail/python-dev/2006-July/067079.html>`__
- `Musings on concurrency and scoping ("replacing" Javascript)
<http://mail.python.org/pipermail/python-dev/2006-July/067103.html>`__
- `About a month until PSF call for test trackers closes!
<http://mail.python.org/pipermail/python-dev/2006-July/067104.html>`__
- `test_ctypes failure on Mac OS X/PowerPC 10.3.9 (Panther)
<http://mail.python.org/pipermail/python-dev/2006-July/067126.html>`__
- `[slighly OT] Native speakers and hurting brains
<http://mail.python.org/pipermail/python-dev/2006-July/067132.html>`__
- `exception too expensive?
<http://mail.python.org/pipermail/python-dev/2006-July/067157.html>`__
- `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2006-July/067175.html>`__
- `xml issue in 2.5
<http://mail.python.org/pipermail/python-dev/2006-July/067187.html>`__
- `Fix for Lib/test/leakers/test_gestalt.py
<http://mail.python.org/pipermail/python-dev/2006-July/067195.html>`__
- `Discussion on Lib/test/crashers/
<http://mail.python.org/pipermail/python-dev/2006-July/067197.html>`__
- `Add new PyErr_WarnEx() to 2.5?
<http://mail.python.org/pipermail/python-dev/2006-July/067198.html>`__
- `Klocwork analysis of source if we want it
<http://mail.python.org/pipermail/python-dev/2006-July/067210.html>`__
- `easy_install
<http://mail.python.org/pipermail/python-dev/2006-July/067232.html>`__
- `subprocess.CalledProcessError.errno (#1223937)
<http://mail.python.org/pipermail/python-dev/2006-July/067234.html>`__
- `Minor: Unix icons for 2.5?
<http://mail.python.org/pipermail/python-dev/2006-July/067260.html>`__
- `PEP 356: python.org/sf/1515343 resolution
<http://mail.python.org/pipermail/python-dev/2006-July/067284.html>`__
- `Autoloading? (Making Queue.Queue easier to use)
<http://mail.python.org/pipermail/python-dev/2006-July/067292.html>`__
- `Long options support
<http://mail.python.org/pipermail/python-dev/2006-July/067309.html>`__
- `Behavior change in subprocess.py
<http://mail.python.org/pipermail/python-dev/2006-July/067311.html>`__
- `Proposal: Add Sudoku Solver To The "this" Module
<http://mail.python.org/pipermail/python-dev/2006-July/067342.html>`__
- `The buffer() function
<http://mail.python.org/pipermail/python-dev/2006-July/067360.html>`__
- `Partial support for dlmodule.c in 64-bits OSes
<http://mail.python.org/pipermail/python-dev/2006-July/067378.html>`__
- `IDLE - firewall warning
<http://mail.python.org/pipermail/python-dev/2006-July/067420.html>`__

From jcarlson at uci.edu  Thu Aug 10 08:41:07 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 09 Aug 2006 23:41:07 -0700
Subject: [Python-Dev] Is this a bug?
In-Reply-To: <ee2a432c0608092153g4e391a0rf754b01d719cb2d8@mail.gmail.com>
References: <20060809084110.E71E.JCARLSON@uci.edu>
	<ee2a432c0608092153g4e391a0rf754b01d719cb2d8@mail.gmail.com>
Message-ID: <20060809233637.1911.JCARLSON@uci.edu>


"Neal Norwitz" <nnorwitz at gmail.com> wrote:
> 
> On 8/9/06, Josiah Carlson <jcarlson at uci.edu> wrote:
> > 2.4 performed these imports silently, while 2.5 complains "SystemError:
> > Parent module 'x' not loaded", which is actually a useful message, and
> > helped me fix it.
> 
> Can you make a small, self-contained test case?  The SystemError
> should be a normal exception and might indicate a deeper problem.


Create a module z.py whose contents are:

#z.py
import sys
import imp

sys.stdout = None
try:
    y = imp.load_source('x.y', 'x/y.py', open('x/y.py'))
except:
    raise
else:
    print "printing should raise an AttributeError, but doesn't"

#end of listing for z.py

Create a module x/y.py whose contents are: (x/__init__.py not required)

#x/y.py
import sys

#end of listing for x/y.py


Running z.py in Python 2.3 and 2.4 produces:
    printing should raise an AttributeError, but doesn't

Running z.py in Python 2.5b2 produces:
    Traceback (most recent call last):
      File "test.py", line 7, in <module>
        y = imp.load_source('x.y', 'x/y.py', open('x/y.py'))
      File "x/y.py", line 1, in <module>
        import sys
    SystemError: Parent module 'x' not loaded


 - Josiah


From martin at v.loewis.de  Thu Aug 10 09:37:08 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 10 Aug 2006 09:37:08 +0200
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <44DA8D43.4040009@canterbury.ac.nz>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
	<44D8E38C.8000505@v.loewis.de>
	<44D9503D.1040809@canterbury.ac.nz> <44D983E3.701@v.loewis.de>
	<44DA8D43.4040009@canterbury.ac.nz>
Message-ID: <44DAE224.50705@v.loewis.de>

Greg Ewing schrieb:
>> In the context of an encryption algorithm, the right to
>> use would be the most prominent one; you wouldn't be
>> allowed to use the algorithm unless you have a patent
>> license.
> 
> But what does "use" *mean* in relation to an
> algorithm?

Perform it: do the steps that the algorithm says you should
do, or let a machine do it. IOW, run the code.

If your point is that software patents are evil: I fully
agree.

Regards,
Martin

From mwh at python.net  Thu Aug 10 10:23:48 2006
From: mwh at python.net (Michael Hudson)
Date: Thu, 10 Aug 2006 09:23:48 +0100
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <fb6fbf560608091746w2dadbe73n4184ce2426757474@mail.gmail.com> (Jim
	Jewett's message of "Wed, 9 Aug 2006 20:46:54 -0400")
References: <fb6fbf560608091746w2dadbe73n4184ce2426757474@mail.gmail.com>
Message-ID: <2mwt9ht2wb.fsf@starship.python.net>

"Jim Jewett" <jimjjewett at gmail.com> writes:

>> It wasn't my idea to stop ignoring exceptions in dict lookups; I would
>> gladly have put this off until Py3k, where the main problem
>> (str-unicode __eq__ raising UnicodeError) will go away.
>
>> But since people are adamant that they want this in sooner,
>
> Is this true for dictionaries specifically?
> 
> Would there really be strong objections to continuing to swallow
> any Exception (not BaseException) raised by __eq__ ?

Armin's reason for changing dictionaries in this way was that enormous
debugging pain was caused by dicts swallowing exceptions raised by
__eq__ methods.  Having the __eq__ methods swallow the exceptions by
themselves would make the situation *worse*, not better.

Cheers,
mwh

-- 
  * vegai wears his reading bra.
  <vegai> umm, I mean glasses                   -- from Twisted.Quotes

From martin at v.loewis.de  Thu Aug 10 10:35:18 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 10 Aug 2006 10:35:18 +0200
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ebd8f0$csd$1@sea.gmane.org>
References: <ebd2sp$nnv$1@sea.gmane.org> <ebd6od$6m4$1@sea.gmane.org>
	<ebd8f0$csd$1@sea.gmane.org>
Message-ID: <44DAEFC6.8000807@v.loewis.de>

Neal Becker schrieb:
> 1) Should assignment to a temporary object be allowed?

Unlike Michael, I think the question does make sense, and
the answer is "no", for the reason Michael gave: you assign
to names, not to objects.

> 2) Should the syntax for creation of a temporary object be a constructor
> call?

Now, *that* question makes no sense. Why do you need special syntax
to create a temporary object?

Regards,
Martin

From mal at egenix.com  Thu Aug 10 10:45:46 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 10 Aug 2006 10:45:46 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
Message-ID: <44DAF23A.103@egenix.com>

Guido van Rossum wrote:
> I've been happily ignoring python-dev for the last three weeks or so,
> and Neal just pointed me to some thorny issues that are close to
> resolution but not quite yet resolved, yet need to be before beta 3 on
> August 18 (Friday next week).
> 
> Here's my take on the dict-suppressing-exceptions issue (I'll send out
> separate messages for each issue where Neal has asked me to weigh in).
> 
> It wasn't my idea to stop ignoring exceptions in dict lookups; I would
> gladly have put this off until Py3k, where the main problem
> (str-unicode __eq__ raising UnicodeError) will go away.
> 
> But since people are adamant that they want this in sooner, I suggest
> that to minimize breakage we could make an exception for str-unicode
> comparisons.
> 
> I came up with the following code to reproduce the issue; this prints
> 0 in 2.2, False in 2.3 and 2.4, but raises UnicodeDecodeError in 2.5
> (head):
> 
>   a = {u"1": 1}
>   x = hash(u"1")
>   class C(str):
>       def __hash__(s): return x
>   print C("\xff") in a
> 
> The following patch makes this print False in 2.5 again.
> 
> Notes about the patch:
> 
> - this also fixes an out-of-date comment that should be fixed even if
> the rest of the idea is rejected (lookdict_string() can return NULL
> when it calls lookdict)
> 
> - the exception could be narrowed even further by only suppressing the
> exception when startkey and key are both either str or unicode
> instances.
> 
> What do people think?

I'd suggest that we still inform the programmers of the problem
by issuing a warning (which they can then silence at will),
maybe a new PyExc_UnicodeWarning.

Note that these exceptions help programmers making their applications
Unicode compatible, so silencing them completely would remove the
possibility to detect the case of mixing strings and Unicode as
keys in a dictionary.

BTW, in Py3k, this case would not trigger at all, since all text
would be Unicode and bytes wouldn't be comparable to Unicode
anyway. However, that's a different discussion which we can have
after Python 2.5 is out the door.

> --- Objects/dictobject.c	(revision 51180)
> +++ Objects/dictobject.c	(working copy)
> @@ -230,7 +230,8 @@
>  lookdict() is general-purpose, and may return NULL if (and only if) a
>  comparison raises an exception (this was new in Python 2.5).
>  lookdict_string() below is specialized to string keys, comparison of which can
> -never raise an exception; that function can never return NULL.  For both, when
> +never raise an exception; that function can never return NULL (except when it
> +decides to replace itself with the more general lookdict()).  For both, when
>  the key isn't found a dictentry* is returned for which the me_value field is
>  NULL; this is the slot in the dict at which the key would have been found, and
>  the caller can (if it wishes) add the <key, value> pair to the returned
> @@ -259,8 +260,13 @@
>  		if (ep->me_hash == hash) {
>  			startkey = ep->me_key;
>  			cmp = PyObject_RichCompareBool(startkey, key, Py_EQ);
> -			if (cmp < 0)
> -				return NULL;
> +			if (cmp < 0) {
> +				if (PyErr_Occurred()==PyExc_UnicodeDecodeError) {

                                   if (PyErr_Warn(PyExc_UnicodeWarning,
                                                  "mixing unicode and strings "
                                                  "as dictionary keys") < 0)
                                        return NULL;

> +				   PyErr_Clear();
> +				}
> +				else
> +					return NULL;
> +                        }
>  			if (ep0 == mp->ma_table && ep->me_key == startkey) {
>  				if (cmp > 0)
>  					return ep;
> @@ -289,8 +295,13 @@
>  		if (ep->me_hash == hash && ep->me_key != dummy) {
>  			startkey = ep->me_key;
>  			cmp = PyObject_RichCompareBool(startkey, key, Py_EQ);
> -			if (cmp < 0)
> -				return NULL;
> +			if (cmp < 0) {
> +				if (PyErr_Occurred()==PyExc_UnicodeDecodeError) {
> +					PyErr_Clear();
> +				}
> +				else
> +					return NULL;
> +                        }
>  			if (ep0 == mp->ma_table && ep->me_key == startkey) {
>  				if (cmp > 0)
>  					return ep;
> 
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 10 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Thu Aug 10 11:10:42 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 10 Aug 2006 11:10:42 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <44D8DF4A.70408@v.loewis.de>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>	<eavofa$n2f$1@sea.gmane.org>	<44D36B36.8030202@egenix.com>	<44D73964.3050402@v.loewis.de>	<44D83C78.4070008@egenix.com>	<44D843C5.1060907@v.loewis.de>
	<44D8A8E2.5090002@egenix.com> <44D8DF4A.70408@v.loewis.de>
Message-ID: <44DAF812.3080603@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg schrieb:
>> Note that we are not discussing changing the behavior of the
>> __eq__ comparison between strings and Unicode, since this has
>> always been to raise exceptions in case the automatic propagation
>> fails.
> 
> Not sure what you are discussing: This is *precisely* what I'm
> discussing. Making that change would solve this problem.

... and introduce a whole set of new problems, namely finding
all the places in your code that are not Unicode compatible.

Instead of reverting the patch and continue to silencing exceptions
in the dict lookup, you are proposing to extend the behavior to all
equal comparisons of strings and Unicode. That would be a major
change in the semantics, not just a bug fix as you seem to imply.

Since the problem of mixing strings and Unicode will go away
in Py3k anyway, I suggest we do everything to help programmers
make their applications Unicode compatible instead of trying
to hide all problem cases from them.

Note that all of these problem go away if you stick to Unicode
for all text data in your application. This will also make
porting the application to Py3k a lot easier.

>
>> The discussion is about silencing exceptions in the dict lookup
>> mechanism - something which used to happen and now no longer
>> is done.

Regards,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 10 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From ncoghlan at gmail.com  Thu Aug 10 14:13:05 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 10 Aug 2006 22:13:05 +1000
Subject: [Python-Dev] Is this a bug?
In-Reply-To: <20060809084110.E71E.JCARLSON@uci.edu>
References: <ebcs51$tgo$1@sea.gmane.org> <20060809084110.E71E.JCARLSON@uci.edu>
Message-ID: <44DB22D1.60002@gmail.com>

Josiah Carlson wrote:
> This is the case for Python 2.3, 2.4, and 2.5 beta.  prefixing the above
> two operations with:
>     sys.modules['_oldmain'] = sys.modules['__main__']
> 
> Is sufficient to prevent Python from tearing down everything after the
> sys.modules['__main__'] reassignment.  Not a big deal, but a sys.modules
> manipulation that has a gotcha.

Early versions of the PEP 338 implementation also ran into the problem of a 
module's globals getting cleared when the module itself goes away - that's why 
runpy makes sure to return a copy of the module's global namespace instead of 
the original.

The core of the problem in the atexit case seems to be the fact that holding 
on to a reference to a function from a module only holds a reference to the 
module's dict, rather than the module object itself.

If you kill the reference to the module from sys.modules, the module object 
goes away, and the dict gets cleared. However, any function objects being held 
elsewhere (e.g. in the atexit list, or as a result of a class being subclassed 
in a different module) still have a reference to the now defunct module 
namespace, and will misbehave when called later on if they attempt to access 
any global variables.

Are we really sure that explicitly clearing the module's globals when the 
module's refcount falls to zero solves more problems than it causes? Or is it 
primarily a legacy of the pre-cyclic GC days when it was absolutely essential 
to manually break the cycle between the module namespace and any functions it 
contained?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ralf at brainbot.com  Thu Aug 10 14:53:39 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 10 Aug 2006 14:53:39 +0200
Subject: [Python-Dev] SimpleXMLWriter missing from elementtree
Message-ID: <ebfa81$3kp$1@sea.gmane.org>

Hi,

any chance to get SimpleXMLWriter (and other modules maybe??) getting 
included into xml.etree? Otherwise people might have to stick to the 
original elementtree, which doesn't really make sense, since most of 
elementtree already is included.

- Ralf


From ralf at brainbot.com  Thu Aug 10 14:59:25 2006
From: ralf at brainbot.com (Ralf Schmitt)
Date: Thu, 10 Aug 2006 14:59:25 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
Message-ID: <44DB2DAD.1060906@brainbot.com>

Guido van Rossum wrote:
> 
> - the exception could be narrowed even further by only suppressing the
> exception when startkey and key are both either str or unicode
> instances.
> 

I always assumed dictionaries would work exactly like this. So, at least 
it would now work as I had always expected (and others possibly too).

- Ralf


From murman at gmail.com  Thu Aug 10 15:48:43 2006
From: murman at gmail.com (Michael Urman)
Date: Thu, 10 Aug 2006 08:48:43 -0500
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <44DAF23A.103@egenix.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
	<44DAF23A.103@egenix.com>
Message-ID: <dcbbbb410608100648u1b30e1c4md33c9ef5b085bd1d@mail.gmail.com>

On 8/10/06, M.-A. Lemburg <mal at egenix.com> wrote:
> Guido van Rossum wrote:
> > But since people are adamant that they want this in sooner, I suggest
> > that to minimize breakage we could make an exception for str-unicode
> > comparisons.
> > What do people think?
>
> I'd suggest that we still inform the programmers of the problem
> by issuing a warning (which they can then silence at will),
> maybe a new PyExc_UnicodeWarning.
>
> BTW, in Py3k, this case would not trigger at all, since all text
> would be Unicode and bytes wouldn't be comparable to Unicode
> anyway. However, that's a different discussion which we can have
> after Python 2.5 is out the door.

I strongly believe that unicode vs str here is the symptom and not the
actual problem. The comparison between two non-us-ascii str/unicode
instances is but one of many ways to raise an exception during
comparison. It's not even obvious ahead of time when it will occur.
Try my example below with (sys.maxint << 1) and -2 instead of 1 and 1.
Do you expect a problem?

Because str/unicode is the common case, we saw it first. If we address
the symptom instead of the problem, someone will be complaining within
a years time because they have a class that they mix in with other
items for a function handler lookup, or who knows what, that works
like the following:

>>> class hasher(object):
...   def __init__(self, hashval):
...     self.hashval = hashval
...   def __hash__(self):
...     return hash(self.hashval)
...   def __eq__(self, o):
...     if not isinstance(o, hasher):
...       raise TypeError("Cannot compare hashval to non hashval")
...     return self.hashval == o.hashval

in python2.4:
>>> dict.fromkeys([1, hasher(1)])
{1: None, <__main__.hasher object at 0xa7a5326c>: None}

in python2.5b2:
>>> dict.fromkeys([1, hasher(1)])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 8, in __eq__
TypeError: Cannot compare hashval to non hashval

Yes this is made up code. But I'm not arguing for a feature; I'm
arguing for backwards compatibility. Because we do not know where
these legitimate uses are, we cannot evaluate their likelihood to
exist nor the level of breakage they will cause. If we make this a
warning for any exception, we can satisfy both imagined camps. Those
in Armin's position can make that warning raise an exception while
debugging, and those using it on purpose can squash it.

I understand the utility of being able to see this case happening. I'm
not sure it's worth the warning. I'm positive it's not worth the
backwards incompatibility of the raised exception.

Michael
-- 
Michael Urman  http://www.tortall.net/mu/blog

From murman at gmail.com  Thu Aug 10 15:52:48 2006
From: murman at gmail.com (Michael Urman)
Date: Thu, 10 Aug 2006 08:52:48 -0500
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <2m4pwlvezn.fsf@starship.python.net>
References: <ebd2sp$nnv$1@sea.gmane.org> <ebd6od$6m4$1@sea.gmane.org>
	<ebd8f0$csd$1@sea.gmane.org> <2m4pwlvezn.fsf@starship.python.net>
Message-ID: <dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>

On 8/9/06, Michael Hudson <mwh at python.net> wrote:
> The question doesn't make sense: in Python, you assign to a name,
> an attribute or a subscript, and that's it.

Just to play devil's advocate here, why not to a function call via a
new __setcall__? I'm not saying there's the use case to justify it,
but I don't see anything that makes it a clear abomination or
impossible with python's syntax.

Michael
-- 
Michael Urman  http://www.tortall.net/mu/blog

From ncoghlan at gmail.com  Thu Aug 10 16:18:11 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 11 Aug 2006 00:18:11 +1000
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>	<ebe6lk$1tk$1@sea.gmane.org>
	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>
Message-ID: <44DB4023.5090904@gmail.com>

Guido van Rossum wrote:
>> It seems like Nick's recent patches solved the problems that were
>> identified.
> 
> Nick, can you summarize how your patches differ from my proposal?

nb_index and __index__ are essentially exactly as you propose. To make an 
object implemented in C usable as an index you would take either the nb_int 
slot or the nb_long slot and put the same function pointer into the nb_index 
slot. For a Python object, you would write either '__index__ = __int__' or 
'__index__ = __long__' as part of the class definition.

operator.index is provided to support writing __getitem__, __setitem__ and 
__delitem__ methods - it raises IndexError on overflow so you don't have to 
catch and reraise to convert an OverflowError to an IndexError.

On the C API side, the 3 functions you suggest are all present (although the 
version returning a Python object is accessed via PyObject_CallMethod), and 
there's a 4th variant that raises IndexError instead of OverflowError (this 
version is convenient when writing mp_subscript and mp_ass_subscript functions).

Avoiding Py_ssize_t -> PyInt -> Py_ssize_t conversions for all integer types 
implemented in C would be nice, but I don't think it's practical (the latest 
version of the patch does at least avoid it for the builtin integer types).

Cheers,
Nick.



P.S. Here's the detailed rationale for the form the patch has evolved to [1]:

In addition to allowing (2**100).__index__() == 2**100, having nb_index return 
a Python object resulted in a decent reduction in code duplication - 
previously the coercion logic to get a Python integer or long value down to a 
Py_ssize_t was present in 3 places (long_index, instance_index, 
slot_nb_index), and would also have needed to be duplicated by any other C 
implemented index type whose value could exceed the range of a Py_ssize_t. 
With the patch, that logic appears only inside abstract.c and extension types 
can just return a PyLong value and let the interpreter figure out how to 
handle overflow. The biggest benefit of this approach is that a single slot 
(nb_index) can be used to implement four different overflow behaviours in the 
core (return PyLong, raise OverflowError, raise IndexError, clip to 
Py_ssize_t), as well as providing a hook to allow extension module authors to 
define their own overflow handling.

If the nb_index slot does not return a true Python integer or long, TypeError 
gets raised. Subclasses are not accepted in order to rule out Armin's 
favourite set of recursion problems :)

The C level API is based on the use cases in the standard library, with one of 
the functions generalised a bit to allow extension modules to easily handle 
type errors and overflow differently if they want to.

The three different use cases for nb_index in the standard library are:
   - concrete sequence indices (want IndexError on overflow)
   - 'true integer' retrieval (want OverflowError on overflow)
   - slice endpoints (want to clip to Py_ssize_t max/min values)

The proposed fix (Travis & Neal provided some useful comments on earlier 
versions) includes a C API function for each of these different use cases:

   PyNumber_Index(PyObject *obj, int *type_err)
   PyNumber_AsSsize_t(PyObject *obj, int *type_err)
   PyNumber_AsClippedSsize_t(PyObject *obj, int *type_err, int *clipped)

type_err is an output variable to say "obj does not provide nb_index" in order 
to get rid of boilerplate dealing with PyErr_Occurred() in mp_subscript and 
mp_ass_subscript implementations (those methods generally didn't want a 
TypeError raised at this point - they wanted to go on and check if the object 
was a slice object instead). It's also useful if you want to provide a 
specific error message for TypeErrors (sequence repetition takes advantage of 
this). You can also leave the pointer as NULL and the functions will raise a 
fairly generic TypeError for you. PyObject_GetItem and friends, use the 
functions that way.

Avoiding repeated code is also why there are two non-clipping variants, one 
raising IndexError and one raising OverflowError. Raising OverflowError in 
PyNumber_Index broke half a dozen unit tests, while raising IndexError for 
things like sequence repetition turned out to break different unit tests.

The clipping variant is for slice indices. The interpreter core doesn't 
actually care whether or not the result gets clipped in this case (it sets the 
last parameter to NULL), but I kept the output variable in the signature for 
the benefit of extension authors.

All 3 of the C API methods return Py_ssize_t. The "give me a Python object" 
case isn't actually needed anywhere in the core, but is available to extension 
modules via:
   PyObject_CallMethod(obj, "__index__", NULL)

As Travis notes, indexing with something other than a builtin integer will be 
slightly slower due to the temporary object created by calling the nb_index 
slot (version 4 of the patch avoids this overhead for ints, version 5 avoids 
it for longs as well). I don't think this is avoidable - a non-PyObject return 
value really doesn't provide the necessary flexibility to detect and handle 
overflow correctly.

[1] http://www.python.org/sf/1530738

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Thu Aug 10 16:26:54 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 07:26:54 -0700
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <44DB4023.5090904@gmail.com>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>
	<ebe6lk$1tk$1@sea.gmane.org>
	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>
	<44DB4023.5090904@gmail.com>
Message-ID: <ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>

On 8/10/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
> >> It seems like Nick's recent patches solved the problems that were
> >> identified.
> >
> > Nick, can you summarize how your patches differ from my proposal?
>
> nb_index and __index__ are essentially exactly as you propose.

Then I don't understand why Travis is objecting against my proposal!

I'll review the rest later (right now I'm just doing email triage :-).

--Guido

> To make an
> object implemented in C usable as an index you would take either the nb_int
> slot or the nb_long slot and put the same function pointer into the nb_index
> slot. For a Python object, you would write either '__index__ = __int__' or
> '__index__ = __long__' as part of the class definition.
>
> operator.index is provided to support writing __getitem__, __setitem__ and
> __delitem__ methods - it raises IndexError on overflow so you don't have to
> catch and reraise to convert an OverflowError to an IndexError.
>
> On the C API side, the 3 functions you suggest are all present (although the
> version returning a Python object is accessed via PyObject_CallMethod), and
> there's a 4th variant that raises IndexError instead of OverflowError (this
> version is convenient when writing mp_subscript and mp_ass_subscript functions).
>
> Avoiding Py_ssize_t -> PyInt -> Py_ssize_t conversions for all integer types
> implemented in C would be nice, but I don't think it's practical (the latest
> version of the patch does at least avoid it for the builtin integer types).
>
> Cheers,
> Nick.
>
>
>
> P.S. Here's the detailed rationale for the form the patch has evolved to [1]:
>
> In addition to allowing (2**100).__index__() == 2**100, having nb_index return
> a Python object resulted in a decent reduction in code duplication -
> previously the coercion logic to get a Python integer or long value down to a
> Py_ssize_t was present in 3 places (long_index, instance_index,
> slot_nb_index), and would also have needed to be duplicated by any other C
> implemented index type whose value could exceed the range of a Py_ssize_t.
> With the patch, that logic appears only inside abstract.c and extension types
> can just return a PyLong value and let the interpreter figure out how to
> handle overflow. The biggest benefit of this approach is that a single slot
> (nb_index) can be used to implement four different overflow behaviours in the
> core (return PyLong, raise OverflowError, raise IndexError, clip to
> Py_ssize_t), as well as providing a hook to allow extension module authors to
> define their own overflow handling.
>
> If the nb_index slot does not return a true Python integer or long, TypeError
> gets raised. Subclasses are not accepted in order to rule out Armin's
> favourite set of recursion problems :)
>
> The C level API is based on the use cases in the standard library, with one of
> the functions generalised a bit to allow extension modules to easily handle
> type errors and overflow differently if they want to.
>
> The three different use cases for nb_index in the standard library are:
>    - concrete sequence indices (want IndexError on overflow)
>    - 'true integer' retrieval (want OverflowError on overflow)
>    - slice endpoints (want to clip to Py_ssize_t max/min values)
>
> The proposed fix (Travis & Neal provided some useful comments on earlier
> versions) includes a C API function for each of these different use cases:
>
>    PyNumber_Index(PyObject *obj, int *type_err)
>    PyNumber_AsSsize_t(PyObject *obj, int *type_err)
>    PyNumber_AsClippedSsize_t(PyObject *obj, int *type_err, int *clipped)
>
> type_err is an output variable to say "obj does not provide nb_index" in order
> to get rid of boilerplate dealing with PyErr_Occurred() in mp_subscript and
> mp_ass_subscript implementations (those methods generally didn't want a
> TypeError raised at this point - they wanted to go on and check if the object
> was a slice object instead). It's also useful if you want to provide a
> specific error message for TypeErrors (sequence repetition takes advantage of
> this). You can also leave the pointer as NULL and the functions will raise a
> fairly generic TypeError for you. PyObject_GetItem and friends, use the
> functions that way.
>
> Avoiding repeated code is also why there are two non-clipping variants, one
> raising IndexError and one raising OverflowError. Raising OverflowError in
> PyNumber_Index broke half a dozen unit tests, while raising IndexError for
> things like sequence repetition turned out to break different unit tests.
>
> The clipping variant is for slice indices. The interpreter core doesn't
> actually care whether or not the result gets clipped in this case (it sets the
> last parameter to NULL), but I kept the output variable in the signature for
> the benefit of extension authors.
>
> All 3 of the C API methods return Py_ssize_t. The "give me a Python object"
> case isn't actually needed anywhere in the core, but is available to extension
> modules via:
>    PyObject_CallMethod(obj, "__index__", NULL)
>
> As Travis notes, indexing with something other than a builtin integer will be
> slightly slower due to the temporary object created by calling the nb_index
> slot (version 4 of the patch avoids this overhead for ints, version 5 avoids
> it for longs as well). I don't think this is avoidable - a non-PyObject return
> value really doesn't provide the necessary flexibility to detect and handle
> overflow correctly.
>
> [1] http://www.python.org/sf/1530738
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>              http://www.boredomandlaziness.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Aug 10 16:32:29 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 07:32:29 -0700
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <dcbbbb410608100648u1b30e1c4md33c9ef5b085bd1d@mail.gmail.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
	<44DAF23A.103@egenix.com>
	<dcbbbb410608100648u1b30e1c4md33c9ef5b085bd1d@mail.gmail.com>
Message-ID: <ca471dc20608100732y571fd72cp72c8b8edce121e40@mail.gmail.com>

On 8/10/06, Michael Urman <murman at gmail.com> wrote:
> I strongly believe that unicode vs str here is the symptom and not the
> actual problem.

No. Everywhere when __eq__ fails, we can safely tell the user that
it's a bug in their __eq__ that they should fix (maybe by making it
return False when the other object isn't a type they recognize). But
we can't fix unicode-vs-str comparison without breaking too much code.
So we have to somehow deal with that without changing the behavior of
that particular __eq__.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Aug 10 16:36:29 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 07:36:29 -0700
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <44DAF23A.103@egenix.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
	<44DAF23A.103@egenix.com>
Message-ID: <ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>

On 8/10/06, M.-A. Lemburg <mal at egenix.com> wrote:
> I'd suggest that we still inform the programmers of the problem
> by issuing a warning (which they can then silence at will),
> maybe a new PyExc_UnicodeWarning.

Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to
issue a warning (and return False) instead of raising
UnicodeException? That won't break much code (it's unlikely that
people *depend* on this exception since it's generally a symptom of
insane mixing of str and unicode). Then no further changes to
dictobject.c are necessary (except fixing that one misleading
comment).

> Note that these exceptions help programmers making their applications
> Unicode compatible, so silencing them completely would remove the
> possibility to detect the case of mixing strings and Unicode as
> keys in a dictionary.

A warning would arguably have the same helping effect. (I suspect
actually that we would have used warnings all along except we didn't
have the warning framework when unicode was first introduced.)

> BTW, in Py3k, this case would not trigger at all, since all text
> would be Unicode and bytes wouldn't be comparable to Unicode
> anyway. However, that's a different discussion which we can have
> after Python 2.5 is out the door.

Which is why I said I would have gladly waited until Py3k to fix this.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Aug 10 16:39:10 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 11 Aug 2006 00:39:10 +1000
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>	
	<ebe6lk$1tk$1@sea.gmane.org>	
	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>	
	<44DB4023.5090904@gmail.com>
	<ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>
Message-ID: <44DB450E.7050002@gmail.com>

Guido van Rossum wrote:
> On 8/10/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Guido van Rossum wrote:
>> >> It seems like Nick's recent patches solved the problems that were
>> >> identified.
>> >
>> > Nick, can you summarize how your patches differ from my proposal?
>>
>> nb_index and __index__ are essentially exactly as you propose.
> 
> Then I don't understand why Travis is objecting against my proposal!

I must admit to being confused by that, too. . .

Cheers,
Nick.

P.S. Just to mess up the release schedule some more, I'm going to be away from 
13-19 August, so the patch either needs to be approved by Neal or Anthony in 
the next few days, or someone else will need to commit it before rc1.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mal at egenix.com  Thu Aug 10 17:05:27 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 10 Aug 2006 17:05:27 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>	<44DAF23A.103@egenix.com>
	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>
Message-ID: <44DB4B37.1000401@egenix.com>

Guido van Rossum wrote:
> On 8/10/06, M.-A. Lemburg <mal at egenix.com> wrote:
>> I'd suggest that we still inform the programmers of the problem
>> by issuing a warning (which they can then silence at will),
>> maybe a new PyExc_UnicodeWarning.
> 
> Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to
> issue a warning (and return False) instead of raising
> UnicodeException? That won't break much code (it's unlikely that
> people *depend* on this exception since it's generally a symptom of
> insane mixing of str and unicode). Then no further changes to
> dictobject.c are necessary (except fixing that one misleading
> comment).

Good idea.

>> Note that these exceptions help programmers making their applications
>> Unicode compatible, so silencing them completely would remove the
>> possibility to detect the case of mixing strings and Unicode as
>> keys in a dictionary.
> 
> A warning would arguably have the same helping effect. (I suspect
> actually that we would have used warnings all along except we didn't
> have the warning framework when unicode was first introduced.)

Probably, yes.

These days, I believe that we should have not gone for the
string-Unicode integration magic at all and instead done the
same of what's planned for Py3k. Sometimes a clean cut is better
than trying to muddle along.

>> BTW, in Py3k, this case would not trigger at all, since all text
>> would be Unicode and bytes wouldn't be comparable to Unicode
>> anyway. However, that's a different discussion which we can have
>> after Python 2.5 is out the door.
> 
> Which is why I said I would have gladly waited until Py3k to fix this.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 10 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From jcarlson at uci.edu  Thu Aug 10 18:01:47 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 10 Aug 2006 09:01:47 -0700
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
Message-ID: <20060810084217.1914.JCARLSON@uci.edu>


"Michael Urman" <murman at gmail.com> wrote:
> 
> On 8/9/06, Michael Hudson <mwh at python.net> wrote:
> > The question doesn't make sense: in Python, you assign to a name,
> > an attribute or a subscript, and that's it.
> 
> Just to play devil's advocate here, why not to a function call via a
> new __setcall__? I'm not saying there's the use case to justify it,
> but I don't see anything that makes it a clear abomination or
> impossible with python's syntax.

Describe the syntax and semantics.  Every time I try to work them out, I
end up with a construct that makes less than no sense, to be used in
cases I have never seen. Further, if you want to call a method
__setcall__ on an object just created, you can use 'x().__setcall__(y)'.
There is no reason to muck up Python's syntax.


 - Josiah


From foom at fuhm.net  Thu Aug 10 18:19:22 2006
From: foom at fuhm.net (James Y Knight)
Date: Thu, 10 Aug 2006 12:19:22 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <20060810084217.1914.JCARLSON@uci.edu>
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
Message-ID: <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>

On Aug 10, 2006, at 12:01 PM, Josiah Carlson wrote:

>
> "Michael Urman" <murman at gmail.com> wrote:
>>
>> On 8/9/06, Michael Hudson <mwh at python.net> wrote:
>>> The question doesn't make sense: in Python, you assign to a name,
>>> an attribute or a subscript, and that's it.
>>
>> Just to play devil's advocate here, why not to a function call via a
>> new __setcall__? I'm not saying there's the use case to justify it,
>> but I don't see anything that makes it a clear abomination or
>> impossible with python's syntax.
>
> Describe the syntax and semantics.  Every time I try to work them  
> out, I
> end up with a construct that makes less than no sense, to be used in
> cases I have never seen. Further, if you want to call a method
> __setcall__ on an object just created, you can use 'x().__setcall__ 
> (y)'.
> There is no reason to muck up Python's syntax.

It makes just as much sense as assigning to an array access, and the  
semantics would be pretty similar. There's similarly "no reason" to  
allow x[5] = True. You can just spell that x.__setitem__(5, True).

x(*args, **kwargs) = val could translate into x.__setcall__(val,  
*args, **kwargs).
x(5) = True could translate into x.__setcall__(True, 5)

Please note I'm actually arguing for this proposal. Just agreeing  
that it is not a completely nonsensical idea.

James

From guido at python.org  Thu Aug 10 18:24:39 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 09:24:39 -0700
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
Message-ID: <ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>

On 8/10/06, James Y Knight <foom at fuhm.net> wrote:
> It makes just as much sense as assigning to an array access, and the
> semantics would be pretty similar.

No. Array references (x[i]) and attribute references (x.a) represent
"locations". Function calls represent values. This is no different
than the distinction between lvalues and rvalues in C.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Aug 10 18:31:23 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 10 Aug 2006 12:31:23 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.co
 m>
References: <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
Message-ID: <5.1.1.6.0.20060810122651.0262dd18@sparrow.telecommunity.com>

At 09:24 AM 8/10/2006 -0700, Guido van Rossum wrote:
>On 8/10/06, James Y Knight <foom at fuhm.net> wrote:
> > It makes just as much sense as assigning to an array access, and the
> > semantics would be pretty similar.
>
>No. Array references (x[i]) and attribute references (x.a) represent
>"locations". Function calls represent values. This is no different
>than the distinction between lvalues and rvalues in C.

IIRC, in Lisp a function call can be an lvalue.  OTOH, that's probably 
because in Lisp there isn't anything besides function calls.  :)

I'm having a hard time imagining a use case in Python, though, except to 
allow things like "somedict.setdefault(key,0) += 1".  And I'm not sure 
that's an improvement, although there have certainly been times that I 
started to write that, and then grumbled and wrote "somedict[key] = 
somedict.setdefault(key,0) + 1" instead.

Honestly, it might make more sense to get rid of augmented assignment in 
Py3K rather than to add this.  It seems that the need for something like 
this springs primarily from the existence of augmented assignment.


From guido at python.org  Thu Aug 10 18:33:50 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 09:33:50 -0700
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <44DB4B37.1000401@egenix.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
	<44DAF23A.103@egenix.com>
	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>
	<44DB4B37.1000401@egenix.com>
Message-ID: <ca471dc20608100933p2ee25e0dl5558bc69766c528a@mail.gmail.com>

On 8/10/06, M.-A. Lemburg <mal at egenix.com> wrote:
> Guido van Rossum wrote:
> > Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to
> > issue a warning (and return False) instead of raising
> > UnicodeException? That won't break much code (it's unlikely that
> > people *depend* on this exception since it's generally a symptom of
> > insane mixing of str and unicode). Then no further changes to
> > dictobject.c are necessary (except fixing that one misleading
> > comment).
>
> Good idea.

Great! Now we need someone to volunteer to write a patch (which should
include doc and NEWS updates) in time for beta 3 (Aug 18).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From foom at fuhm.net  Thu Aug 10 18:49:04 2006
From: foom at fuhm.net (James Y Knight)
Date: Thu, 10 Aug 2006 12:49:04 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
Message-ID: <14E22ABA-4AE4-4C35-B3B4-D6C7EE8D528E@fuhm.net>

On Aug 10, 2006, at 12:19 PM, James Y Knight wrote:
> Please note I'm actually arguing for this proposal. Just agreeing
> that it is not a completely nonsensical idea.

ERK! Big typo there. I meant to say:

Please note I'm ****NOT*** actually arguing for this proposal.

Sorry for any confusion.

James

From guido at python.org  Thu Aug 10 20:14:14 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 11:14:14 -0700
Subject: [Python-Dev] cgi DoS attack
In-Reply-To: <ca471dc20608091458i4b17540cnd0c8e674ab1846d6@mail.gmail.com>
References: <ca471dc20608091458i4b17540cnd0c8e674ab1846d6@mail.gmail.com>
Message-ID: <ca471dc20608101114s3e47a196me50d3c66cc99915f@mail.gmail.com>

FWIW, I've checked this in (while stuck in a meeting, Neal will know
what I mean :-).

svn r51190-r51192/

On 8/9/06, Guido van Rossum <guido at python.org> wrote:
> This is regarding #1112549. I think this can go in. It should also be
> backported to 2.4 and to 2.3 (if we ever release another one of that).
> I reviewed the code and added some minor comments to the SF tracker.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jcarlson at uci.edu  Thu Aug 10 20:22:03 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 10 Aug 2006 11:22:03 -0700
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <5.1.1.6.0.20060810122651.0262dd18@sparrow.telecommunity.com>
References: <ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.co m>
	<5.1.1.6.0.20060810122651.0262dd18@sparrow.telecommunity.com>
Message-ID: <20060810111304.191A.JCARLSON@uci.edu>


"Phillip J. Eby" <pje at telecommunity.com> wrote:
> At 09:24 AM 8/10/2006 -0700, Guido van Rossum wrote:
> >On 8/10/06, James Y Knight <foom at fuhm.net> wrote:
> > > It makes just as much sense as assigning to an array access, and the
> > > semantics would be pretty similar.
> >
> >No. Array references (x[i]) and attribute references (x.a) represent
> >"locations". Function calls represent values. This is no different
> >than the distinction between lvalues and rvalues in C.
> 
> I'm having a hard time imagining a use case in Python, though, except to 
> allow things like "somedict.setdefault(key,0) += 1".  And I'm not sure 
> that's an improvement, although there have certainly been times that I 
> started to write that, and then grumbled and wrote "somedict[key] = 
> somedict.setdefault(key,0) + 1" instead.

Thankfully default dicts in Python 2.5 (they are in, right?) should
solve this particular nit of the language.  And by using default dicts
as an instance __dict__, you get similar behavior for attributes.


> Honestly, it might make more sense to get rid of augmented assignment in 
> Py3K rather than to add this.  It seems that the need for something like 
> this springs primarily from the existence of augmented assignment.

I'm not sure that your characterization is entirely accurate.  I've
used and seen augmented assignments being used for many years now, and
this is the first time that I've seen anyone propose that it should work.


 - Josiah


From oliphant.travis at ieee.org  Thu Aug 10 20:25:00 2006
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Thu, 10 Aug 2006 12:25:00 -0600
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>	<ebe6lk$1tk$1@sea.gmane.org>	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>	<44DB4023.5090904@gmail.com>
	<ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>
Message-ID: <ebftlt$buf$1@sea.gmane.org>

Guido van Rossum wrote:
> On 8/10/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Guido van Rossum wrote:
>>>> It seems like Nick's recent patches solved the problems that were
>>>> identified.
>>> Nick, can you summarize how your patches differ from my proposal?
>> nb_index and __index__ are essentially exactly as you propose.
> 
> Then I don't understand why Travis is objecting against my proposal!

I must have missed his most recent patch that changed the result to 
return a Python object.  I thought earlier versions didn't do that.

My objection is not particularly solid.  At this point it's largely a 
wish to avoid the extra overhead (there are some camps that already 
complain about NumPy having too much indexing overhead --- although they 
should be using Python lists for there purposes if indexing overhead 
really is a problem).

As it appears that several people feel this is the best way forward, 
then I'll re-work my NumPy code.   I still appreciate the change that 
allows other Python objects to be integers and eliminates the "only true 
integers allowed" flavor of several places in the Python code.

Thanks for all your hard work.

-Travis


From foom at fuhm.net  Thu Aug 10 20:28:20 2006
From: foom at fuhm.net (James Y Knight)
Date: Thu, 10 Aug 2006 14:28:20 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
Message-ID: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>

On Aug 10, 2006, at 12:24 PM, Guido van Rossum wrote:
> On 8/10/06, James Y Knight <foom at fuhm.net> wrote:
>> It makes just as much sense as assigning to an array access, and the
>> semantics would be pretty similar.
>
> No. Array references (x[i]) and attribute references (x.a) represent
> "locations". Function calls represent values. This is no different
> than the distinction between lvalues and rvalues in C.

Yes, function calls cannot be lvalues right now. However, there is no  
reason that a function call _could not_ be an lvalue. That is exactly  
what the addition of __setcall__  would allow.


On Aug 10, 2006, at 12:31 PM, Phillip J. Eby wrote:
> Honestly, it might make more sense to get rid of augmented  
> assignment in Py3K rather than to add this.  It seems that the need  
> for something like this springs primarily from the existence of  
> augmented assignment.

It makes just as much (and just as little) sense to have normal  
assignment to function calls as it does augmented assignment to  
function calls. I don't see any reason to single out augmented  
assignment here.

Anyhow, enough time wasted on this. I don't really think python  
should add this feature, but it _does_ make sense, and would have  
understandable and consistent semantics if it were added.

James

From armin.ronacher at active-4.com  Thu Aug 10 20:37:17 2006
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Thu, 10 Aug 2006 18:37:17 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?Unicode_Data_in_Python2=2E5_is_missing_a_u?=
	=?utf-8?b?Y2RfNF8xXzAgb2JqZWN0?=
Message-ID: <loom.20060810T202810-913@post.gmane.org>

Hi,

I discovered that unicodedata in python2.5 implements unicode 4.1. While
this is ok it's possible enforce unicode 3.2 by using the ucd_3_2_0 object.
But it's not possible to enforce a ucd_4_1_0 standard because that object
does not exist by now.

In the description of #1031288 (http://www.python.org/sf/1031288) Martin v.
L?wis says the following:

| Python relies on the unicodedata 3.2.0, as the IDNA RFCs
| mandate that Unicode 3.2 is used to implement IDNA. So any
| integration of 4.0.1 must
| a) still maintain access to the 3.2.0 data

And furthermore the docstring claims that this module just implements unicode
3.2.0 whereas unidata_version gives me 4.1.0

Doesn't that mean that there should also be an way to enforce unicode 4.1.0?

Regards,
Armin


From martin at v.loewis.de  Thu Aug 10 21:05:46 2006
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 10 Aug 2006 21:05:46 +0200
Subject: [Python-Dev] Unicode Data in Python2.5 is missing a ucd_4_1_0
 object
In-Reply-To: <loom.20060810T202810-913@post.gmane.org>
References: <loom.20060810T202810-913@post.gmane.org>
Message-ID: <44DB838A.10600@v.loewis.de>

Armin Ronacher schrieb:
> I discovered that unicodedata in python2.5 implements unicode 4.1. While
> this is ok it's possible enforce unicode 3.2 by using the ucd_3_2_0 object.
> But it's not possible to enforce a ucd_4_1_0 standard because that object
> does not exist by now.

Not sure what you mean by "enforce". Isn't

assert unicodedata.unidata_version == '4.1.0'

enough enforcement?

> And furthermore the docstring claims that this module just implements unicode
> 3.2.0 whereas unidata_version gives me 4.1.0

That's a bug; thanks for pointing it out. I just fixed it in r51194.

> Doesn't that mean that there should also be an way to enforce unicode 4.1.0?

You mean, that there should be a ucd_4_1_0 object? No, why do you think
there should be one? We don't plan to provide a copy of the UCD for each
UCD version that was ever released (e.g. we skipped some between 3.2 and
4.1 also).

Regards,
Martin

From martin at v.loewis.de  Thu Aug 10 21:06:57 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 10 Aug 2006 21:06:57 +0200
Subject: [Python-Dev] SimpleXMLWriter missing from elementtree
In-Reply-To: <ebfa81$3kp$1@sea.gmane.org>
References: <ebfa81$3kp$1@sea.gmane.org>
Message-ID: <44DB83D1.6040009@v.loewis.de>

Ralf Schmitt schrieb:
> any chance to get SimpleXMLWriter (and other modules maybe??) getting 
> included into xml.etree? 

Not in Python 2.5, no.

Regards,
Martin

From martin at v.loewis.de  Thu Aug 10 21:11:42 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 10 Aug 2006 21:11:42 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>	<44DAF23A.103@egenix.com>
	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>
Message-ID: <44DB84EE.8020606@v.loewis.de>

Guido van Rossum schrieb:
> Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to
> issue a warning (and return False) instead of raising
> UnicodeException? 

I'm in favour of having this __eq__ just return False. I don't think
the warning is necessary, and believe that people will complain about
getting it, but if that is the only way to achieve consensus,
so be it.

> A warning would arguably have the same helping effect. (I suspect
> actually that we would have used warnings all along except we didn't
> have the warning framework when unicode was first introduced.)

I believe we would have neither had we rich comparisons available
at the time. That unicode-vs-str-__eq__ raises an exception is just
a fall-out of not supporting __cmp__ (and it is good that __cmp__
is not supported - there is no meaningful way to order str/unicode
if the string is not in the system encoding).

Regards,
Martin

From martin at v.loewis.de  Thu Aug 10 21:13:22 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 10 Aug 2006 21:13:22 +0200
Subject: [Python-Dev] returning longs from __hash__()
In-Reply-To: <20060809094725.GA16199@code0.codespeak.net>
References: <20060808102608.GA9098@code0.codespeak.net>	<44D8E148.5050701@v.loewis.de>	<1f7befae0608081320j61bacd63l8ea8d3a9f32f0424@mail.gmail.com>	<44D8FED0.9060108@v.loewis.de>
	<20060809094725.GA16199@code0.codespeak.net>
Message-ID: <44DB8552.5030505@v.loewis.de>

Armin Rigo schrieb:
> This bug will keep showing up forever :-)  It's unsafe against a user
> subclassing 'long' and overriding __hash__ of that subclass to return
> the object itself -- it would cause an infinite C recursion.

I see you've fixed it - thanks.

Martin

From guido at python.org  Thu Aug 10 21:28:08 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 12:28:08 -0700
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
	<1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
Message-ID: <ca471dc20608101228m5769e191h779ab5ad36c27a5a@mail.gmail.com>

On 8/10/06, James Y Knight <foom at fuhm.net> wrote:
> On Aug 10, 2006, at 12:24 PM, Guido van Rossum wrote:
> > On 8/10/06, James Y Knight <foom at fuhm.net> wrote:
> >> It makes just as much sense as assigning to an array access, and the
> >> semantics would be pretty similar.
> >
> > No. Array references (x[i]) and attribute references (x.a) represent
> > "locations". Function calls represent values. This is no different
> > than the distinction between lvalues and rvalues in C.
>
> Yes, function calls cannot be lvalues right now. However, there is no
> reason that a function call _could not_ be an lvalue. That is exactly
> what the addition of __setcall__  would allow.

I have to admit that I don't find it particularly useful -- I still
don't think I like the idea much of using function calls as assignment
targets.

You wrote

  x(5) = True

would mean

  x.__setcall__(True, 5)

What would x(5) += 1 mean? The best I can come up with is

  __tmp = x(5)   # or x.__call__(5)
  if hasattr(__tmp, "__iadd__"):
    __val = __tmp.__iadd__(1)
  else:
    __val = __tmp + 1
  if __val is NotImplemented:
    raise TypeError(...)
  __tmp.__setcall__(__val, 5)

since this is essentially how x[5] += 1 works (except that the
hasattr() check is hidden inside PyNumber_InPlaceAdd and optimized
away to class definition time).

I expect that most people interested in having f() += 1 to work would
have to implement a dummy __setcall__() because their __iadd__ returns
self and there's no additional work to be done for the assignment.

I'm not convinced that all this complexity is worth it. For lists, +=
is syntactic sugar for .extend(). I expect that most use cases you can
come up with can similarly be argued away.

> On Aug 10, 2006, at 12:31 PM, Phillip J. Eby wrote:
> > Honestly, it might make more sense to get rid of augmented
> > assignment in Py3K rather than to add this.  It seems that the need
> > for something like this springs primarily from the existence of
> > augmented assignment.

I assume this was meant tongue-in-cheek. I see no reason to get rid of
+=. The opportunity for hypergeneralization (== ill-advised
generalization based on the misunderstanding of some mechanism) does
not automatically mean a mechanism should not be added (although it
can sometimes be a warning sign).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From armin.ronacher at active-4.com  Thu Aug 10 22:06:32 2006
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Thu, 10 Aug 2006 20:06:32 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Unicode_Data_in_Python2=2E5_is_missing_a_u?=
	=?utf-8?b?Y2RfNF8xXzAgb2JqZWN0?=
References: <loom.20060810T202810-913@post.gmane.org>
	<44DB838A.10600@v.loewis.de>
Message-ID: <loom.20060810T220456-500@post.gmane.org>

Hi,

Martin v. L?wis <martin <at> v.loewis.de> writes:
> > Doesn't that mean that there should also be an way to enforce unicode 4.1.0?
> 
> You mean, that there should be a ucd_4_1_0 object? No, why do you think
> there should be one? We don't plan to provide a copy of the UCD for each
> UCD version that was ever released (e.g. we skipped some between 3.2 and
> 4.1 also).

Right, I didn't know that. From that old bug report it sounded like a programmer
should be able to select a specific UCD version.

Regards,
Armin


From mcherm at mcherm.com  Thu Aug 10 22:35:14 2006
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu, 10 Aug 2006 13:35:14 -0700
Subject: [Python-Dev] Dict suppressing exceptions
Message-ID: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com>

Michael Urman writes:
> I strongly believe that unicode vs str here is the symptom and not the
> actual problem. The comparison between two non-us-ascii str/unicode
> instances is but one of many ways to raise an exception during
> comparison.
       [... example ...]
> Yes this is made up code. But I'm not arguing for a feature; I'm
> arguing for backwards compatibility. Because we do not know where
> these legitimate uses are, we cannot evaluate their likelihood to
> exist nor the level of breakage they will cause. If we make this a
> warning for any exception, we can satisfy both imagined camps. Those
> in Armin's position can make that warning raise an exception while
> debugging, and those using it on purpose can squash it.

See also my example from the beginning of this thread
(http://mail.python.org/pipermail/python-dev/2006-August/067978.html).
The example wasn't from real code, but it WAS quite plausible --
straightforward use of a popular Python Cookbook recipe.

But we seem to have reached a rough consensus:

Later, Guido writes:
> How about we change unicode-vs-str __eq__ to
> issue a warning (and return False) instead of raising
> UnicodeException?
      [... Marc-Andre Lemburg agrees ...]
> Great! Now we need someone to volunteer to write a patch (which should
> include doc and NEWS updates) in time for beta 3 (Aug 18).

I don't *strongly* object to this consensus, but if you haven't
glanced at my original example, take a look - it might convince you.
The proposed solution will not help with my example. I'm not sure the
motivation for breaking code like that example -- the bug-hunting
motivation is satisfied by issuing a warning as Michael Urman proposes,
then use an exception after one more release when people have had time
to fix their code.

-- Michael Chermside


From pje at telecommunity.com  Thu Aug 10 22:57:44 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 10 Aug 2006 16:57:44 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ca471dc20608101228m5769e191h779ab5ad36c27a5a@mail.gmail.co
 m>
References: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
	<2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
	<1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
Message-ID: <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com>

At 12:28 PM 8/10/2006 -0700, Guido van Rossum wrote:
>>On Aug 10, 2006, at 12:31 PM, Phillip J. Eby wrote:
>> > Honestly, it might make more sense to get rid of augmented
>> > assignment in Py3K rather than to add this.  It seems that the need
>> > for something like this springs primarily from the existence of
>> > augmented assignment.
>
>I assume this was meant tongue-in-cheek.

Not entirely, no.


>  I see no reason to get rid of
>+=. The opportunity for hypergeneralization (== ill-advised
>generalization based on the misunderstanding of some mechanism) does
>not automatically mean a mechanism should not be added (although it
>can sometimes be a warning sign).

I think it's a warning sign because I *know* what augmented assignment's 
semantics are supposed to be and I *still* try to use it with setdefault() 
sometimes -- but only when I'm doing an augmented assignment.  I never 
mistakenly try to assign to a function call in any other circumstance.

However, I'm also not clear that trying to assign to a function call *is* 
ill-advised.  One of the things that attracted me to Python in the first 
place is that it had a lot of features that would be considered 
"hypergeneralization" in other languages, e.g. the ability to create your 
own sequences, mappings, and callable objects in the first place.

That being said, the benefit of hypergeneralizing assignment seems small 
compared to its price.  So eliminating augmented assignment seems a more 
attractive way to get rid of the nuisance of the perennially repeated 
proposals to "fix" or otherwise extend it.  Sometimes it seems like half 
the time I want to use augmented assignment, I can't use it anyway for one 
reason or another, so it doesn't seem like that big of a loss.


From arigo at tunes.org  Thu Aug 10 23:12:54 2006
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 10 Aug 2006 23:12:54 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <44DB84EE.8020606@v.loewis.de>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
	<44DAF23A.103@egenix.com>
	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>
	<44DB84EE.8020606@v.loewis.de>
Message-ID: <20060810211254.GA24181@code0.codespeak.net>

Hi,

On Thu, Aug 10, 2006 at 09:11:42PM +0200, "Martin v. L?wis" wrote:
> I'm in favour of having this __eq__ just return False. I don't think
> the warning is necessary, (...)

+1


Armin

From steven.bethard at gmail.com  Thu Aug 10 23:18:06 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu, 10 Aug 2006 15:18:06 -0600
Subject: [Python-Dev] DRAFT: python-dev summary for 2006-07-16 to 2006-07-31
Message-ID: <d11dcfba0608101418v79b1fc58o7a114f1ab946d707@mail.gmail.com>

Here's the summary for the second half of July. Thanks in advance for
your comments and corrections!


=============
Announcements
=============

-------------------
Python 2.5 schedule
-------------------

After inserting a third beta release to allow some more time for
testing the new features, Python continues to make progress towards
the final Python 2.5 release. See `PEP 356`_ for more details and the
full schedule.

.. _PEP 356: http://www.python.org/dev/peps/pep-0356/

Contributing threads:

- `outstanding bugs to fix for 2.5
<http://mail.python.org/pipermail/python-dev/2006-July/067680.html>`__
- `Py2.5 release schedule
<http://mail.python.org/pipermail/python-dev/2006-July/067768.html>`__

-------------------------------
How to submit a patch to Python
-------------------------------

Just a few reminders for all those still new to python-dev. When
submitting a new patch to SourceForge, don't assign it to anyone. Most
python developers get email notifications for new patches and will
assign it to themselves if appropriate. If you feel like the approach
the patch takes might need discussion, it's alright to present it to
python-dev and ask for some feedback. If you do, be sure to put the
patch number and url (e.g. http://bugs.python.org/<sourceforge_id>)
near the top of the message, so that developers can easily find it.

And if you don't want to wait for your patch to be looked at (which
may take some time as all developers are volunteers), a few of the
folks here, including Martin v. L?wis, have offered a five-for-one
deal. Simply find five other patches, and check them for things like:

* Does the code look okay?
* Does Python build with it applied?
* Do all unit tests pass?
* Does the patch have tests?
* Does the patch have documentation?

Then post your notes to the five patch trackers and post a final
message to python-dev indicating the patches you reviewed and the
patch which you'd like to have someone look at for you.

Contributing threads:

- `new guy <http://mail.python.org/pipermail/python-dev/2006-July/067509.html>`__
- `first draft of bug guidelines for www.python.org/dev/
<http://mail.python.org/pipermail/python-dev/2006-July/067601.html>`__
- `Patch submitted, now what?
<http://mail.python.org/pipermail/python-dev/2006-July/067858.html>`__

----------------------------------------
Demos of trackers to replace SourceForge
----------------------------------------

There are currently three potential trackers that have successfully
imported the SourceForge data with demos online: roundup_, jira_ and
launchpad_. Try 'em out, and send your discussions and comments to
infrastructure at python.org and put your reports and reviews `on the
wiki`_.

.. _roundup: http://efod.se/python-tracker/
.. _jira: http://jira.python.atlassian.com/secure/Dashboard.jspa
.. _launchpad: https://demo.launchpad.net/products/python/+bugs
.. _on the wiki: http://wiki.python.org/moin/CallForTrackers

Contributing thread:

- `More tracker demos online
<http://mail.python.org/pipermail/python-dev/2006-July/067686.html>`__


=========
Summaries
=========

------------------------------
Restricted execution in Python
------------------------------

Brett Cannon decided this fortnight to go for an all-out capabilities
based restricted execution design, and posted a `new design
document`_. As part of this work, Brett planned to rewrite most of the
import machinery in pure Python code, hopefully cleaning up some of
the idiosyncrasies of the current import.c mechanisms, and allowing
him to do things like restrict imports to only .py files, not .pyc
files. Armin Rigo pointed out that a good place to start would be the
`PyPy import implementation`_.

On the restricted execution front, one of the things that is now
likely to happen in Brett's sandboxing branch is that
``object.__subclasses__()`` and dangerous constructors like the one
for the code object will be completely removed from the Python level.
This means a few backwards incompatible changes, but Brett suggested
that they should only break pretty advanced and esoteric Python code.
Since it's for his Ph.D. dissertation, he didn't want to tie his hands
by requiring full backwards compatibility, and he was fine with
waiting to merge his branch until Python 3000.

.. _new design document:
http://svn.python.org/view/python/branches/bcannon-sandboxing/securing_python.txt
.. _PyPy import implementation:
http://codespeak.net/svn/pypy/dist/pypy/module/__builtin__/importing.py

Contributing threads:

- `Capabilities / Restricted Execution
<http://mail.python.org/pipermail/python-dev/2006-July/067481.html>`__
- `new security doc using object-capabilities
<http://mail.python.org/pipermail/python-dev/2006-July/067555.html>`__

--------------------------
Character case and locales
--------------------------

Mihai Ibanescu asked about a `bug in the logging module`_ due to the
fact that ``'INFO'.lower() != 'info'`` in some locales. Marc-Andre
Lemburg and Martin v. L?wis explained that since in Unicode, nearly
all case-conversions are only script-dependent, not
language-dependent,  ``u'INFO'.lower() == u'info'`` should always be
true.

.. _bug in the logging module: http://bugs.python.org/1524081

Contributing thread:

- `logging module broken because of locale
<http://mail.python.org/pipermail/python-dev/2006-July/067504.html>`__

-----------------------------------------------
Progress on the C version of the decimal module
-----------------------------------------------

After looking at the current progress in converting the decimal module
to C, Raymond Hettinger suggested that rather than using the Python
implementation as an outline of the C implementation, a separate C
implementation should be developed and then later wrapped as necessary
to provide the Python APIs. Tim Peters explained their incremental
approach: leaving most of the module written in Python, and converting
methods to C code one at a time. Raymond had originally supported this
approach, but after viewing the current C code, thought that it would
result in C code that was too complex and convoluted.

There was some extended discussion on the mechanism in the current
decimal module for holding flags, which uses a dict mapping error
types to the counts of their occurrences. Raymond in particular wanted
the C decimal module to be able to change this API if it was too
complex to implement. A number of others agreed that the API had been
a bad decision, and it looked like there would at least be a note in
the documentation for Python 2.5 suggesting that users should not rely
on the counting feature.

Contributing thread:

- `Strategy for converting the decimal module to C
<http://mail.python.org/pipermail/python-dev/2006-July/067537.html>`__

---------------------------------------
PEP 357: Integer clipping and __index__
---------------------------------------

Armin Rigo pointed out that the current implementation of
``__index__()`` was incorrectly truncating long integers::

    >>> (2**100).__index__()
    2147483647

As the original ``__index__()`` method was intended only to allow
things other than plain Python ints as slice indices, truncating to
the maximum value was fine. However, when ``__index__()`` also became
the "can you faithfully act like an integer" check, this truncation
was no longer acceptable. Nick Coghlan spent some time reworking the
`PEP 357`_ C API so that all the use cases of ``__index__()`` were
covered. His `patch for fixing __index__`_ changes the nb_index slot
to return a PyInt or PyLong instead of a C int, and introduces the C
API functions PyNumber_Index, PyNumber_AsSsize_t and
PyNumber_AsClippedSsize_t(), all of which have an output variable
signifying whether or not the object they received had an
``__index__`` method.

.. _PEP 357: http://www.python.org/dev/peps/pep-0357/
.. _patch for fixing __index__: http://bugs.python.org/1530738

Contributing thread:

- `Bad interaction of __index__ and sequence repeat
<http://mail.python.org/pipermail/python-dev/2006-July/067774.html>`__

--------------------------------------------------------
PEP 302: Non-importer objects on sys.path_importer_cache
--------------------------------------------------------

Phillip J. Eby asked about how to best fix the non-PEP-302 compliant
changes to the import machinery made by the Need for Speed Sprint.
`PEP 302`_ indicates that everything on sys.path_importer_cache should
be either None or a valid importer object, but the Need for Speed
changes added True and False values to that as well. After getting
approval to make the appropriate changes necessary to stay
PEP-302-compliant, Phillip added ``imp.NullImporter`` to replace False
values, and kept ``None`` to mean that the builtin import machinery
should be used.

.. _PEP 302: http://www.python.org/dev/peps/pep-0302/

Contributing threads:

- `Undocumented PEP 302 protocol change by need-for-speed sprint
<http://mail.python.org/pipermail/python-dev/2006-July/067587.html>`__
- `Release manager pronouncement needed: PEP 302 Fix
<http://mail.python.org/pipermail/python-dev/2006-July/067713.html>`__

---------------------------------------------------------------
Running the test suites of user projects when Python is updated
---------------------------------------------------------------

Grig Gheorghiu volunteered to do some of the work to get community
buildbots running, that is, buildbots running the test suites of
Python user projects whenever the Python core repository was updated.
People were quite enthusiastic and Martin v. L?wis offered to set up a
post-commit
hook on the python repository to trigger a build on Grig's buildbots
if necessary.

Contributing threads:

- `Community buildbots
<http://mail.python.org/pipermail/python-dev/2006-July/067476.html>`__
- `Community buildbots (was Re: User's complaints)
<http://mail.python.org/pipermail/python-dev/2006-July/067477.html>`__
- `Community buildbots -- reprise
<http://mail.python.org/pipermail/python-dev/2006-July/067688.html>`__

--------------------------------------
Safe dumper/loader using Python syntax
--------------------------------------

Sylvain Fourmanoit presented his miniconf_ module which is a safe and
cross-version dumper/loader for simple objects using the Python
syntax. People generally liked the module, and Phillip J. Eby helped
Sylvain clean up the implementation a bit. There was some discussion
of including it in the Python 2.6 stdlib and perhaps Bob Ippolito's
simplejson_ module alongside it.

.. _miniconf: http://cheeseshop.python.org/pypi?:action=display&name=miniconf
.. _simplejson: http://undefined.org/python/#simplejson

Contributing threads:

- `New miniconf module
<http://mail.python.org/pipermail/python-dev/2006-July/067710.html>`__
- `JSON implementation in Python 2.6
<http://mail.python.org/pipermail/python-dev/2006-July/067720.html>`__

-------------------------------
Programmatically sending Ctrl-C
-------------------------------

In testing his `patch to make sockets and Ctrl-C play nicely`_, Tony
Nelson found that he needed a portable way to send a Ctrl-C-like
signal. For Unix, he was using ``signal.alarm``, but was wondering if
there was a way to get something similar on Windows. Martin v. L?wis
pointed out GenerateConsoleCtrlEvent, but also noted that this would
send the Ctrl-C to all processes. In the end, Tony decided to punt on
Windows, and just stick with the Unix tests.

.. _patch to make sockets and Ctrl-C play nicely: http://bugs.python.org/1519025

Contributing threads:

- `Socket Timeouts patch 1519025
<http://mail.python.org/pipermail/python-dev/2006-July/067647.html>`__
- `Testing Socket Timeouts patch 1519025
<http://mail.python.org/pipermail/python-dev/2006-July/067828.html>`__

------------------------------------------
Documenting performance of container types
------------------------------------------

Neal Becker asked about documentation the performance of the basic
Python container types, e.g. that lookup in a dict is O(1) and
deletion from the beginning of a list is O(N). A number of people
agreed that having such information would be helpful, but there was
some concern that Guido should be the one to decide what performances
guarantees were made by the language, and not just the CPython
implementation. The discussion trailed off before any final decisions
on how to update the documentation were made.

Contributing thread:

- `Document performance requirements?
<http://mail.python.org/pipermail/python-dev/2006-July/067609.html>`__

---------------------------
Running the uuid test suite
---------------------------

Georg Brandl fixed a bug that was causing the new uuid module's test
suite not to run at all. The resulting tests indicated a number of
problems in determining a MAC address.  Neal Norwitz patched the uuid
module so that it should work at least on Linux, Tru64, Solaris, and
HP-UX, and Tim Peters patched it so that test_uuid no longer thinks
that the uuid module knows multiple ways of getting a well-defined MAC
address (which it doesn't on Windows).

Contributing threads:

- `uuid test suite failing
<http://mail.python.org/pipermail/python-dev/2006-July/067744.html>`__
- `how about adding ping's uuid module to the standard lib ?
<http://mail.python.org/pipermail/python-dev/2006-July/067764.html>`__
- `Another uuid problem
<http://mail.python.org/pipermail/python-dev/2006-July/067777.html>`__
- `test_uuid <http://mail.python.org/pipermail/python-dev/2006-July/067830.html>`__

------------------------------------
CPython and checking for NULL values
------------------------------------

Neal Norwitz took a look at some of the issues raised by the automatic
analysis of Python's source code offered by Klocwork_. There was a
fairly long discussion around a Py_XINCREF of a variable that was
required by the documentation to be non-NULL (and thus the "X" is
unnecessary). There was some suggestion of trying to check for NULL
values anyway, but the rest of Python doesn't do such checks.

.. _Klocwork: http://www.klocwork.com/

Contributing thread:

- `remaining issues from Klocwork static analysis
<http://mail.python.org/pipermail/python-dev/2006-July/067681.html>`__

---------------------------------------------
Excluding certain constructs from Python code
---------------------------------------------

Boris Borcic suggested that to make changing version of Python easier,
style sheets should be introduced such that you could allow or
disallow particular constructs that you liked or didn't like.  People
thought this was generally a very bad idea as it would essentially
introduce a bunch of language variants, and coders might not be able
to read each others' source code without first applying the
appropriate transformation.

Contributing threads:

- `Python Style Sheets ? Re: User's complaints
<http://mail.python.org/pipermail/python-dev/2006-July/067432.html>`__
- `Python Style Sheets ? Re: User's complaints
<http://mail.python.org/pipermail/python-dev/2006-July/067491.html>`__

---------------------------------------------------------
Making attributes with leading single underscores private
---------------------------------------------------------

David Hopwood proposed enforcing the `PEP 8`_ convention that
attributes with a single underscore are private to that object. His
approach revolved around allowing only the first argument of a
function to access attributes starting with '_', but Armin Rigo and
others felt that this was not likely to be enforceable, giving an
example where subclassing allowed access to supposedly private
attributes. People generally felt that without an implementation to
back up the proposal, there wasn't much to discuss.

.. _PEP 8: http://www.python.org/dev/peps/pep-0008/

Contributing thread:

- `Internal namespace proposal
<http://mail.python.org/pipermail/python-dev/2006-July/067727.html>`__

-----------------------------------
Loading module attributes on demand
-----------------------------------

In order to reduce the memory consumption of GTK+ applications written
in Python, Johan Dahlin was looking into dynamically generating module
attributes so that they would only be loaded when the application
actually accessed them. He was able to produce this behavior by
subclassing ModuleType, overridding __getattribute__, and then putting
this object onto sys.path, but he felt like this was kind of a hackish
solution. Phillip J. Eby pointed out the importing_ package and said
that the __getattribute__ approach was generally okay, though it would
cause problems for pydoc and inspect which don't handle subclasses of
ModuleType well. Andrew Bennetts pointed out mercurial's demandload_
which allows modules to be imported on demand, but this didn't really
solve Johan's problem because all attributes of the modules themselves
were still imported at the same time.

.. _importing: http://cheeseshop.python.org/pypi/Importing
.. _demandload:
http://selenic.com/repo/hg?f=cb4715847a81;file=mercurial/demandload.py

Contributing thread:

- `Dynamic module namspaces
<http://mail.python.org/pipermail/python-dev/2006-July/067485.html>`__

-------------------------------
Improving the Python test suite
-------------------------------

Matt Fleming has put together `a wiki page`_ indicating the tests that
are currently missing from Python's test suite, as well as the tests
that are incomplete. He plans on working his way through the list when
he gets some time, but help for any of the tests is quite welcome.

.. _a wiki page: http://wiki.python.org/moin/ImprovingLibTests

Contributing thread:

- `Improving unit tests for the standard library
<http://mail.python.org/pipermail/python-dev/2006-July/067723.html>`__

----------------------------------
Improving the Python documentation
----------------------------------

Georg Brandl, referring to `bug 469773`_, suggested that python gain a
"Using Python" page containing the man page and how to invoke the
interpreter. He also suggested that creating a list of frequently
needed documentation sections that are hard to find for newbies could
go a long way towards making the Python documentation more
user-friendly.

.. _bug 469773: http://bugs.python.org/469773

Contributing thread:

- `Using Python docs
<http://mail.python.org/pipermail/python-dev/2006-July/067823.html>`__


================
Deferred Threads
================
- `struct module and coercing floats to integers
<http://mail.python.org/pipermail/python-dev/2006-July/067798.html>`__
- `Rounding float to int directly (Re: struct module and coercing
floats to integers)
<http://mail.python.org/pipermail/python-dev/2006-July/067819.html>`__


==================
Previous Summaries
==================
- `Support for PyGetSetDefs in pydoc
<http://mail.python.org/pipermail/python-dev/2006-July/067590.html>`__


===============
Skipped Threads
===============
- `Problem with super() usage
<http://mail.python.org/pipermail/python-dev/2006-July/067480.html>`__
- `Pronouncement on SF #1520294 sought
<http://mail.python.org/pipermail/python-dev/2006-July/067483.html>`__
- `I have submitted a patch that implement IrDA socket support .
<http://mail.python.org/pipermail/python-dev/2006-July/067484.html>`__
- `User's complaints
<http://mail.python.org/pipermail/python-dev/2006-July/067501.html>`__
- `Pickling objects that return string from reduce
<http://mail.python.org/pipermail/python-dev/2006-July/067508.html>`__
- `[Python-checkins] r50708 - in python/trunk: Lib/test/test_sys.py
Misc/NEWS Python/pystate.c
<http://mail.python.org/pipermail/python-dev/2006-July/067549.html>`__
- `Python sprint in NY and CA, Aug. 21-24
<http://mail.python.org/pipermail/python-dev/2006-July/067554.html>`__
- `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2006-July/067563.html>`__
- `os.utime and os.chmod failures (etc) Python 2.5b2
<http://mail.python.org/pipermail/python-dev/2006-July/067566.html>`__
- `FW: Bug? Certainly a new *behavior* from subprocess in 2.5 on Win32
<http://mail.python.org/pipermail/python-dev/2006-July/067597.html>`__
- `Behavior change in subprocess.py
<http://mail.python.org/pipermail/python-dev/2006-July/067599.html>`__
- `segfault when using PyGILState_Ensure/Release in Python2.3.4
<http://mail.python.org/pipermail/python-dev/2006-July/067605.html>`__
- `Ireland PyPy sprint 21th-27th August 2006
<http://mail.python.org/pipermail/python-dev/2006-July/067610.html>`__
- `Python 2.4, VS 2005 & Profile Guided Optmization
<http://mail.python.org/pipermail/python-dev/2006-July/067646.html>`__
- `Python sprint in Arlington July 29/30
<http://mail.python.org/pipermail/python-dev/2006-July/067667.html>`__
- `setup.py and cross-compiling
<http://mail.python.org/pipermail/python-dev/2006-July/067669.html>`__
- `2.5: uses of sys.exc_type, exc_value
<http://mail.python.org/pipermail/python-dev/2006-July/067711.html>`__
- `[Windows, buildbot] kill_python.c mystery
<http://mail.python.org/pipermail/python-dev/2006-July/067722.html>`__
- `patch for mbcs codec (again)
<http://mail.python.org/pipermail/python-dev/2006-July/067724.html>`__
- `Which version of distutils to ship with Python 2.5?
<http://mail.python.org/pipermail/python-dev/2006-July/067726.html>`__
- `Patch for building ctypes on more OpenBSD target platforms
<http://mail.python.org/pipermail/python-dev/2006-July/067735.html>`__
- `Release manager: pdb bugfix incompatibility
<http://mail.python.org/pipermail/python-dev/2006-July/067741.html>`__
- `patching pydoc?
<http://mail.python.org/pipermail/python-dev/2006-July/067776.html>`__
- `Fwd: patching pydoc?
<http://mail.python.org/pipermail/python-dev/2006-July/067788.html>`__
- `Patch Against shutil.copytree Bug
<http://mail.python.org/pipermail/python-dev/2006-July/067802.html>`__
- `httplib and bad response chunking
<http://mail.python.org/pipermail/python-dev/2006-July/067803.html>`__
- `cgi.FieldStorage DOS (sf bug #1112549)
<http://mail.python.org/pipermail/python-dev/2006-July/067811.html>`__
- `Eliminating loops
<http://mail.python.org/pipermail/python-dev/2006-July/067813.html>`__

From guido at python.org  Thu Aug 10 23:25:00 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 14:25:00 -0700
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com>
References: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com>
Message-ID: <ca471dc20608101425q742effd5p28780351d4e630b2@mail.gmail.com>

On 8/10/06, Michael Chermside <mcherm at mcherm.com> wrote:
> See also my example from the beginning of this thread
> (http://mail.python.org/pipermail/python-dev/2006-August/067978.html).
> The example wasn't from real code, but it WAS quite plausible --
> straightforward use of a popular Python Cookbook recipe.
[...]
> I don't *strongly* object to this consensus, but if you haven't
> glanced at my original example, take a look - it might convince you.

Alas, I have no idea what it does. Can you come up with an example
that doesn't require enums and localization?


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mal at egenix.com  Thu Aug 10 23:28:21 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 10 Aug 2006 23:28:21 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com>
References: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com>
Message-ID: <44DBA4F5.4080805@egenix.com>

Michael Chermside wrote:
>> How about we change unicode-vs-str __eq__ to
>> issue a warning (and return False) instead of raising
>> UnicodeException?
>       [... Marc-Andre Lemburg agrees ...]
>> Great! Now we need someone to volunteer to write a patch (which should
>> include doc and NEWS updates) in time for beta 3 (Aug 18).
> 
> I don't *strongly* object to this consensus, but if you haven't
> glanced at my original example, take a look - it might convince you.
> The proposed solution will not help with my example. I'm not sure the
> motivation for breaking code like that example -- the bug-hunting
> motivation is satisfied by issuing a warning as Michael Urman proposes,
> then use an exception after one more release when people have had time
> to fix their code.

The warning framework gives programmers great flexibility in handling
these situation:

* they can silence the warning, just report it once, always report it
* they can have the warning raise an exception
* they can log the warning in some way

It's very flexible to accommodate for all kinds of ways to handle
the situation.

By introducing a new category of warning for these Unicode-related
warnings, adding specific warning filters will be easy.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 10 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Thu Aug 10 23:31:37 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 10 Aug 2006 23:31:37 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <ca471dc20608100933p2ee25e0dl5558bc69766c528a@mail.gmail.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>	<44DAF23A.103@egenix.com>	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>	<44DB4B37.1000401@egenix.com>
	<ca471dc20608100933p2ee25e0dl5558bc69766c528a@mail.gmail.com>
Message-ID: <44DBA5B9.5030009@egenix.com>

Guido van Rossum wrote:
> On 8/10/06, M.-A. Lemburg <mal at egenix.com> wrote:
>> Guido van Rossum wrote:
>>> Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to
>>> issue a warning (and return False) instead of raising
>>> UnicodeException? That won't break much code (it's unlikely that
>>> people *depend* on this exception since it's generally a symptom of
>>> insane mixing of str and unicode). Then no further changes to
>>> dictobject.c are necessary (except fixing that one misleading
>>> comment).
>> Good idea.
> 
> Great! Now we need someone to volunteer to write a patch (which should
> include doc and NEWS updates) in time for beta 3 (Aug 18).

I'll give it a go.

Not sure what kinds of docs we'll need for this,
though (and more importantly: where to put them, e.g. into the tutorial
in the dictionary section, somewhere into the Unicode section ?)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 10 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From guido at python.org  Thu Aug 10 23:33:38 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 14:33:38 -0700
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <44DB84EE.8020606@v.loewis.de>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
	<44DAF23A.103@egenix.com>
	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>
	<44DB84EE.8020606@v.loewis.de>
Message-ID: <ca471dc20608101433x282b3986p28bd1da8473e5a1c@mail.gmail.com>

On 8/10/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Guido van Rossum schrieb:
> > Hmm... Here's an idea... How about we change unicode-vs-str __eq__ to
> > issue a warning (and return False) instead of raising
> > UnicodeException?
>
> I'm in favour of having this __eq__ just return False. I don't think
> the warning is necessary, and believe that people will complain about
> getting it, but if that is the only way to achieve consensus,
> so be it.

Not sure I agree. I think that users would find it disturbing (and it
could hide bugs) if u"\xff" == "\xff" returned False without warning
while u"\x7f" == "\x7f" returns True.

> > A warning would arguably have the same helping effect. (I suspect
> > actually that we would have used warnings all along except we didn't
> > have the warning framework when unicode was first introduced.)
>
> I believe we would have neither had we rich comparisons available
> at the time. That unicode-vs-str-__eq__ raises an exception is just
> a fall-out of not supporting __cmp__ (and it is good that __cmp__
> is not supported - there is no meaningful way to order str/unicode
> if the string is not in the system encoding).

Maybe. I agree that 3-way compare has no business guessing the
ordering of u"\xff" and "\xff". I'm not so sure that if we'd had rich
comparison we'd have chosen __eq__ to return False for these.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Aug 10 23:36:16 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 14:36:16 -0700
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <20060810211254.GA24181@code0.codespeak.net>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
	<44DAF23A.103@egenix.com>
	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>
	<44DB84EE.8020606@v.loewis.de>
	<20060810211254.GA24181@code0.codespeak.net>
Message-ID: <ca471dc20608101436n1a63f74sfe1c3f4d9e090059@mail.gmail.com>

On 8/10/06, Armin Rigo <arigo at tunes.org> wrote:
> Hi,
>
> On Thu, Aug 10, 2006 at 09:11:42PM +0200, "Martin v. L?wis" wrote:
> > I'm in favour of having this __eq__ just return False. I don't think
> > the warning is necessary, (...)
>
> +1

Can you explain why you believe that no warning is necessary?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Aug 10 23:39:04 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 14:39:04 -0700
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com>
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
	<1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
	<5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com>
Message-ID: <ca471dc20608101439q14f0532ai71609898cc8b67f2@mail.gmail.com>

On 8/10/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> That being said, the benefit of hypergeneralizing assignment seems small
> compared to its price.  So eliminating augmented assignment seems a more
> attractive way to get rid of the nuisance of the perennially repeated
> proposals to "fix" or otherwise extend it.  Sometimes it seems like half
> the time I want to use augmented assignment, I can't use it anyway for one
> reason or another, so it doesn't seem like that big of a loss.

I think you are strongly understimating the benefits of augmented
assignments. I held out against it for many years. But the users
didn't stop asking for it. They're mostly happy. That some aren't
happy until we've hypergeneralized it doesn't mean we shouldn't have
done it in the first place. (Similar as with lambda. :)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Aug 10 23:44:22 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 14:44:22 -0700
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <ebftlt$buf$1@sea.gmane.org>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>
	<ebe6lk$1tk$1@sea.gmane.org>
	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>
	<44DB4023.5090904@gmail.com>
	<ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>
	<ebftlt$buf$1@sea.gmane.org>
Message-ID: <ca471dc20608101444x744e4e2dkb52e2d747f1cbc3a@mail.gmail.com>

Thanks for your understanding.

Anyway, Nick's patch differs in at least one significant way from my
proposal -- (10**10).__index__() returns sys.maxint in his patch,
while I want it to return 10000000000L. So this is still an open
issue.

--Guido

On 8/10/06, Travis E. Oliphant <oliphant.travis at ieee.org> wrote:
> Guido van Rossum wrote:
> > On 8/10/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> Guido van Rossum wrote:
> >>>> It seems like Nick's recent patches solved the problems that were
> >>>> identified.
> >>> Nick, can you summarize how your patches differ from my proposal?
> >> nb_index and __index__ are essentially exactly as you propose.
> >
> > Then I don't understand why Travis is objecting against my proposal!
>
> I must have missed his most recent patch that changed the result to
> return a Python object.  I thought earlier versions didn't do that.
>
> My objection is not particularly solid.  At this point it's largely a
> wish to avoid the extra overhead (there are some camps that already
> complain about NumPy having too much indexing overhead --- although they
> should be using Python lists for there purposes if indexing overhead
> really is a problem).
>
> As it appears that several people feel this is the best way forward,
> then I'll re-work my NumPy code.   I still appreciate the change that
> allows other Python objects to be integers and eliminates the "only true
> integers allowed" flavor of several places in the Python code.
>
> Thanks for all your hard work.
>
> -Travis
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rrr at ronadam.com  Thu Aug 10 23:50:56 2006
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 10 Aug 2006 16:50:56 -0500
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <44DBA4F5.4080805@egenix.com>
References: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com>
	<44DBA4F5.4080805@egenix.com>
Message-ID: <44DBAA40.2020502@ronadam.com>

M.-A. Lemburg wrote:
> Michael Chermside wrote:
>>> How about we change unicode-vs-str __eq__ to
>>> issue a warning (and return False) instead of raising
>>> UnicodeException?
>>       [... Marc-Andre Lemburg agrees ...]
>>> Great! Now we need someone to volunteer to write a patch (which should
>>> include doc and NEWS updates) in time for beta 3 (Aug 18).
>> I don't *strongly* object to this consensus, but if you haven't
>> glanced at my original example, take a look - it might convince you.
>> The proposed solution will not help with my example. I'm not sure the
>> motivation for breaking code like that example -- the bug-hunting
>> motivation is satisfied by issuing a warning as Michael Urman proposes,
>> then use an exception after one more release when people have had time
>> to fix their code.
> 
> The warning framework gives programmers great flexibility in handling
> these situation:
> 
> * they can silence the warning, just report it once, always report it
> * they can have the warning raise an exception
> * they can log the warning in some way
> 
> It's very flexible to accommodate for all kinds of ways to handle
> the situation.
> 
> By introducing a new category of warning for these Unicode-related
> warnings, adding specific warning filters will be easy.

This sounds like the correct tool to me. +1

Would it be possible to generate warnings when either Unicode or stings 
are coerced to the other implicitly but not explicitly?

That may also generate a warning in the case of the dictionary problem 
being discussed, (I think), and may be more useful for checking for 
non-intentionally mixed Unicode and strings.

Ron



From greg.ewing at canterbury.ac.nz  Fri Aug 11 02:20:14 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 11 Aug 2006 12:20:14 +1200
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <44DAE224.50705@v.loewis.de>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
	<44D8E38C.8000505@v.loewis.de> <44D9503D.1040809@canterbury.ac.nz>
	<44D983E3.701@v.loewis.de> <44DA8D43.4040009@canterbury.ac.nz>
	<44DAE224.50705@v.loewis.de>
Message-ID: <44DBCD3E.8070304@canterbury.ac.nz>

Martin v. L?wis wrote:

> Perform it: do the steps that the algorithm says you should
> do, or let a machine do it. IOW, run the code.

That can't be right, because it would mean that
anyone who runs a program that contains a
patented algorithm, even one bought or otherwise
obtained from someone else, would need to
individually negotiate a licence with the
patent owner. That clearly doesn't happen.

> If your point is that software patents are evil: I fully
> agree.

My point is that software patents don't even
seem to make *sense* -- especially in conjunction
with open source software.

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Aug 11 02:35:36 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 11 Aug 2006 12:35:36 +1200
Subject: [Python-Dev] Is module clearing still necessary? [Re: Is this a
	bug?]
In-Reply-To: <44DB22D1.60002@gmail.com>
References: <ebcs51$tgo$1@sea.gmane.org> <20060809084110.E71E.JCARLSON@uci.edu>
	<44DB22D1.60002@gmail.com>
Message-ID: <44DBD0D8.4000404@canterbury.ac.nz>

Nick Coghlan wrote:

> Early versions of the PEP 338 implementation also ran into the problem of a 
> module's globals getting cleared when the module itself goes away

A while back it was suggested that the module-clearing
thing might be dropped now that we have cyclic GC, but
I don't remember there being much discussion about it.
Has anyone had any thoughts on that?

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Aug 11 03:21:37 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 11 Aug 2006 13:21:37 +1200
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
References: <ebd2sp$nnv$1@sea.gmane.org> <ebd6od$6m4$1@sea.gmane.org>
	<ebd8f0$csd$1@sea.gmane.org> <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
Message-ID: <44DBDBA1.7090402@canterbury.ac.nz>

Michael Urman wrote:

> Just to play devil's advocate here, why not to a function call via a
> new __setcall__?

You still haven't answered the question of what this
buys you over just giving your object a suitably
named method that does whatever your __setcall__
method would have done.

Can you show a realistic use case for this, that
demonstrates a clear advantage to having new
syntax?

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Aug 11 05:42:27 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 11 Aug 2006 15:42:27 +1200
Subject: [Python-Dev] Split augmented assignment into two operator sets?
 [Re: SyntaxError: can't assign to function call]
In-Reply-To: <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com>
References: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
	<2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
	<1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
	<5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com>
Message-ID: <44DBFCA3.5040708@canterbury.ac.nz>

Phillip J. Eby wrote:

> I think it's a warning sign because I *know* what augmented assignment's 
> semantics are supposed to be and I *still* try to use it with setdefault() 
> sometimes -- but only when I'm doing an augmented assignment.  I never 
> mistakenly try to assign to a function call in any other circumstance.

I think this problem arises because of the dual semantics
of augmented assignment. One tends to have two *different*
mental models of what it does, depending on whether the
object in question is immutable:

   (1) For immutable objects:

      x += y  <-->  x = x + y

   (2) For mutable objects:

      x += y  <-->  x.__iadd__(y)

and it's easy to forget that case (2) also happens
to have the side effect of assigning the result
back to x, which is usually redundant in the case
of an immutable object, and thus to slip into the
error of thinking that

      x(z) += y  <--->  x(z).__iadd__(y)

should be a reasonable thing to expect. And in
the case of a mutable result from x(z), it's
hard to argue convincingly that it's not
reasonable.

As to what this is a warning of, maybe it's that
the dual semantics were a mistake in the first
place, and that there really should have been
two sets of operators, e.g.

   x += y  <--->   x = x + y      # literally

   x +: y  <--->   x.__iadd__(y)  # with *no* assignment

The +: family of operators would then just be
normal infix operators, not assignments at all,
and one would be able to say

   x(z) +: y

without any trouble (provided the result of x(z)
were mutable, since immutables would not have
an __iadd__ method at all).

Another benefit to this is that the +: operators
would be usable on objects in an outer scope,
since they are not assignments and would therefore
not imply the localness of anything.

Something to consider for Py3k?

PS: Why an extra set of operators, rather than
ordinary methods? One of the motivations for the
augmented assignment operators was Numeric and the
desire to be able to express in-place operations
on arrays concisely. The +: family would fill this
need.

PPS: Yes, I know the use of : might be ill-advised.
It's just an example - any other concise notation
would do as well.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Fri Aug 11 05:45:07 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 20:45:07 -0700
Subject: [Python-Dev] Split augmented assignment into two operator sets?
	[Re: SyntaxError: can't assign to function call]
In-Reply-To: <44DBFCA3.5040708@canterbury.ac.nz>
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
	<1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
	<5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com>
	<44DBFCA3.5040708@canterbury.ac.nz>
Message-ID: <ca471dc20608102045t268a8305v2f8fbf8dda30811f@mail.gmail.com>

-1. I just don't think we should add more operators -- it'll just
cause more question marks on users' faces...

--Guido

On 8/10/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Phillip J. Eby wrote:
>
> > I think it's a warning sign because I *know* what augmented assignment's
> > semantics are supposed to be and I *still* try to use it with setdefault()
> > sometimes -- but only when I'm doing an augmented assignment.  I never
> > mistakenly try to assign to a function call in any other circumstance.
>
> I think this problem arises because of the dual semantics
> of augmented assignment. One tends to have two *different*
> mental models of what it does, depending on whether the
> object in question is immutable:
>
>    (1) For immutable objects:
>
>       x += y  <-->  x = x + y
>
>    (2) For mutable objects:
>
>       x += y  <-->  x.__iadd__(y)
>
> and it's easy to forget that case (2) also happens
> to have the side effect of assigning the result
> back to x, which is usually redundant in the case
> of an immutable object, and thus to slip into the
> error of thinking that
>
>       x(z) += y  <--->  x(z).__iadd__(y)
>
> should be a reasonable thing to expect. And in
> the case of a mutable result from x(z), it's
> hard to argue convincingly that it's not
> reasonable.
>
> As to what this is a warning of, maybe it's that
> the dual semantics were a mistake in the first
> place, and that there really should have been
> two sets of operators, e.g.
>
>    x += y  <--->   x = x + y      # literally
>
>    x +: y  <--->   x.__iadd__(y)  # with *no* assignment
>
> The +: family of operators would then just be
> normal infix operators, not assignments at all,
> and one would be able to say
>
>    x(z) +: y
>
> without any trouble (provided the result of x(z)
> were mutable, since immutables would not have
> an __iadd__ method at all).
>
> Another benefit to this is that the +: operators
> would be usable on objects in an outer scope,
> since they are not assignments and would therefore
> not imply the localness of anything.
>
> Something to consider for Py3k?
>
> PS: Why an extra set of operators, rather than
> ordinary methods? One of the motivations for the
> augmented assignment operators was Numeric and the
> desire to be able to express in-place operations
> on arrays concisely. The +: family would fill this
> need.
>
> PPS: Yes, I know the use of : might be ill-advised.
> It's just an example - any other concise notation
> would do as well.
>
> --
> Greg Ewing, Computer Science Dept, +--------------------------------------+
> University of Canterbury,          | Carpe post meridiem!                 |
> Christchurch, New Zealand          | (I'm not a morning person.)          |
> greg.ewing at canterbury.ac.nz        +--------------------------------------+
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From oliphant.travis at ieee.org  Fri Aug 11 06:07:57 2006
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Thu, 10 Aug 2006 22:07:57 -0600
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <ca471dc20608101444x744e4e2dkb52e2d747f1cbc3a@mail.gmail.com>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>	<ebe6lk$1tk$1@sea.gmane.org>	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>	<44DB4023.5090904@gmail.com>	<ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>	<ebftlt$buf$1@sea.gmane.org>
	<ca471dc20608101444x744e4e2dkb52e2d747f1cbc3a@mail.gmail.com>
Message-ID: <ebgvqu$d81$1@sea.gmane.org>

Guido van Rossum wrote:
> Thanks for your understanding.
> 
> Anyway, Nick's patch differs in at least one significant way from my
> proposal -- (10**10).__index__() returns sys.maxint in his patch,
> while I want it to return 10000000000L. So this is still an open
> issue.
>

I've reviewed Nick's patch and finally see the light for the need of 
nb_index to return a Python object.  It also nicely parallels nb_int and 
nb_long so the same code can often be used (when it's appropriate for 
the object to be an integer).

I understand the general rationale for the C-API, but I share 
reservations about using output arguments to convey error information. 
This seems quite a bit different from what Python normally does.

I am going to beef up PEP-357 to address the main problem of nb_index 
returning a Py_ssize_t and then look at the C-API question more closely.

I'm also going out of town next week (and have a bunch to prepare for 
it) so I'll only have a few hours to look at it.

-Travis



From oliphant.travis at ieee.org  Fri Aug 11 06:10:16 2006
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Thu, 10 Aug 2006 22:10:16 -0600
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>	
	<ebe6lk$1tk$1@sea.gmane.org>
	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>
Message-ID: <44DC0328.9040209@ieee.org>

Guido van Rossum wrote:
>
> What do you think (10**10).__index__() should return (when called from 
> Python)?
>
I'm with Guido on this point.  I think (10**10).__index__() should 
return the full long integer when called from within Python.

-Travis



From aahz at pythoncraft.com  Fri Aug 11 06:23:07 2006
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 10 Aug 2006 21:23:07 -0700
Subject: [Python-Dev] Split augmented assignment into two operator sets?
	[Re: SyntaxError: can't assign to function call]
In-Reply-To: <44DBFCA3.5040708@canterbury.ac.nz>
References: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
	<2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
	<1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
	<5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com>
	<44DBFCA3.5040708@canterbury.ac.nz>
Message-ID: <20060811042307.GA19407@panix.com>

On Fri, Aug 11, 2006, Greg Ewing wrote:
> Phillip J. Eby wrote:
>> 
>> I think it's a warning sign because I *know* what augmented
>> assignment's semantics are supposed to be and I *still* try to use it
>> with setdefault() sometimes -- but only when I'm doing an augmented
>> assignment.  I never mistakenly try to assign to a function call in
>> any other circumstance.
>
> I think this problem arises because of the dual semantics of augmented
> assignment. One tends to have two *different* mental models of what it
> does, depending on whether the object in question is immutable:
>
>    (1) For immutable objects:
> 
>       x += y  <-->  x = x + y
> 
>    (2) For mutable objects:
> 
>       x += y  <-->  x.__iadd__(y)

What I try to do is always remember that

    x += y  <-->  x = x.__iadd__(y)

which mostly solves the problem.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it."  --Brian W. Kernighan

From foom at fuhm.net  Fri Aug 11 06:53:41 2006
From: foom at fuhm.net (James Y Knight)
Date: Fri, 11 Aug 2006 00:53:41 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com>
References: <1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
	<2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
	<1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
	<5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com>
Message-ID: <5676E6A1-1FFE-40FC-9A99-90D82B2FE900@fuhm.net>


On Aug 10, 2006, at 4:57 PM, Phillip J. Eby wrote:

> However, I'm also not clear that trying to assign to a function  
> call *is* ill-advised.  One of the things that attracted me to  
> Python in the first place is that it had a lot of features that  
> would be considered "hypergeneralization" in other languages, e.g.  
> the ability to create your own sequences, mappings, and callable  
> objects in the first place.
>
> That being said, the benefit of hypergeneralizing assignment seems  
> small compared to its price.

Well, it's a mostly obvious extension of an existing idea, so the  
price doesn't seem all that high. The main problem is that so far,  
there have been 0 convincing use cases. So no matter how moderate the  
price, it's definitely bigger than the benefit. But anyhow, speaking  
of hypergeneralization...since this has 0 use cases anyhow, might as  
well hyperhypergeneralize it...

Well, why should assignment be limited to only local variables, item  
access, and function calls. Why shouldn't you be able to potentially  
assign to _any_ expression!

Since
x + a turns into (very roughly...) x.__add__(a),

then,
x + a = 5 could turn into x.__add__.__setcall__(5, a).

Of course, since normal __add__ functions don't have a __setcall__,  
doing this will raise an error. But, a user defined __add__ could  
have one! And what would such a user defined __add__.__setcall__  
actually *do*? Well, that would be a use case, and I sure don't have  
any of those!

Ta Da. Who's going to make the patch? ;)

James

From guido at python.org  Fri Aug 11 06:58:51 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 21:58:51 -0700
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <5676E6A1-1FFE-40FC-9A99-90D82B2FE900@fuhm.net>
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
	<1D5D4538-399F-4442-AF38-32B632F05EC0@fuhm.net>
	<5.1.1.6.0.20060810164933.02713da8@sparrow.telecommunity.com>
	<5676E6A1-1FFE-40FC-9A99-90D82B2FE900@fuhm.net>
Message-ID: <ca471dc20608102158n61fe9ca5rb5677713e39a5d6b@mail.gmail.com>

On 8/10/06, James Y Knight <foom at fuhm.net> wrote:
> On Aug 10, 2006, at 4:57 PM, Phillip J. Eby wrote:
> > That being said, the benefit of hypergeneralizing assignment seems
> > small compared to its price.
>
> Well, it's a mostly obvious extension of an existing idea, so the
> price doesn't seem all that high.

If you still think that, you don't realize the problem with hypergeneralization.

The extension is *not* obvious, and there are several problems (as I
pointed out before).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Aug 11 07:47:50 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 10 Aug 2006 22:47:50 -0700
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <44DC0FB6.2040206@ieee.org>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>
	<ebe6lk$1tk$1@sea.gmane.org>
	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>
	<44DB4023.5090904@gmail.com>
	<ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>
	<ebftlt$buf$1@sea.gmane.org>
	<ca471dc20608101444x744e4e2dkb52e2d747f1cbc3a@mail.gmail.com>
	<44DC0FB6.2040206@ieee.org>
Message-ID: <ca471dc20608102247l4ea551c8x7b55313e256e4717@mail.gmail.com>

(adding back python-dev in the CC: list)

Right. I guess I didn't recompile after patching. Silly me (as Orlijn
would say :-).

Neal+Anthony, do you need me to review Nick's patch? If I don't have
to I'd rather pay more attention to py3k, which I've sadly neglected
in the past month (apart from giving a talk about it on at least 4
different occasions :-).

--Guido

On 8/10/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
> Guido van Rossum wrote:
> > Thanks for your understanding.
> >
> > Anyway, Nick's patch differs in at least one significant way from my
> > proposal -- (10**10).__index__() returns sys.maxint in his patch,
> > while I want it to return 10000000000L. So this is still an open
> > issue.
>
> Just to clarify.  I installed Nick's recent patch and it seems that
> (10**10).__index__() actually returns what Guido wants so I think we are
> all on the same page.
>
> Python 2.5b3 (trunk:51199M, Aug 10 2006, 21:59:31)
> [GCC 3.4.1 (Mandrakelinux 10.1 3.4.1-4mdk)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> (10**10).__index__()
> 10000000000L
>
>
> -Travis
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Fri Aug 11 10:28:09 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 11 Aug 2006 10:28:09 +0200
Subject: [Python-Dev] Is this a bug?
In-Reply-To: <ebcs51$tgo$1@sea.gmane.org>
References: <ebcs51$tgo$1@sea.gmane.org>
Message-ID: <ebhf2p$hlm$1@sea.gmane.org>

Georg Brandl wrote:
> Is this considered a bug? Sure, deleting modules from sys.modules
> isn't quite common, but it happened to me on one occasion.
> 
> Python 2.4.3 (#1, Jul 29 2006, 10:52:20)
>  >>> import logging
>  >>> import sys
>  >>> del logging
>  >>> del sys.modules['logging']
>  >>> ^D
> Error in atexit._run_exitfuncs:
> Traceback (most recent call last):
>    File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs
>      func(*targs, **kargs)
>    File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown
>      for h in _handlerList[:]: # was _handlers.keys():
> TypeError: unsubscriptable object
> Error in sys.exitfunc:
> Traceback (most recent call last):
>    File "/usr/lib/python2.4/atexit.py", line 24, in _run_exitfuncs
>      func(*targs, **kargs)
>    File "/usr/lib/python2.4/logging/__init__.py", line 1328, in shutdown
>      for h in _handlerList[:]: # was _handlers.keys():
> TypeError: unsubscriptable object

I've now fixed the logging issue, but what bothers me additionally is the
duplication of tracebacks here. The problem is in atexit._run_exitfuncs:

     exc_info = None
     while _exithandlers:
         func, targs, kargs = _exithandlers.pop()
         try:
             func(*targs, **kargs)
         except SystemExit:
             exc_info = sys.exc_info()
         except:
             import traceback
             print >> sys.stderr, "Error in atexit._run_exitfuncs:"
             traceback.print_exc()
             exc_info = sys.exc_info()

     if exc_info is not None:
         raise exc_info[0], exc_info[1], exc_info[2]


So the last exception is always reraised and therefore also printed
by call_sys_exitfunc. Is this really wanted behavior?

Georg


From oliphant.travis at ieee.org  Fri Aug 11 09:51:04 2006
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Fri, 11 Aug 2006 01:51:04 -0600
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <ca471dc20608102247l4ea551c8x7b55313e256e4717@mail.gmail.com>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>	<ebe6lk$1tk$1@sea.gmane.org>	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>	<44DB4023.5090904@gmail.com>	<ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>	<ebftlt$buf$1@sea.gmane.org>	<ca471dc20608101444x744e4e2dkb52e2d747f1cbc3a@mail.gmail.com>	<44DC0FB6.2040206@ieee.org>
	<ca471dc20608102247l4ea551c8x7b55313e256e4717@mail.gmail.com>
Message-ID: <44DC36E8.6040507@ieee.org>


Here is my C-API proposal

1) PyIndex_Check(obj)

   Similar to PyIter_Check(obj) as it just checks for whether or not the 
object can call nb_index.  Actually implemented as a macro.

2) PyObject* PyNumber_Index(obj)

   Simple interface around nb_index that calls it if possible and returns
   TypeError if not (or if the result does not end up in an exact
   int-or-long

3) Py_ssize_t PyNumber_AsSsize_t(obj, err)

    This converts the object to a Py_ssize_t by using the nb_index call
    if possible.  The err argument is a Python error object and specifies
    what error should be raised should the conversion from Int-or-Long to
    Py_ssize_t cause Overflow.

    If err is NULL, then no error will be raised and the result will be 
clipped.  Other-wise the corresponding error will be set.  Internally 
PyExc_IndexError and PyExc_OverflowError will be the errors used.


-Travis







From tomerfiliba at gmail.com  Fri Aug 11 11:24:26 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Fri, 11 Aug 2006 11:24:26 +0200
Subject: [Python-Dev] PyThreadState_SetAsyncExc bug?
Message-ID: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com>

while working on a library for raising exceptions in the context
of another thread, i've come across a bug in PyThreadState_SetAsyncExc.
if i raise an instance, sys.exc_info() confuses the exception value for
the exception type, and the exception value is set None. if i raise the
type itself, the interpreter creates an instance internally, but then i can't
pass arguments to the exception.

code:
=====================================
import threading
import ctypes


def _async_raise(tid, excobj):
    res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid,
ctypes.py_object(excobj))
    if res == 0:
        raise ValueError("nonexistent thread id")
    elif res > 1:
        # """if it returns a number greater than one, you're in trouble,
        # and you should call it again with exc=NULL to revert the effect"""
        ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None)
        raise SystemError("PyThreadState_SetAsyncExc failed")

class Thread(threading.Thread):
    def raise_exc(self, excobj):
        assert self.isAlive(), "thread must be started"
        for tid, tobj in threading._active.items():
            if tobj is self:
                _async_raise(tid, excobj)
                break

        # the thread was alive when we entered the loop, but was not found
        # in the dict, hence it must have been already terminated.
should we raise
        # an exception here? silently ignore?

    def terminate(self):
        self.raise_exc(SystemExit())

if __name__ == "__main__":
    import time
    import sys

    i_am_active = False

    def f():
        global i_am_active
        i_am_active = True
        try:
            try:
                while True:
                    time.sleep(0.01)
            except IOError, ex:
                print "IOError handler"
            except TypeError, ex:
                print "TypeError handler"
                print "ex=", repr(ex)
                typ, val, tb = sys.exc_info()
                print "typ=", repr(typ)
                print "val=", repr(val)
                print "tb=", tb
        finally:
            i_am_active = False

    t1 = Thread(target = f)
    t1.start()
    time.sleep(1)
    t1.raise_exc(TypeError("blah blah"))
    while i_am_active:
        time.sleep(0.01)
    print "!! thread terminated"

output:
=====================================
TypeError handler
ex= None
typ= <exceptions.TypeError instance at 0x00C15D28>  # should be the type
val= None # should be the instance
tb= <traceback object at 0x00C159E0>
!! thread terminated

if i change:
t1.raise_exc(TypeError("blah blah"))

to:
t1.raise_exc(TypeError)

i get:
=====================================
TypeError handler
ex= <exceptions.TypeError instance at 0x00C159B8>
typ= <class exceptions.TypeError at 0x00B945A0>
val= <exceptions.TypeError instance at 0x00C159B8>
tb= <traceback object at 0x00C15D00>
!! thread terminated

but then of course i can't pass arguments to the exception



-tomer

From arigo at tunes.org  Fri Aug 11 11:27:57 2006
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 11 Aug 2006 11:27:57 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <ca471dc20608101436n1a63f74sfe1c3f4d9e090059@mail.gmail.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
	<44DAF23A.103@egenix.com>
	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>
	<44DB84EE.8020606@v.loewis.de>
	<20060810211254.GA24181@code0.codespeak.net>
	<ca471dc20608101436n1a63f74sfe1c3f4d9e090059@mail.gmail.com>
Message-ID: <20060811092757.GA27929@code0.codespeak.net>

Hi,

On Thu, Aug 10, 2006 at 02:36:16PM -0700, Guido van Rossum wrote:
> > On Thu, Aug 10, 2006 at 09:11:42PM +0200, "Martin v. L?wis" wrote:
> > > I'm in favour of having this __eq__ just return False. I don't think
> > > the warning is necessary, (...)
> >
> > +1
> 
> Can you explain why you believe that no warning is necessary?

Ah... mostly out of ignorance, I fear.  I did not realize there were
strange cases like u"\xff" == "\xff".  I will leave this to more
unicode-minded people to decide, but I'm still definitely of the idea
that the __eq__ should not raise an exception.


A bientot,

Armin

From tomerfiliba at gmail.com  Fri Aug 11 11:27:32 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Fri, 11 Aug 2006 11:27:32 +0200
Subject: [Python-Dev] PyThreadState_SetAsyncExc bug?
In-Reply-To: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com>
References: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com>
Message-ID: <1d85506f0608110227l283485bei1815d624efe3fe56@mail.gmail.com>

opened a new bug:
http://sourceforge.net/tracker/index.php?func=detail&aid=1538556&group_id=5470&atid=105470




On 8/11/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> while working on a library for raising exceptions in the context
> of another thread, i've come across a bug in PyThreadState_SetAsyncExc.
> if i raise an instance, sys.exc_info() confuses the exception value for
> the exception type, and the exception value is set None. if i raise the
> type itself, the interpreter creates an instance internally, but then i can't
> pass arguments to the exception.
>
> code:
> =====================================
> import threading
> import ctypes
>
>
> def _async_raise(tid, excobj):
>     res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid,
> ctypes.py_object(excobj))
>     if res == 0:
>         raise ValueError("nonexistent thread id")
>     elif res > 1:
>         # """if it returns a number greater than one, you're in trouble,
>         # and you should call it again with exc=NULL to revert the effect"""
>         ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None)
>         raise SystemError("PyThreadState_SetAsyncExc failed")
>
> class Thread(threading.Thread):
>     def raise_exc(self, excobj):
>         assert self.isAlive(), "thread must be started"
>         for tid, tobj in threading._active.items():
>             if tobj is self:
>                 _async_raise(tid, excobj)
>                 break
>
>         # the thread was alive when we entered the loop, but was not found
>         # in the dict, hence it must have been already terminated.
> should we raise
>         # an exception here? silently ignore?
>
>     def terminate(self):
>         self.raise_exc(SystemExit())
>
> if __name__ == "__main__":
>     import time
>     import sys
>
>     i_am_active = False
>
>     def f():
>         global i_am_active
>         i_am_active = True
>         try:
>             try:
>                 while True:
>                     time.sleep(0.01)
>             except IOError, ex:
>                 print "IOError handler"
>             except TypeError, ex:
>                 print "TypeError handler"
>                 print "ex=", repr(ex)
>                 typ, val, tb = sys.exc_info()
>                 print "typ=", repr(typ)
>                 print "val=", repr(val)
>                 print "tb=", tb
>         finally:
>             i_am_active = False
>
>     t1 = Thread(target = f)
>     t1.start()
>     time.sleep(1)
>     t1.raise_exc(TypeError("blah blah"))
>     while i_am_active:
>         time.sleep(0.01)
>     print "!! thread terminated"
>
> output:
> =====================================
> TypeError handler
> ex= None
> typ= <exceptions.TypeError instance at 0x00C15D28>  # should be the type
> val= None # should be the instance
> tb= <traceback object at 0x00C159E0>
> !! thread terminated
>
> if i change:
> t1.raise_exc(TypeError("blah blah"))
>
> to:
> t1.raise_exc(TypeError)
>
> i get:
> =====================================
> TypeError handler
> ex= <exceptions.TypeError instance at 0x00C159B8>
> typ= <class exceptions.TypeError at 0x00B945A0>
> val= <exceptions.TypeError instance at 0x00C159B8>
> tb= <traceback object at 0x00C15D00>
> !! thread terminated
>
> but then of course i can't pass arguments to the exception
>
>
>
> -tomer
>

From mal at egenix.com  Fri Aug 11 11:48:25 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 11 Aug 2006 11:48:25 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <20060811092757.GA27929@code0.codespeak.net>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>	<44DAF23A.103@egenix.com>	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>	<44DB84EE.8020606@v.loewis.de>	<20060810211254.GA24181@code0.codespeak.net>	<ca471dc20608101436n1a63f74sfe1c3f4d9e090059@mail.gmail.com>
	<20060811092757.GA27929@code0.codespeak.net>
Message-ID: <44DC5269.9030508@egenix.com>

Armin Rigo wrote:
> Hi,
> 
> On Thu, Aug 10, 2006 at 02:36:16PM -0700, Guido van Rossum wrote:
>>> On Thu, Aug 10, 2006 at 09:11:42PM +0200, "Martin v. L?wis" wrote:
>>>> I'm in favour of having this __eq__ just return False. I don't think
>>>> the warning is necessary, (...)
>>> +1
>> Can you explain why you believe that no warning is necessary?
> 
> Ah... mostly out of ignorance, I fear.  I did not realize there were
> strange cases like u"\xff" == "\xff". 

This is not all that strange. Perhaps this example looks closer
to life ;-)

if u'D\xfcsseldorf' != 'D\xfcsseldorf':
    move_to_Cologne()

Now, we wouldn't want that to go through without a warning,
do we... ;-)

[Background: there's a bit of rivalry between those two cities;
 see e.g. see http://en.wikipedia.org/wiki/D%C3%BCsseldorf]

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 11 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From ndbecker2 at gmail.com  Fri Aug 11 12:31:22 2006
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 11 Aug 2006 06:31:22 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
Message-ID: <ebhmaj$60e$1@sea.gmane.org>

Guido van Rossum wrote:

> On 8/10/06, James Y Knight <foom at fuhm.net> wrote:
>> It makes just as much sense as assigning to an array access, and the
>> semantics would be pretty similar.
> 
> No. Array references (x[i]) and attribute references (x.a) represent
> "locations". Function calls represent values. This is no different
> than the distinction between lvalues and rvalues in C.
> 

Except this syntax is valid in c++ where X() is a constructor call:

X(whatever) += 2; is (or can be) valid c++


From oliphant.travis at ieee.org  Fri Aug 11 12:53:48 2006
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Fri, 11 Aug 2006 04:53:48 -0600
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <44DC36E8.6040507@ieee.org>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>	<ebe6lk$1tk$1@sea.gmane.org>	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>	<44DB4023.5090904@gmail.com>	<ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>	<ebftlt$buf$1@sea.gmane.org>	<ca471dc20608101444x744e4e2dkb52e2d747f1cbc3a@mail.gmail.com>	<44DC0FB6.2040206@ieee.org>	<ca471dc20608102247l4ea551c8x7b55313e256e4717@mail.gmail.com>
	<44DC36E8.6040507@ieee.org>
Message-ID: <ebhnjs$blv$1@sea.gmane.org>

Travis E. Oliphant wrote:
> Here is my C-API proposal
> 
> 1) PyIndex_Check(obj)
> 
>    Similar to PyIter_Check(obj) as it just checks for whether or not the 
> object can call nb_index.  Actually implemented as a macro.
> 
> 2) PyObject* PyNumber_Index(obj)
> 
>    Simple interface around nb_index that calls it if possible and returns
>    TypeError if not (or if the result does not end up in an exact
>    int-or-long
> 
> 3) Py_ssize_t PyNumber_AsSsize_t(obj, err)
> 
>     This converts the object to a Py_ssize_t by using the nb_index call
>     if possible.  The err argument is a Python error object and specifies
>     what error should be raised should the conversion from Int-or-Long to
>     Py_ssize_t cause Overflow.
> 
>     If err is NULL, then no error will be raised and the result will be 
> clipped.  Other-wise the corresponding error will be set.  Internally 
> PyExc_IndexError and PyExc_OverflowError will be the errors used.
> 

This proposal is implemented as patch 1538606 
http://sourceforge.net/tracker/index.php?func=detail&aid=1538606&group_id=5470&atid=305470


-Travis


From ncoghlan at gmail.com  Fri Aug 11 13:44:26 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 11 Aug 2006 21:44:26 +1000
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <ebhnjs$blv$1@sea.gmane.org>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>	<ebe6lk$1tk$1@sea.gmane.org>	<ca471dc20608092108p3ad07fdei8f01e0e244f1f7a6@mail.gmail.com>	<44DB4023.5090904@gmail.com>	<ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>	<ebftlt$buf$1@sea.gmane.org>	<ca471dc20608101444x744e4e2dkb52e2d747f1cbc3a@mail.gmail.com>	<44DC0FB6.2040206@ieee.org>	<ca471dc20608102247l4ea551c8x7b55313e256e4717@mail.gmail.com>	<44DC36E8.6040507@ieee.org>
	<ebhnjs$blv$1@sea.gmane.org>
Message-ID: <44DC6D9A.7060401@gmail.com>

Travis E. Oliphant wrote:
> Travis E. Oliphant wrote:
>> Here is my C-API proposal
>>
>> 1) PyIndex_Check(obj)
>>
>>    Similar to PyIter_Check(obj) as it just checks for whether or not the 
>> object can call nb_index.  Actually implemented as a macro.
>>
>> 2) PyObject* PyNumber_Index(obj)
>>
>>    Simple interface around nb_index that calls it if possible and returns
>>    TypeError if not (or if the result does not end up in an exact
>>    int-or-long
>>
>> 3) Py_ssize_t PyNumber_AsSsize_t(obj, err)
>>
>>     This converts the object to a Py_ssize_t by using the nb_index call
>>     if possible.  The err argument is a Python error object and specifies
>>     what error should be raised should the conversion from Int-or-Long to
>>     Py_ssize_t cause Overflow.
>>
>>     If err is NULL, then no error will be raised and the result will be 
>> clipped.  Other-wise the corresponding error will be set.  Internally 
>> PyExc_IndexError and PyExc_OverflowError will be the errors used.
>>
> 
> This proposal is implemented as patch 1538606 
> http://sourceforge.net/tracker/index.php?func=detail&aid=1538606&group_id=5470&atid=305470

This interface is a lot simpler to explain and use than the one in my patch, 
and more in tune with the rest of the C API.

I had a minor documentation suggestion which I put on the tracker item, but 
other than that +1 on using Travis's patch instead of mine (I've already 
rejected my own tracker item).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From tim.peters at gmail.com  Fri Aug 11 13:49:53 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 11 Aug 2006 07:49:53 -0400
Subject: [Python-Dev] PyThreadState_SetAsyncExc bug?
In-Reply-To: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com>
References: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com>
Message-ID: <1f7befae0608110449o3ca0f5a6gf5edb59a7490705e@mail.gmail.com>

[tomer filiba]
> while working on a library for raising exceptions in the context
> of another thread, i've come across a bug in PyThreadState_SetAsyncExc.
> if i raise an instance, sys.exc_info() confuses the exception value for
> the exception type, and the exception value is set None. if i raise the
> type itself, the interpreter creates an instance internally, but then i can't
> pass arguments to the exception.

That appears to be the way it was designed; i.e., AFAICT, it's working
as intended.  This follows from the code in ceval.c that raises the
exception:

				if (tstate->async_exc != NULL) {
					x = tstate->async_exc;
					tstate->async_exc = NULL;
					PyErr_SetNone(x);
					Py_DECREF(x);
					why = WHY_EXCEPTION;
					goto on_error;
				}

PyErr_SetNone(x) there gives no possibility that setting an /instance/
could work as you hope -- `x` has to be an exception type, and
tstate->async_exc is simply the `exc` argument that was passed to
PyThreadState_SetAsyncExc().

From tomerfiliba at gmail.com  Fri Aug 11 14:08:39 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Fri, 11 Aug 2006 14:08:39 +0200
Subject: [Python-Dev] PyThreadState_SetAsyncExc bug?
In-Reply-To: <1f7befae0608110449o3ca0f5a6gf5edb59a7490705e@mail.gmail.com>
References: <1d85506f0608110224s7a398788g1bbab4c97c7979c9@mail.gmail.com>
	<1f7befae0608110449o3ca0f5a6gf5edb59a7490705e@mail.gmail.com>
Message-ID: <1d85506f0608110508n148670aq973e81feab4ad245@mail.gmail.com>

so it should be fixed, or at least checked for conformness by the code.


-tomer

On 8/11/06, Tim Peters <tim.peters at gmail.com> wrote:
>
> [tomer filiba]
> > while working on a library for raising exceptions in the context
> > of another thread, i've come across a bug in PyThreadState_SetAsyncExc.
> > if i raise an instance, sys.exc_info() confuses the exception value for
> > the exception type, and the exception value is set None. if i raise the
> > type itself, the interpreter creates an instance internally, but then i
> can't
> > pass arguments to the exception.
>
> That appears to be the way it was designed; i.e., AFAICT, it's working
> as intended.  This follows from the code in ceval.c that raises the
> exception:
>
>                                 if (tstate->async_exc != NULL) {
>                                         x = tstate->async_exc;
>                                         tstate->async_exc = NULL;
>                                         PyErr_SetNone(x);
>                                         Py_DECREF(x);
>                                         why = WHY_EXCEPTION;
>                                         goto on_error;
>                                 }
>
> PyErr_SetNone(x) there gives no possibility that setting an /instance/
> could work as you hope -- `x` has to be an exception type, and
> tstate->async_exc is simply the `exc` argument that was passed to
> PyThreadState_SetAsyncExc().
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060811/93b5d7e3/attachment.html 

From mcherm at mcherm.com  Fri Aug 11 14:37:06 2006
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri, 11 Aug 2006 05:37:06 -0700
Subject: [Python-Dev] Dict suppressing exceptions
Message-ID: <20060811053706.ljvf9ge314gsg4g0@login.werra.lunarpages.com>

Guido writes:
> Alas, I have no idea what it does. Can you come up with an example
> that doesn't require enums and localization?

Sorry. Here's the short version:

Fact 1: Sometimes people create objects that raise exceptions when
compared for equality. Maybe it's a bad idea to do this, and objects
should never raise exceptions when compared except to signal internal
inconsistancy -- but the practice is common enough to show up in a
5-star recipe for enums in the Cookbook.

Fact 2: Unrelated objects are often put in dictionarys together. I
used a "localization dictionary" as an example. I feel this is a
legitimate practice.

If dictionarys stop suppressing exceptions raised by equality tests
then those two facts aren't compatible. Programs that used to work
fine will break with 2.4. I'm OK with your proposed solution (fix
the case of str-unicode but let everyone else using objects that
raise exceptions suffer), but I think it would be more friendly to
use warnings for now and exceptions after another full release
cycle. The warnings should solve the underlying issue (hard-to-debug
problems).

-- Michael Chermside


From guido at python.org  Fri Aug 11 16:56:51 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 11 Aug 2006 07:56:51 -0700
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <20060811092757.GA27929@code0.codespeak.net>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>
	<44DAF23A.103@egenix.com>
	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>
	<44DB84EE.8020606@v.loewis.de>
	<20060810211254.GA24181@code0.codespeak.net>
	<ca471dc20608101436n1a63f74sfe1c3f4d9e090059@mail.gmail.com>
	<20060811092757.GA27929@code0.codespeak.net>
Message-ID: <ca471dc20608110756v497aa93aodb292a23abdf6a8@mail.gmail.com>

On 8/11/06, Armin Rigo <arigo at tunes.org> wrote:
> Hi,
>
> On Thu, Aug 10, 2006 at 02:36:16PM -0700, Guido van Rossum wrote:
> > > On Thu, Aug 10, 2006 at 09:11:42PM +0200, "Martin v. L?wis" wrote:
> > > > I'm in favour of having this __eq__ just return False. I don't think
> > > > the warning is necessary, (...)
> > >
> > > +1
> >
> > Can you explain why you believe that no warning is necessary?
>
> Ah... mostly out of ignorance, I fear.  I did not realize there were
> strange cases like u"\xff" == "\xff".  I will leave this to more
> unicode-minded people to decide, but I'm still definitely of the idea
> that the __eq__ should not raise an exception.

Me too, and that's what we'll do in py3k. But in 2.5, we're bound by
the decisions we made in 1999-2000 about unicode. (Unless Martin has a
convincing reason not to have a warning?)

Marc-Andre, how's the patch coming along?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Aug 11 16:57:46 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 11 Aug 2006 07:57:46 -0700
Subject: [Python-Dev] __index__ clipping
In-Reply-To: <44DC6D9A.7060401@gmail.com>
References: <ca471dc20608091446m2ebd72adn74f48d4ba20b1b50@mail.gmail.com>
	<44DB4023.5090904@gmail.com>
	<ca471dc20608100726p41403f52w8e6c1444358062e5@mail.gmail.com>
	<ebftlt$buf$1@sea.gmane.org>
	<ca471dc20608101444x744e4e2dkb52e2d747f1cbc3a@mail.gmail.com>
	<44DC0FB6.2040206@ieee.org>
	<ca471dc20608102247l4ea551c8x7b55313e256e4717@mail.gmail.com>
	<44DC36E8.6040507@ieee.org> <ebhnjs$blv$1@sea.gmane.org>
	<44DC6D9A.7060401@gmail.com>
Message-ID: <ca471dc20608110757q19186e5eh367d9552e180b607@mail.gmail.com>

OK, I'll withdraw from this discussion again. Thanks all!

On 8/11/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Travis E. Oliphant wrote:
> > Travis E. Oliphant wrote:
> >> Here is my C-API proposal
> >>
> >> 1) PyIndex_Check(obj)
> >>
> >>    Similar to PyIter_Check(obj) as it just checks for whether or not the
> >> object can call nb_index.  Actually implemented as a macro.
> >>
> >> 2) PyObject* PyNumber_Index(obj)
> >>
> >>    Simple interface around nb_index that calls it if possible and returns
> >>    TypeError if not (or if the result does not end up in an exact
> >>    int-or-long
> >>
> >> 3) Py_ssize_t PyNumber_AsSsize_t(obj, err)
> >>
> >>     This converts the object to a Py_ssize_t by using the nb_index call
> >>     if possible.  The err argument is a Python error object and specifies
> >>     what error should be raised should the conversion from Int-or-Long to
> >>     Py_ssize_t cause Overflow.
> >>
> >>     If err is NULL, then no error will be raised and the result will be
> >> clipped.  Other-wise the corresponding error will be set.  Internally
> >> PyExc_IndexError and PyExc_OverflowError will be the errors used.
> >>
> >
> > This proposal is implemented as patch 1538606
> > http://sourceforge.net/tracker/index.php?func=detail&aid=1538606&group_id=5470&atid=305470
>
> This interface is a lot simpler to explain and use than the one in my patch,
> and more in tune with the rest of the C API.
>
> I had a minor documentation suggestion which I put on the tracker item, but
> other than that +1 on using Travis's patch instead of mine (I've already
> rejected my own tracker item).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>              http://www.boredomandlaziness.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Aug 11 16:59:34 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 11 Aug 2006 07:59:34 -0700
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ebhmaj$60e$1@sea.gmane.org>
References: <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<20060810084217.1914.JCARLSON@uci.edu>
	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>
	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
	<ebhmaj$60e$1@sea.gmane.org>
Message-ID: <ca471dc20608110759l6f517cdao2f0f1788300e0b09@mail.gmail.com>

On 8/11/06, Neal Becker <ndbecker2 at gmail.com> wrote:
> Guido van Rossum wrote:
>
> > On 8/10/06, James Y Knight <foom at fuhm.net> wrote:
> >> It makes just as much sense as assigning to an array access, and the
> >> semantics would be pretty similar.
> >
> > No. Array references (x[i]) and attribute references (x.a) represent
> > "locations". Function calls represent values. This is no different
> > than the distinction between lvalues and rvalues in C.
> >
>
> Except this syntax is valid in c++ where X() is a constructor call:
>
> X(whatever) += 2; is (or can be) valid c++

As I said before, C++ has a fundamentally different concept of what
assignment means; it is of no use for understanding Python's
assignment. Actually it is a big hindrance knowing about C++
assignment because it's difficult to explain to C++ users why Python
can't and won't allow assignment to be overloaded.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From chrisspen at gmail.com  Fri Aug 11 17:15:25 2006
From: chrisspen at gmail.com (Chris S)
Date: Fri, 11 Aug 2006 11:15:25 -0400
Subject: [Python-Dev] Elementtree and Namespaces in 2.5
Message-ID: <beb7688e0608110815k44369b92tdd4dbcecd00b1a8@mail.gmail.com>

I'm happy to see Elementtree being considered for inclusion with 2.5.
However, before committing to this decision, there's an issue
regarding it's namespace parsing that should be addressed. Although
Elmenttree is in most respects an excellent XML parser, a huge gotcha
that often makes Elementtree unsuitable for many applications lies in
the way it arbitrarily renames namespaces.

For example, given:

<h:html xmlns:xdc="http://www.xml.com/books"
        xmlns:h="http://www.w3.org/HTML/1998/html4">
 <h:head><h:title>Book Review</h:title></h:head>
 <h:body>
  <xdc:bookreview>
   <xdc:title>XML: A Primer</xdc:title>
   <h:table>
    <h:tr align="center">
     <h:td>Author</h:td><h:td>Price</h:td>
     <h:td>Pages</h:td><h:td>Date</h:td></h:tr>
    <h:tr align="left">
     <h:td><xdc:author>Simon St. Laurent</xdc:author></h:td>
     <h:td><xdc:price>31.98</xdc:price></h:td>
     <h:td><xdc:pages>352</xdc:pages></h:td>
     <h:td><xdc:date>1998/01</xdc:date></h:td>
    </h:tr>
   </h:table>
  </xdc:bookreview>
 </h:body>
</h:html>

Elementtree would rewrite this as:

<ns0:html xmlns:ns0="http://www.w3.org/HTML/1998/html4">
 <ns0:head><ns0:title>Book Review</ns0:title></ns0:head>
 <ns0:body>
  <ns1:bookreview xmlns:ns1="http://www.xml.com/books">
   <ns1:title>XML: A Primer</ns1:title>
   <ns0:table>
    <ns0:tr align="center">
     <ns0:td>Author</ns0:td><ns0:td>Price</ns0:td>
     <ns0:td>Pages</ns0:td><ns0:td>Date</ns0:td></ns0:tr>
    <ns0:tr align="left">
     <ns0:td><ns1:author>Simon St. Laurent</ns1:author></ns0:td>
     <ns0:td><ns1:price>31.98</ns1:price></ns0:td>
     <ns0:td><ns1:pages>352</ns1:pages></ns0:td>
     <ns0:td><ns1:date>1998/01</ns1:date></ns0:td>
    </ns0:tr>
   </ns0:table>
  </ns1:bookreview>
 </ns0:body>
</ns0:html>

There's been some discussion in comp.lang.python about this
functionality (http://groups.google.com/group/comp.lang.python/browse_thread/thread/31b2e9f4a8f7338c/363f46513fb8de04?&rnum=3&hl=en)
and while most users and the w3 spec
(http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting)
agree this feature is actually a bug, Fredrik Lundh has refused to fix
this problem. Of course, this is his right. Unfortunately,
Elementtree's design makes a work-around rather awkward. Therefore, we
might want to rethink inclusion of Elementtree in the stdlib, or at
least patch the stdlib's version of Elementtree to produce an output
more in line with the w3 standard.

Sincerely,
Chris Spencer

From mal at egenix.com  Fri Aug 11 18:10:31 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 11 Aug 2006 18:10:31 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <ca471dc20608110756v497aa93aodb292a23abdf6a8@mail.gmail.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>	<44DAF23A.103@egenix.com>	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>	<44DB84EE.8020606@v.loewis.de>	<20060810211254.GA24181@code0.codespeak.net>	<ca471dc20608101436n1a63f74sfe1c3f4d9e090059@mail.gmail.com>	<20060811092757.GA27929@code0.codespeak.net>
	<ca471dc20608110756v497aa93aodb292a23abdf6a8@mail.gmail.com>
Message-ID: <44DCABF7.6040709@egenix.com>

Guido van Rossum wrote:
> Marc-Andre, how's the patch coming along?

I'm working on it.

Since we only want equal compares to generate the warning,
I have to add a rich compare function to Unicode objects.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 11 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mcherm at mcherm.com  Fri Aug 11 19:38:25 2006
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri, 11 Aug 2006 10:38:25 -0700
Subject: [Python-Dev] Elementtree and Namespaces in 2.5
Message-ID: <20060811103825.2s08k51nxnccgcwg@login.werra.lunarpages.com>

Chris Spencer writes:
> there's an issue
> regarding [ElementTree's] namespace parsing that should be addressed.
     [... it performs namespace rewriting ...]
> while most users and the w3 spec
> (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting)
> agree this feature is actually a bug, Fredrik Lundh has refused to fix
> this problem. Of course, this is his right. Unfortunately,
> Elementtree's design makes a work-around rather awkward. Therefore, we
> might want to rethink inclusion of Elementtree in the stdlib, or at
> least patch the stdlib's version of Elementtree to produce an output
> more in line with the w3 standard.

The good news for you is that:
  (1) Including ElementTree in the stdlib does not (immediately) take
     away from any of the other XML librarys out there.
  (2) Including ElementTree in the stdlib *increases* the chance that
     requests from users will lead to a change in the library.
  (3) Changing ElementTree to preserve namespace prefixes would be
     backward compatible with a version that didn't preserve it, so
     nothing happening now forecloses fixing this in future releases.

Now, in my opinion, the W3 flubbed badly in the way they designed
namespaces, prefixes, and allowing namespace prefixes to appear
within element content and attributes. Creating a universal namespace
with local aliases to prevent name clashes is nice, making the local
aliases significant so that we haven't prevented name clashes after
all but have merely introduced vast complexity (and *encouraged*
clashes by shortening the names) is... not so nice.

But that's beside the point. Even if we presume 100% agreement on
the need to change ElementTree, the best thing to do is still to
release ElementTree in the stdlib exactly as it is in 2.5 and change
it for 2.6.

-- Michael Chermside


From martin at v.loewis.de  Fri Aug 11 19:47:44 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Aug 2006 19:47:44 +0200
Subject: [Python-Dev] openSSL and windows binaries - license
In-Reply-To: <44DBCD3E.8070304@canterbury.ac.nz>
References: <fb6fbf560608081009t13eaad8emb332e07ddb76d2b1@mail.gmail.com>
	<44D8E38C.8000505@v.loewis.de>
	<44D9503D.1040809@canterbury.ac.nz> <44D983E3.701@v.loewis.de>
	<44DA8D43.4040009@canterbury.ac.nz> <44DAE224.50705@v.loewis.de>
	<44DBCD3E.8070304@canterbury.ac.nz>
Message-ID: <44DCC2C0.1060703@v.loewis.de>

Greg Ewing schrieb:
> That can't be right, because it would mean that
> anyone who runs a program that contains a
> patented algorithm, even one bought or otherwise
> obtained from someone else, would need to
> individually negotiate a licence with the
> patent owner. That clearly doesn't happen.

No, that isn't necessary. If you buy a patented screwdriver,
and the maker of the screwdriver isn't the patent owner
but has an appropriate license, you don't need to negotiate
with the patent owner to use the screwdriver.

Same with software patents: whoever sold you the software
should have negotiated a transferable license that allows
use of the algorithm in this software; transfer of the
license would likely be bound to usage within this product.

So this doesn't get completely OT: The IDEA algorithm is
licensed free of charge for non-commercial use (I believe
to anybody, ask the patent owner if uncertain); commercial
users need to buy a license (I would expect that transferable
licenses are also available for sale).

Regards,
Martin

From martin at v.loewis.de  Fri Aug 11 20:35:20 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Aug 2006 20:35:20 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <ca471dc20608110756v497aa93aodb292a23abdf6a8@mail.gmail.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>	
	<44DAF23A.103@egenix.com>	
	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>	
	<44DB84EE.8020606@v.loewis.de>	
	<20060810211254.GA24181@code0.codespeak.net>	
	<ca471dc20608101436n1a63f74sfe1c3f4d9e090059@mail.gmail.com>	
	<20060811092757.GA27929@code0.codespeak.net>
	<ca471dc20608110756v497aa93aodb292a23abdf6a8@mail.gmail.com>
Message-ID: <44DCCDE8.70506@v.loewis.de>

Guido van Rossum schrieb:
> Me too, and that's what we'll do in py3k. But in 2.5, we're bound by
> the decisions we made in 1999-2000 about unicode. (Unless Martin has a
> convincing reason not to have a warning?)

Only the general anti-warning argument: it's not the developer which
will see the warning, but the end user. In this case, they might easily
get hundreds of warning in a short time, and these fill their Apache
log files. They complain about Python eating their disk space.

Regards,
Martin

From theller at python.net  Fri Aug 11 21:01:44 2006
From: theller at python.net (Thomas Heller)
Date: Fri, 11 Aug 2006 21:01:44 +0200
Subject: [Python-Dev] Errors after running make test
Message-ID: <ebik6r$i6v$1@sea.gmane.org>

There are errors now after the testsuite has finished.  Taken from the very end of the
amd64 test log (for example).

http://www.python.org/dev/buildbot/trunk/amd64%20gentoo%20trunk/builds/1403/step-test/0

[...]
293 tests OK.
26 tests skipped:
    test_aepack test_al test_applesingle test_bsddb185 test_cd test_cl
    test_curses test_dl test_gl test_imageop test_imgfile
    test_linuxaudiodev test_macfs test_macostools test_nis
    test_ossaudiodev test_pep277 test_plistlib test_rgbimg
    test_scriptpackages test_startfile test_sunaudiodev
    test_unicode_file test_winreg test_winsound test_zipfile64
Those skips are all expected on linux2.
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/logging/__init__.py", line 1351, in shutdown
    h.flush()
  File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/logging/__init__.py", line 731, in flush
    self.stream.flush()
ValueError: I/O operation on closed file
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/logging/__init__.py", line 1351, in shutdown
    h.flush()
  File "/home/buildslave/buildslave/python/trunk.norwitz-amd64/build/Lib/logging/__init__.py", line 731, in flush
    self.stream.flush()
ValueError: I/O operation on closed file
[449107 refs]
program finished with exit code 0


From martin at v.loewis.de  Fri Aug 11 21:08:51 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Aug 2006 21:08:51 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com>
References: <20060810133514.pk8p9e2di4a0o4wg@login.werra.lunarpages.com>
Message-ID: <44DCD5C3.6040907@v.loewis.de>

Michael Chermside schrieb:
> I don't *strongly* object to this consensus, but if you haven't
> glanced at my original example, take a look - it might convince you.
> The proposed solution will not help with my example.

I ignored your example the first time because it was too complicated
to understand.

Now I looked at it, and think that the recipe is broken. It should
add an __eq__ method which is

  def __eq__(self, other):
      return type(self) is type(other) \
             and self.EnumType is other.EnumType \
             and self.__value==other.__value)

Regards,
Martin

From martin at v.loewis.de  Fri Aug 11 21:35:52 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Aug 2006 21:35:52 +0200
Subject: [Python-Dev] Elementtree and Namespaces in 2.5
In-Reply-To: <beb7688e0608110815k44369b92tdd4dbcecd00b1a8@mail.gmail.com>
References: <beb7688e0608110815k44369b92tdd4dbcecd00b1a8@mail.gmail.com>
Message-ID: <44DCDC18.3000701@v.loewis.de>

Chris S schrieb:
> I'm happy to see Elementtree being considered for inclusion with 2.5.
> However, before committing to this decision, there's an issue
> regarding it's namespace parsing that should be addressed. Although
> Elmenttree is in most respects an excellent XML parser, a huge gotcha
> that often makes Elementtree unsuitable for many applications lies in
> the way it arbitrarily renames namespaces.

Terminology alert: it doesn't rename namespaces. It renames namespace
prefixes. The namespaces stay unmodified.

> functionality (http://groups.google.com/group/comp.lang.python/browse_thread/thread/31b2e9f4a8f7338c/363f46513fb8de04?&rnum=3&hl=en)
> and while most users and the w3 spec
> (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting)
> agree this feature is actually a bug, Fredrik Lundh has refused to fix
> this problem.

You are misunderstanding. The "w3 spec" does not say it is a bug; you
are looking at the wrong spec. The right spec to look at is

http://www.w3.org/TR/REC-xml-names/

You are looking at a resolution of an issue in the spec of
Canonical XML (i.e. not even the actual spec, just the resolution
of an issue).

Now, C14N specifies that the canonical form of an XML document has
the same namespace prefixes as the original document. So ElementTree
does not implement C14N for that reason, but I guess there are many
more reasons why ElementTree is not a C14N implementation. For example,
it puts the attributes before the namespace attributes, when C14N
says it should be the other way 'round.

> Of course, this is his right. Unfortunately,
> Elementtree's design makes a work-around rather awkward. Therefore, we
> might want to rethink inclusion of Elementtree in the stdlib

That is not enough reason. Yes, it makes certain applications
impossible, e.g. when namespace prefixes are inside attribute
values. It just means you can't use it for that application,
then. XML has many other applications, and so does ElementTree.

> or at
> least patch the stdlib's version of Elementtree to produce an output
> more in line with the w3 standard.

That is out of the question. It was agreed that ElementTree is added
to the standard library only if Fredrik Lundh gets veto rights on
all changes.

Regards,
Martin

From martin at v.loewis.de  Fri Aug 11 21:40:50 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Aug 2006 21:40:50 +0200
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
References: <ebd2sp$nnv$1@sea.gmane.org>
	<ebd6od$6m4$1@sea.gmane.org>	<ebd8f0$csd$1@sea.gmane.org>
	<2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
Message-ID: <44DCDD42.1010202@v.loewis.de>

Michael Urman schrieb:
> On 8/9/06, Michael Hudson <mwh at python.net> wrote:
>> The question doesn't make sense: in Python, you assign to a name,
>> an attribute or a subscript, and that's it.
> 
> Just to play devil's advocate here, why not to a function call via a
> new __setcall__? 

Just try specifying this one day. I'm sure a dozen readers of the list
will rip apart the first specification you make as underspecified or
unimplementable (hint: what is the precise syntax for the left-hand
side? how to parse it? what is the precise parameter list?)

Regards,
Martin

From martin at v.loewis.de  Fri Aug 11 21:48:43 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Aug 2006 21:48:43 +0200
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <ebhmaj$60e$1@sea.gmane.org>
References: <2m4pwlvezn.fsf@starship.python.net>	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>	<20060810084217.1914.JCARLSON@uci.edu>	<64585004-6C07-4248-A183-F1AA354BEE89@fuhm.net>	<ca471dc20608100924v4c174c89u854775f81fe81f05@mail.gmail.com>
	<ebhmaj$60e$1@sea.gmane.org>
Message-ID: <44DCDF1B.2010804@v.loewis.de>

Neal Becker schrieb:
>> No. Array references (x[i]) and attribute references (x.a) represent
>> "locations". Function calls represent values. This is no different
>> than the distinction between lvalues and rvalues in C.
>>
> 
> Except this syntax is valid in c++ where X() is a constructor call:
> 
> X(whatever) += 2; is (or can be) valid c++

That's actually the less-interesting case. You would have to overload
+= to make it work, right?

The more interesting case is when X is a function that returns a
reference:

int& X(int);

void foo(){
  X(1) += 2;
}

int bar, foobar;
int& X(int t){
  if(t)  return bar;
  return foobar;
}

Here, which variable gets incremented depends on whether the t
argument is true; no overloading of assignment comes into play.

The trick is that C++ has functions that *return* lvalues;
neither C nor Python has such things.

Regards,
Martin

From martin at v.loewis.de  Fri Aug 11 21:52:02 2006
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 11 Aug 2006 21:52:02 +0200
Subject: [Python-Dev] Unicode Data in Python2.5 is missing a ucd_4_1_0
 object
In-Reply-To: <loom.20060810T220456-500@post.gmane.org>
References: <loom.20060810T202810-913@post.gmane.org>	<44DB838A.10600@v.loewis.de>
	<loom.20060810T220456-500@post.gmane.org>
Message-ID: <44DCDFE2.5000705@v.loewis.de>

Armin Ronacher schrieb:
> Right, I didn't know that. From that old bug report it sounded like a programmer
> should be able to select a specific UCD version.

Well, my comment was that _IDNA_ needs to access _3.2_. This isn't as
general as "any application needs to access any version". We likely drop
3.2 when IDNA stops using it, and we likely drop 4.1 when moving to 5.0.
Of course, now the infrastructure is there for keeping old versions
efficiently, so if compelling reasons are brought forward to keep an
old version, it would be possible.

Regards,
Martin

From pje at telecommunity.com  Fri Aug 11 21:53:08 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 11 Aug 2006 15:53:08 -0400
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <44DCDD42.1010202@v.loewis.de>
References: <dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<ebd2sp$nnv$1@sea.gmane.org> <ebd6od$6m4$1@sea.gmane.org>
	<ebd8f0$csd$1@sea.gmane.org> <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
Message-ID: <5.1.1.6.0.20060811154306.03be1b28@sparrow.telecommunity.com>

At 09:40 PM 8/11/2006 +0200, Martin v. L?wis wrote:
>Michael Urman schrieb:
> > On 8/9/06, Michael Hudson <mwh at python.net> wrote:
> >> The question doesn't make sense: in Python, you assign to a name,
> >> an attribute or a subscript, and that's it.
> >
> > Just to play devil's advocate here, why not to a function call via a
> > new __setcall__?
>
>Just try specifying this one day. I'm sure a dozen readers of the list
>will rip apart the first specification you make as underspecified or
>unimplementable (hint: what is the precise syntax for the left-hand
>side? how to parse it? what is the precise parameter list?)

Actually, this isn't as hard as you're implying.  In at least the 
"compiler.ast" package, such an operation would be represented as a 
CallFunc node as the child of an Assign node.  Wrapping the call node's 
main child expression in a Getattr for __setcall__ would then effect the 
relatively simple AST transformation required.

This is because Python's grammar *already* allows arbitrary LHS 
expressions; it's just that the compiler rejects LHS nodes that aren't a 
Getattr, Subscript, or Name.

(The above should not be confused with advocacy for the feature; I'm just 
pointing out that specifying it -- or even implementing it -- isn't that 
complicated.)


From martin at v.loewis.de  Fri Aug 11 22:10:20 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Aug 2006 22:10:20 +0200
Subject: [Python-Dev] SyntaxError: can't assign to function call
In-Reply-To: <5.1.1.6.0.20060811154306.03be1b28@sparrow.telecommunity.com>
References: <dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<ebd2sp$nnv$1@sea.gmane.org> <ebd6od$6m4$1@sea.gmane.org>
	<ebd8f0$csd$1@sea.gmane.org> <2m4pwlvezn.fsf@starship.python.net>
	<dcbbbb410608100652p6092c36fud191672de7f5ca54@mail.gmail.com>
	<5.1.1.6.0.20060811154306.03be1b28@sparrow.telecommunity.com>
Message-ID: <44DCE42C.6030708@v.loewis.de>

Phillip J. Eby schrieb:
> Actually, this isn't as hard as you're implying.  In at least the
> "compiler.ast" package, such an operation would be represented as a
> CallFunc node as the child of an Assign node.  Wrapping the call node's
> main child expression in a Getattr for __setcall__ would then effect the
> relatively simple AST transformation required.

Well, you are experienced with that stuff, and would be one of the
people who could rip apart a specification attempt. There are just many
details to it, for example, what is the evaluation order when I write

f(g()) = h()

Or what about chained assignments

a = b() = c()

Is the return value of __setcall__ used or not?

What about augmented assignments?

Any specific question that somebody asks gets an easy (although
perhaps arbitrary) answer. Still, I guess many people would be
surprised how large the specification of a seemingly simple feature
will be.

Regards,
Martin

From jimjjewett at gmail.com  Fri Aug 11 22:22:07 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 11 Aug 2006 16:22:07 -0400
Subject: [Python-Dev] Dict suppressing exceptions
Message-ID: <fb6fbf560608111322r34415dc3q1608815415c3dfd4@mail.gmail.com>

Martin v. L?wis wrote:

> Now I looked at it, and think that the recipe is broken. It should
> add an __eq__ method which is

>   def __eq__(self, other):
>       return type(self) is type(other) \
>           and self.EnumType is other.EnumType \
>           and self.__value==other.__value)

Absolutely.  But the point is that there are a lot of these broken
objects out there; this one was on a 5-star recipe that has been
around for a while, and still no one caught it.  That suggests the bug
could reasonably be in any 3rd-party library.

The existence of a (common) bug in someone else's code shouldn't keep
me from using a dictionary of objects.

hash was just changed to support the common use of id, even though
some people argued it was *really* a bug in the classes themselves.
This is a similar situation.

-jJ

From mcherm at mcherm.com  Fri Aug 11 23:02:56 2006
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri, 11 Aug 2006 14:02:56 -0700
Subject: [Python-Dev] Dict suppressing exceptions
Message-ID: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com>

Martin v. L?wis writes:
> Now I looked at it, and think that the recipe is broken.

Not broken, but perhaps wrongheaded. The recipe went out of its way
to ensure that it would raise an exception of enum values from different
enumerations were compared. There's nothing out there saying that this
is a bad thing to do.

I propose that we institute a new policy. The policy should state:

    __eq__ methods should always return True or False. They should
    only raise an exception if there is some internal error within
    one of the objects being compared -- they should never raise
    an exception because the other object is of an unexpected type.

    On the other hand, __lt__, __gt__ and friends SHOULD raise an
    exception when the object being compared is of some type for
    which the ordering does not make sense (e.g.: unicode vs
    byte-string or complex vs anything).

I think we should note this policy someplace official -- perhaps
in the Language Reference where __eq__ and __lt__ are defined. But
I do not think that these changes should be made until Py3K.

What do others think? Is this the "right" approach?

-- Michael Chermside


From mal at egenix.com  Fri Aug 11 23:11:10 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 11 Aug 2006 23:11:10 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <44DCABF7.6040709@egenix.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>	<44DAF23A.103@egenix.com>	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>	<44DB84EE.8020606@v.loewis.de>	<20060810211254.GA24181@code0.codespeak.net>	<ca471dc20608101436n1a63f74sfe1c3f4d9e090059@mail.gmail.com>	<20060811092757.GA27929@code0.codespeak.net>	<ca471dc20608110756v497aa93aodb292a23abdf6a8@mail.gmail.com>
	<44DCABF7.6040709@egenix.com>
Message-ID: <44DCF26E.6090003@egenix.com>

M.-A. Lemburg wrote:
> Guido van Rossum wrote:
>> Marc-Andre, how's the patch coming along?
> 
> I'm working on it.
> 
> Since we only want equal compares to generate the warning,
> I have to add a rich compare function to Unicode objects.

Here's an initial version:

http://sourceforge.net/tracker/index.php?func=detail&aid=1538956&group_id=5470&atid=305470

The title of the patch is slightly incorrect - SF doesn't allow
more descriptive titles... :-(

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 11 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From martin at v.loewis.de  Fri Aug 11 23:54:19 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 11 Aug 2006 23:54:19 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com>
References: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com>
Message-ID: <44DCFC8B.3070100@v.loewis.de>

Michael Chermside schrieb:
> Not broken, but perhaps wrongheaded. The recipe went out of its way
> to ensure that it would raise an exception of enum values from different
> enumerations were compared. There's nothing out there saying that this
> is a bad thing to do.

And it's actually fine for three-way compare: different enums don't
naturally arrange.

> I propose that we institute a new policy. The policy should state:
> 
>    __eq__ methods should always return True or False. They should
>    only raise an exception if there is some internal error within
>    one of the objects being compared -- they should never raise
>    an exception because the other object is of an unexpected type.

That policy is currently difficult to implement, but reasonable
(difficult because it is quite some code to write).

>    On the other hand, __lt__, __gt__ and friends SHOULD raise an
>    exception when the object being compared is of some type for
>    which the ordering does not make sense (e.g.: unicode vs
>    byte-string or complex vs anything).

Right; same for three-way compare.

> What do others think? Is this the "right" approach?

For the moment, the first section gets augmented by "issue a
warning if you think the user is comparing things incorrectly".

Regards,
Martin

From alexander.belopolsky at gmail.com  Sat Aug 12 00:07:44 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Fri, 11 Aug 2006 18:07:44 -0400
Subject: [Python-Dev] What is the status of file.readinto?
Message-ID: <d38f5330608111507s1407a1damacf9f229fa40b6db@mail.gmail.com>

I know that it is a subject of an annual discussion, but since I could
not find any mention of it in the last year, let me ask this question
again: why is file.readinto "Undocumented"? It is quite useful,
particularly with numpy.

Would a patch adding "readinto" to StringIO be acceptable?

From ncoghlan at iinet.net.au  Sat Aug 12 04:47:10 2006
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat, 12 Aug 2006 12:47:10 +1000
Subject: [Python-Dev] 2.6 idea: a 'function' builtin to parallel classmethod
	and staticmethod
Message-ID: <44DD412E.9060109@iinet.net.au>

It's sometimes useful to be able to use an existing callable as a method of a 
new class. If the callable is a real function, this is easy. You just 
including the following line in the class definition:

   method = some_callable

However, callable objects without a function-like __get__ method can't be used 
that way. So, to avoid a dependency on an implementation detail of 
some_callable (i.e. whether or not it is a true function object), you have to 
write:


   def method():
       return some_callable()

(and you can lose useful metadata in the process!)

However, if you're adding a callable as a class method or static method, there 
is OOWTDI:

   method = classmethod(some_callable)
   method = staticmethod(some_callable)

It would be nice if there was a similar mechanism for normal instance methods 
as well:

   method = function(some_callable)

This came up during the PEP 343 implementation - "context = 
contextlib.closing" is a tempting thing to write in order to provide a 
"x.context()" method for use in a with statement, but it doesn't actually work 
properly (because closing is a class, not a function).

Similarly, you can't create a method simply by applying functools.partial to 
an existing function - the result won't have a __get__ method, so it will be 
treated like a normal attribute instead of as an instance method.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Sat Aug 12 06:03:40 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 11 Aug 2006 21:03:40 -0700
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com>
References: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com>
Message-ID: <ca471dc20608112103s798b1748y6e83bd5734d38fc1@mail.gmail.com>

On 8/11/06, Michael Chermside <mcherm at mcherm.com> wrote:
> Martin v. L?wis writes:
> > Now I looked at it, and think that the recipe is broken.
>
> Not broken, but perhaps wrongheaded. The recipe went out of its way
> to ensure that it would raise an exception of enum values from different
> enumerations were compared. There's nothing out there saying that this
> is a bad thing to do.

Except a little voice in my head. :-)

> I propose that we institute a new policy. The policy should state:
>
>     __eq__ methods should always return True or False. They should
>     only raise an exception if there is some internal error within
>     one of the objects being compared -- they should never raise
>     an exception because the other object is of an unexpected type.

That's pretty much what I want to do in py3k. There, ordering will
raise a TypeError by default, but == compares object identity by
default, and it's strongly recommended to return False for any
unrecognized type.

I want the recommendation to be a *little* more relaxed than your
proposal; IMO it's fine to raise an exception when your __eq__
implementation is incomplete, i.e. you haven't thought enough about
how your class should compare to other classes. The "don't silence
exceptions" rule will then cause loud complaints when your class gets
compared to something it didn't expect being compared to, signalling
that you have more work to do. (And yes, this can cause sleeper bombs
when you use these objects as dict keys together with types that it
doesn't understand, and the bomb will go off if there's a hash
collision; but I've got fairly controlled situations in mind, not
robust libraries or frameworks.)

>     On the other hand, __lt__, __gt__ and friends SHOULD raise an
>     exception when the object being compared is of some type for
>     which the ordering does not make sense (e.g.: unicode vs
>     byte-string or complex vs anything).

Yes, that's the py3k rule.

> I think we should note this policy someplace official -- perhaps
> in the Language Reference where __eq__ and __lt__ are defined. But
> I do not think that these changes should be made until Py3K.

Fine to update the docs now.

> What do others think? Is this the "right" approach?

Yes.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Aug 12 06:06:13 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 11 Aug 2006 21:06:13 -0700
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <44DCFC8B.3070100@v.loewis.de>
References: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com>
	<44DCFC8B.3070100@v.loewis.de>
Message-ID: <ca471dc20608112106o34636a74y4c4f832cc904d34f@mail.gmail.com>

On 8/11/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Michael Chermside schrieb:
> > I propose that we institute a new policy. The policy should state:
> >
> >    __eq__ methods should always return True or False. They should
> >    only raise an exception if there is some internal error within
> >    one of the objects being compared -- they should never raise
> >    an exception because the other object is of an unexpected type.
>
> That policy is currently difficult to implement, but reasonable
> (difficult because it is quite some code to write).

Why? Are you thinking of the standard library, or of an end user's
__eq__ method? Returning False from your __eq__ if other's type is
unexpected doesn't seem a lot of code. Or am I misunderstanding
something?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Sat Aug 12 07:57:00 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 12 Aug 2006 07:57:00 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <ca471dc20608112106o34636a74y4c4f832cc904d34f@mail.gmail.com>
References: <20060811140256.rg49nou0ojhw8cgs@login.werra.lunarpages.com>	
	<44DCFC8B.3070100@v.loewis.de>
	<ca471dc20608112106o34636a74y4c4f832cc904d34f@mail.gmail.com>
Message-ID: <44DD6DAC.9060400@v.loewis.de>

Guido van Rossum schrieb:

>> >    __eq__ methods should always return True or False. They should
>> >    only raise an exception if there is some internal error within
>> >    one of the objects being compared -- they should never raise
>> >    an exception because the other object is of an unexpected type.
>>
>> That policy is currently difficult to implement, but reasonable
>> (difficult because it is quite some code to write).
> 
> Why? Are you thinking of the standard library, or of an end user's
> __eq__ method? Returning False from your __eq__ if other's type is
> unexpected doesn't seem a lot of code. Or am I misunderstanding
> something?

It seemed like a lot of code to me: In the specific enum example,
I first wrote

  def __eq__(self, other):
    return self.EnumType is other.EnumType \
       and self.__value==other.__value

So this was wrong, as it did not take into account 'other' being
something completely different, and I wrote

  def __eq__(self, other):
    return type(self) is type(other) \
       and self.EnumType is other.EnumType \
       and self.__value==other.__value

Having two additional continuation lines seems quite difficult to
me, yet a version that breaks the expression into multiple statements
is even longer

  def __eq__(self, other):
    if type(self) is not type(other):
      return False
    if self.EnumType is not other.EnumType:
      return False
    return self.__value==other.__value

Compare this to the current two-line __cmp__ implementation:

  def __cmp__(self, other):
    assert self.Enumtype is other.EnumType
    return cmp(self.__value, other.__value)

This covers all comparisons just fine in two lines of method
body; to implement the __eq__ policy, you need another 6
lines. For consistency, you should also implement __ne__,
probably as

  def __ne__(self, other):
    return not self.__eq__(other)

I expect many people get this wrong, for example

http://pyref.infogami.com/__eq__
UserList.py

Also, there is confusion as to whether NotImplemented
should ever be returned in these. decimal.py believes
it does (for different types), sets.py believes it
doesn't.

Regards,
Martin

From martin at v.loewis.de  Sat Aug 12 08:14:04 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 12 Aug 2006 08:14:04 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <fb6fbf560608111322r34415dc3q1608815415c3dfd4@mail.gmail.com>
References: <fb6fbf560608111322r34415dc3q1608815415c3dfd4@mail.gmail.com>
Message-ID: <44DD71AC.5060402@v.loewis.de>

Jim Jewett schrieb:
> hash was just changed to support the common use of id, even though
> some people argued it was *really* a bug in the classes themselves.
> This is a similar situation.

Similar still different. In the case of __hash__ returning id(),
it is very clear what the intention of the program is, and it is
implementable to give the program its intended meaning (i.e.
use the object address as the source of the hash value).

In this case, it is not clear what the intention of the program is.
__cmp__ raises an exception, and you say the intention is that this
exception should be discarded? Errors should never pass silently,
unless explicitly silenced.

Some users will find "Oops, I didn't know all these years I had
this error there", and correct their code. Some will say "why
did they break my program", and correct their code - the corrected
code will run just fine on older versions of Python as well.

Regards,
Martin

From martin at v.loewis.de  Sat Aug 12 08:20:36 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 12 Aug 2006 08:20:36 +0200
Subject: [Python-Dev] 2.6 idea: a 'function' builtin to parallel
 classmethod and staticmethod
In-Reply-To: <44DD412E.9060109@iinet.net.au>
References: <44DD412E.9060109@iinet.net.au>
Message-ID: <44DD7334.2070003@v.loewis.de>

Nick Coghlan schrieb:
> It would be nice if there was a similar mechanism for normal instance methods 
> as well:
> 
>    method = function(some_callable)
> 

If you know you have to put something around it, what's wrong with writing

  method = lambda *args:some_callable(*args)

If that happens often enough, you can write

def function(f):
  return lambda *args:f(*args)

Regards,
Martin

From ncoghlan at gmail.com  Sat Aug 12 08:38:49 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 12 Aug 2006 16:38:49 +1000
Subject: [Python-Dev] 2.6 idea: a 'function' builtin to parallel
 classmethod and staticmethod
In-Reply-To: <44DD7334.2070003@v.loewis.de>
References: <44DD412E.9060109@iinet.net.au> <44DD7334.2070003@v.loewis.de>
Message-ID: <44DD7779.4060404@gmail.com>

Martin v. L?wis wrote:
> Nick Coghlan schrieb:
>> It would be nice if there was a similar mechanism for normal instance methods 
>> as well:
>>
>>    method = function(some_callable)
>>
> 
> If you know you have to put something around it, what's wrong with writing
> 
>   method = lambda *args:some_callable(*args)
> 
> If that happens often enough, you can write
> 
> def function(f):
>   return lambda *args:f(*args)

Both of those are inferior to defining a new method with the right name and 
arguments and a docstring directing readers to the relevant function (which is 
what I would do for this situation at the moment).

The idea of the builtin would be to let introspection tools know that the 
documentation of the existing callable applies to the method as well, so that 
things like help(x.method) could retrieve something more useful than:

 >>> help(C().method)
Help on method method in module __main__:

method(*args, **kwds) method of __main__.C instance

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From tim.peters at gmail.com  Sat Aug 12 09:24:05 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 12 Aug 2006 03:24:05 -0400
Subject: [Python-Dev] Recent logging spew
Message-ID: <1f7befae0608120024q24cd5474p5aa0a3d5f13eae0b@mail.gmail.com>

We're getting an awful lot of these turds at the ends of test runs now:

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Code\bb_slave\trunk.peters-windows\build\lib\atexit.py",
line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py",
line 1351, in shutdown
    h.flush()
  File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py",
line 731, in flush
    self.stream.flush()
ValueError: I/O operation on closed file
Error in sys.exitfunc:
Traceback (most recent call last):
  File "C:\Code\bb_slave\trunk.peters-windows\build\lib\atexit.py",
line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py",
line 1351, in shutdown
    h.flush()
  File "C:\Code\bb_slave\trunk.peters-windows\build\lib\logging\__init__.py",
line 731, in flush
    self.stream.flush()
ValueError: I/O operation on closed file
[450168 refs]
program finished with exit code 0

Not sure what caused it, but suspect this:

"""
Author: georg.brandl
Date: Fri Aug 11 09:26:10 2006
New Revision: 51206

Modified:
  python/trunk/Lib/logging/__init__.py
  python/trunk/Misc/NEWS
Log:
logging's atexit hook now runs even if the rest of the module has
already been cleaned up.
"""

Anyway, can we stop this?  It doesn't look like "a feature" from here ;-)

From g.brandl at gmx.net  Sat Aug 12 10:33:21 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 12 Aug 2006 10:33:21 +0200
Subject: [Python-Dev] Recent logging spew
In-Reply-To: <1f7befae0608120024q24cd5474p5aa0a3d5f13eae0b@mail.gmail.com>
References: <1f7befae0608120024q24cd5474p5aa0a3d5f13eae0b@mail.gmail.com>
Message-ID: <ebk3oh$ujd$1@sea.gmane.org>

Tim Peters wrote:
> We're getting an awful lot of these turds at the ends of test runs now:
> 
> Error in atexit._run_exitfuncs:


> Not sure what caused it, but suspect this:
> 
> """
> Author: georg.brandl
> Date: Fri Aug 11 09:26:10 2006
> New Revision: 51206
> 
> Modified:
>   python/trunk/Lib/logging/__init__.py
>   python/trunk/Misc/NEWS
> Log:
> logging's atexit hook now runs even if the rest of the module has
> already been cleaned up.
> """
> 
> Anyway, can we stop this?  It doesn't look like "a feature" from here ;-)

If it was a feature, it couldn't have gone in right now ;-)

But yes, I confirmed that that change caused it. I've now found the problem:
one of the test_logging tests reassigned _handlerList, causing it to go
out of sync with the handlerList stored in shutdown().

Fixed in rev. 51235.

Georg


From tomerfiliba at gmail.com  Sat Aug 12 15:42:17 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Sat, 12 Aug 2006 15:42:17 +0200
Subject: [Python-Dev] dict containment annoyance
Message-ID: <1d85506f0608120642y398e3fb4p9ce56c51e915b6d4@mail.gmail.com>

>>> a={1:2, 3:4}
>>> [] in a
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: list objects are unhashable
>>>

imo, the expression should just evaluate to False instead of raising an
exception.
it's a question of semantics -- i asked whether the object (a list, in this
case)
is contained in the dict. i didn't ask whether it's hashable or not. if the
object is
unhashable, then surely it's not contained in the dict, therefore the answer
is
False, rather than an exception.

dict.__contains__ should just swallow these exceptions, and return False.


-tomer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060812/6a4c7877/attachment.html 

From aahz at pythoncraft.com  Sat Aug 12 15:42:44 2006
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 12 Aug 2006 06:42:44 -0700
Subject: [Python-Dev] [Python-3000] Python 2.5 release schedule (was:
	threading, part 2)
In-Reply-To: <20060812012526.195B.JCARLSON@uci.edu>
References: <c56e219d0608111851i6f053407q96e5989fdff77848@mail.gmail.com>
	<20060812012526.195B.JCARLSON@uci.edu>
Message-ID: <20060812134244.GA29374@panix.com>

[added python-dev to make sure everyone sees this]

On Sat, Aug 12, 2006, Josiah Carlson wrote:
>
> All other feature additions are too late in the Beta cycle (Beta 3 is
> next week)

For some reason, this is the second time I've seen this claim.  Beta 3
was released August 3 and next week is rc1.  We are right now in
complete feature lockdown; even documenting an existing API IMO requires
approval from the Release Manager.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it."  --Brian W. Kernighan

From aahz at pythoncraft.com  Sat Aug 12 15:48:26 2006
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 12 Aug 2006 06:48:26 -0700
Subject: [Python-Dev] dict containment annoyance
In-Reply-To: <1d85506f0608120642y398e3fb4p9ce56c51e915b6d4@mail.gmail.com>
References: <1d85506f0608120642y398e3fb4p9ce56c51e915b6d4@mail.gmail.com>
Message-ID: <20060812134826.GC29374@panix.com>

On Sat, Aug 12, 2006, tomer filiba wrote:
>
> >>>a={1:2, 3:4}
> >>>[] in a
> Traceback (most recent call last):
>  File "<stdin>", line 1, in ?
> TypeError: list objects are unhashable
> >>>
> 
> imo, the expression should just evaluate to False instead of raising an
> exception.

-1

This is seriously no different from an attempt to do

>>> a = {}
>>> a[ [] ] = 1
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it."  --Brian W. Kernighan

From david.nospam.hopwood at blueyonder.co.uk  Sat Aug 12 18:26:33 2006
From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood)
Date: Sat, 12 Aug 2006 17:26:33 +0100
Subject: [Python-Dev] dict containment annoyance
In-Reply-To: <1d85506f0608120642y398e3fb4p9ce56c51e915b6d4@mail.gmail.com>
References: <1d85506f0608120642y398e3fb4p9ce56c51e915b6d4@mail.gmail.com>
Message-ID: <44DE0139.5080108@blueyonder.co.uk>

tomer filiba wrote:
>>>> a={1:2, 3:4}
>>>> [] in a
> 
> Traceback (most recent call last):
>  File "<stdin>", line 1, in ?
> TypeError: list objects are unhashable
> 
> imo, the expression should just evaluate to False instead of raising an
> exception. it's a question of semantics -- i asked whether the object
> (a list, in this case) is contained in the dict. i didn't ask whether
> it's hashable or not.

However, if it isn't hashable, asking whether it is in a dict is very
likely to be a programming error.

-- 
David Hopwood <david.nospam.hopwood at blueyonder.co.uk>



From tomerfiliba at gmail.com  Sat Aug 12 18:57:02 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Sat, 12 Aug 2006 18:57:02 +0200
Subject: [Python-Dev] dict containment annoyance
Message-ID: <1d85506f0608120957r7d856211g5f382a0ab796c74c@mail.gmail.com>

[Aahz]
> -1
>
> This is seriously no different from an attempt to do
>
> >>> a = {}
> >>> a[ [] ] = 1

how so? i'm not trying to modify/store anything in a dict.
i'm only testing if a certain object is contained in the dict.
that's totally different.
imagine this:
>>> x = 6
>>> x < 2
am i trying to modify 'x' in any way? i'm only testing if x is
less than 2.

i fail to see your point

[David Hopwood]
> However, if it isn't hashable, asking whether it is in a dict is very
> likely to be a programming error.

not really. my program holds a blacklist of "bad objects", which
shouldn't be granted access to. instead of a list, i'm using a set
(performance, whatever). the "bad objects" are all hashable,
and i want to see if an arbitrary object is "good" or "bad", by looking
it up in the blacklist. the arbitrary object can be hashable or not,
that's not the question.

just like `"hello" != 2` returns True, instead of raising IncompatibleTypes,
i want `x in y` to return False when `x` is not contained in `y`
(with no regard to whether or not it's possible for `y` to contain `x`).

the logic is simple: every `x` is either contained in `y` or not.
if `x` *cannot* be contained in `y`, then the answer is a "strong no",
but that's still a "no".



-tomer

From exarkun at divmod.com  Sat Aug 12 19:07:52 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sat, 12 Aug 2006 13:07:52 -0400
Subject: [Python-Dev] dict containment annoyance
In-Reply-To: <1d85506f0608120957r7d856211g5f382a0ab796c74c@mail.gmail.com>
Message-ID: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm>

On Sat, 12 Aug 2006 18:57:02 +0200, tomer filiba <tomerfiliba at gmail.com> wrote:
>
>the logic is simple: every `x` is either contained in `y` or not.
>if `x` *cannot* be contained in `y`, then the answer is a "strong no",
>but that's still a "no".
>

def blacklisted(o):
    try:
        # Is the object contained in the blacklist set?
        return o in _blacklistset
    except TypeError:
        # If it *cannot* be contained in the blacklist set,
        # then it probably isn't.
        return False

Jean-Paul

From nnorwitz at gmail.com  Sat Aug 12 19:13:03 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sat, 12 Aug 2006 10:13:03 -0700
Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk:
	Doc/api/abstract.tex Include/abstract.h Include/object.h
	Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c
	Modules/mmapmodule.c Modules/operator.c Objects/abstract.c
	Objects/classobject.c Objects/
Message-ID: <ee2a432c0608121013q5a3a3043od63325c332b1d0ba@mail.gmail.com>

I checked in this fix for the __index__ clipping issue that's been
discussed.  This patch is an improvement, but still needs some work.
Please pull out any parts you have an issue with and suggest a patch
to address your concern.

n

---------- Forwarded message ----------
From: neal.norwitz <python-checkins at python.org>
Date: Aug 12, 2006 10:03 AM
Subject: [Python-checkins] r51236 - in python/trunk:
Doc/api/abstract.tex Include/abstract.h Include/object.h
Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c
Modules/mmapmodule.c Modules/operator.c Objects/abstract.c
Objects/classobject.c Objects/intobject.c Objects/listobject.c
Objects/longobject.c Objects/sliceobject.c Objects/stringobject.c
Objects/tupleobject.c Objects/typeobject.c Objects/unicodeobject.c
Python/ceval.c
To: python-checkins at python.org


Author: neal.norwitz
Date: Sat Aug 12 19:03:09 2006
New Revision: 51236

Modified:
   python/trunk/Doc/api/abstract.tex
   python/trunk/Include/abstract.h
   python/trunk/Include/object.h
   python/trunk/Lib/test/test_index.py
   python/trunk/Misc/NEWS
   python/trunk/Modules/arraymodule.c
   python/trunk/Modules/mmapmodule.c
   python/trunk/Modules/operator.c
   python/trunk/Objects/abstract.c
   python/trunk/Objects/classobject.c
   python/trunk/Objects/intobject.c
   python/trunk/Objects/listobject.c
   python/trunk/Objects/longobject.c
   python/trunk/Objects/sliceobject.c
   python/trunk/Objects/stringobject.c
   python/trunk/Objects/tupleobject.c
   python/trunk/Objects/typeobject.c
   python/trunk/Objects/unicodeobject.c
   python/trunk/Python/ceval.c
Log:
Patch #1538606, Patch to fix __index__() clipping.

I modified this patch some by fixing style, some error checking, and adding
XXX comments.  This patch requires review and some changes are to be expected.
I'm checking in now to get the greatest possible review and establish a
baseline for moving forward.  I don't want this to hold up release if possible.



Modified: python/trunk/Doc/api/abstract.tex
==============================================================================
--- python/trunk/Doc/api/abstract.tex   (original)
+++ python/trunk/Doc/api/abstract.tex   Sat Aug 12 19:03:09 2006
@@ -693,12 +693,31 @@
   \samp{float(\var{o})}.\bifuncindex{float}
 \end{cfuncdesc}

-\begin{cfuncdesc}{Py_ssize_t}{PyNumber_Index}{PyObject *o}
-  Returns the \var{o} converted to a Py_ssize_t integer on success, or
-  -1 with an exception raised on failure.
+\begin{cfuncdesc}{PyObject*}{PyNumber_Index}{PyObject *o}
+  Returns the \var{o} converted to a Python int or long on success or \NULL{}
+  with a TypeError exception raised on failure.
   \versionadded{2.5}
 \end{cfuncdesc}

+\begin{cfuncdesc}{Py_ssize_t}{PyNumber_AsSsize_t}{PyObject *o, PyObject *exc}
+  Returns \var{o} converted to a Py_ssize_t value if \var{o}
+  can be interpreted as an integer. If \var{o} can be converted to a Python
+  int or long but the attempt to convert to a Py_ssize_t value
+  would raise an \exception{OverflowError}, then the \var{exc} argument
+  is the type of exception that will be raised (usually \exception{IndexError}
+  or \exception{OverflowError}).  If \var{exc} is \NULL{}, then the exception
+  is cleared and the value is clipped to \var{PY_SSIZE_T_MIN}
+  for a negative integer or \var{PY_SSIZE_T_MAX} for a positive integer.
+  \versionadded{2.5}
+\end{cfuncdesc}
+
+\begin{cfuncdesc}{int}{PyIndex_Check}{PyObject *o}
+  Returns True if \var{o} is an index integer (has the nb_index slot of
+  the tp_as_number structure filled in).
+  \versionadded{2.5}
+\end{cfuncdesc}
+
+
 \section{Sequence Protocol \label{sequence}}

 \begin{cfuncdesc}{int}{PySequence_Check}{PyObject *o}

Modified: python/trunk/Include/abstract.h
==============================================================================
--- python/trunk/Include/abstract.h     (original)
+++ python/trunk/Include/abstract.h     Sat Aug 12 19:03:09 2006
@@ -758,13 +758,27 @@

        */

-     PyAPI_FUNC(Py_ssize_t) PyNumber_Index(PyObject *);
+#define PyIndex_Check(obj) \
+   ((obj)->ob_type->tp_as_number != NULL && \
+    PyType_HasFeature((obj)->ob_type, Py_TPFLAGS_HAVE_INDEX) && \
+    (obj)->ob_type->tp_as_number->nb_index != NULL)
+
+     PyAPI_FUNC(PyObject *) PyNumber_Index(PyObject *o);

        /*
-        Returns the object converted to Py_ssize_t on success
-        or -1 with an error raised on failure.
+        Returns the object converted to a Python long or int
+        or NULL with an error raised on failure.
        */

+     PyAPI_FUNC(Py_ssize_t) PyNumber_AsSsize_t(PyObject *o, PyObject *exc);
+
+       /*
+        Returns the object converted to Py_ssize_t by going through
+        PyNumber_Index first.  If an overflow error occurs while
+        converting the int-or-long to Py_ssize_t, then the second argument
+        is the error-type to return.  If it is NULL, then the overflow error
+        is cleared and the value is clipped.
+       */

      PyAPI_FUNC(PyObject *) PyNumber_Int(PyObject *o);


Modified: python/trunk/Include/object.h
==============================================================================
--- python/trunk/Include/object.h       (original)
+++ python/trunk/Include/object.h       Sat Aug 12 19:03:09 2006
@@ -208,7 +208,7 @@
        binaryfunc nb_inplace_true_divide;

        /* Added in release 2.5 */
-       lenfunc nb_index;
+       unaryfunc nb_index;
 } PyNumberMethods;

 typedef struct {

Modified: python/trunk/Lib/test/test_index.py
==============================================================================
--- python/trunk/Lib/test/test_index.py (original)
+++ python/trunk/Lib/test/test_index.py Sat Aug 12 19:03:09 2006
@@ -1,6 +1,7 @@
 import unittest
 from test import test_support
 import operator
+from sys import maxint

 class oldstyle:
     def __index__(self):
@@ -10,68 +11,115 @@
     def __index__(self):
         return self.ind

+class TrapInt(int):
+    def __index__(self):
+        return self
+
+class TrapLong(long):
+    def __index__(self):
+        return self
+
 class BaseTestCase(unittest.TestCase):
     def setUp(self):
         self.o = oldstyle()
         self.n = newstyle()
-        self.o2 = oldstyle()
-        self.n2 = newstyle()

     def test_basic(self):
         self.o.ind = -2
         self.n.ind = 2
-        assert(self.seq[self.n] == self.seq[2])
-        assert(self.seq[self.o] == self.seq[-2])
-        assert(operator.index(self.o) == -2)
-        assert(operator.index(self.n) == 2)
+        self.assertEqual(operator.index(self.o), -2)
+        self.assertEqual(operator.index(self.n), 2)
+
+    def test_slice(self):
+        self.o.ind = 1
+        self.n.ind = 2
+        slc = slice(self.o, self.o, self.o)
+        check_slc = slice(1, 1, 1)
+        self.assertEqual(slc.indices(self.o), check_slc.indices(1))
+        slc = slice(self.n, self.n, self.n)
+        check_slc = slice(2, 2, 2)
+        self.assertEqual(slc.indices(self.n), check_slc.indices(2))

+    def test_wrappers(self):
+        self.o.ind = 4
+        self.n.ind = 5
+        self.assertEqual(6 .__index__(), 6)
+        self.assertEqual(-7L.__index__(), -7)
+        self.assertEqual(self.o.__index__(), 4)
+        self.assertEqual(self.n.__index__(), 5)
+
+    def test_infinite_recursion(self):
+        self.failUnlessRaises(TypeError, operator.index, TrapInt())
+        self.failUnlessRaises(TypeError, operator.index, TrapLong())
+        self.failUnless(slice(TrapInt()).indices(0)==(0,0,1))
+        self.failUnlessRaises(TypeError, slice(TrapLong()).indices, 0)
+
     def test_error(self):
         self.o.ind = 'dumb'
         self.n.ind = 'bad'
-        myfunc = lambda x, obj: obj.seq[x]
         self.failUnlessRaises(TypeError, operator.index, self.o)
         self.failUnlessRaises(TypeError, operator.index, self.n)
-        self.failUnlessRaises(TypeError, myfunc, self.o, self)
-        self.failUnlessRaises(TypeError, myfunc, self.n, self)
+        self.failUnlessRaises(TypeError, slice(self.o).indices, 0)
+        self.failUnlessRaises(TypeError, slice(self.n).indices, 0)
+
+
+class SeqTestCase(unittest.TestCase):
+    # This test case isn't run directly. It just defines common tests
+    # to the different sequence types below
+    def setUp(self):
+        self.o = oldstyle()
+        self.n = newstyle()
+        self.o2 = oldstyle()
+        self.n2 = newstyle()
+
+    def test_index(self):
+        self.o.ind = -2
+        self.n.ind = 2
+        self.assertEqual(self.seq[self.n], self.seq[2])
+        self.assertEqual(self.seq[self.o], self.seq[-2])

     def test_slice(self):
         self.o.ind = 1
         self.o2.ind = 3
         self.n.ind = 2
         self.n2.ind = 4
-        assert(self.seq[self.o:self.o2] == self.seq[1:3])
-        assert(self.seq[self.n:self.n2] == self.seq[2:4])
+        self.assertEqual(self.seq[self.o:self.o2], self.seq[1:3])
+        self.assertEqual(self.seq[self.n:self.n2], self.seq[2:4])

     def test_repeat(self):
         self.o.ind = 3
         self.n.ind = 2
-        assert(self.seq * self.o == self.seq * 3)
-        assert(self.seq * self.n == self.seq * 2)
-        assert(self.o * self.seq == self.seq * 3)
-        assert(self.n * self.seq == self.seq * 2)
+        self.assertEqual(self.seq * self.o, self.seq * 3)
+        self.assertEqual(self.seq * self.n, self.seq * 2)
+        self.assertEqual(self.o * self.seq, self.seq * 3)
+        self.assertEqual(self.n * self.seq, self.seq * 2)

     def test_wrappers(self):
-        n = self.n
-        n.ind = 5
-        assert n.__index__() == 5
-        assert 6 .__index__() == 6
-        assert -7L.__index__() == -7
-        assert self.seq.__getitem__(n) == self.seq[5]
-        assert self.seq.__mul__(n) == self.seq * 5
-        assert self.seq.__rmul__(n) == self.seq * 5
-
-    def test_infinite_recusion(self):
-        class Trap1(int):
-            def __index__(self):
-                return self
-        class Trap2(long):
-            def __index__(self):
-                return self
-        self.failUnlessRaises(TypeError, operator.getitem, self.seq, Trap1())
-        self.failUnlessRaises(TypeError, operator.getitem, self.seq, Trap2())
+        self.o.ind = 4
+        self.n.ind = 5
+        self.assertEqual(self.seq.__getitem__(self.o), self.seq[4])
+        self.assertEqual(self.seq.__mul__(self.o), self.seq * 4)
+        self.assertEqual(self.seq.__rmul__(self.o), self.seq * 4)
+        self.assertEqual(self.seq.__getitem__(self.n), self.seq[5])
+        self.assertEqual(self.seq.__mul__(self.n), self.seq * 5)
+        self.assertEqual(self.seq.__rmul__(self.n), self.seq * 5)
+
+    def test_infinite_recursion(self):
+        self.failUnlessRaises(TypeError, operator.getitem, self.seq, TrapInt())
+        self.failUnlessRaises(TypeError, operator.getitem, self.seq,
TrapLong())
+
+    def test_error(self):
+        self.o.ind = 'dumb'
+        self.n.ind = 'bad'
+        indexobj = lambda x, obj: obj.seq[x]
+        self.failUnlessRaises(TypeError, indexobj, self.o, self)
+        self.failUnlessRaises(TypeError, indexobj, self.n, self)
+        sliceobj = lambda x, obj: obj.seq[x:]
+        self.failUnlessRaises(TypeError, sliceobj, self.o, self)
+        self.failUnlessRaises(TypeError, sliceobj, self.n, self)


-class ListTestCase(BaseTestCase):
+class ListTestCase(SeqTestCase):
     seq = [0,10,20,30,40,50]

     def test_setdelitem(self):
@@ -82,36 +130,36 @@
         del lst[self.n]
         lst[self.o] = 'X'
         lst[self.n] = 'Y'
-        assert lst == list('abYdefghXj')
+        self.assertEqual(lst, list('abYdefghXj'))

         lst = [5, 6, 7, 8, 9, 10, 11]
         lst.__setitem__(self.n, "here")
-        assert lst == [5, 6, "here", 8, 9, 10, 11]
+        self.assertEqual(lst, [5, 6, "here", 8, 9, 10, 11])
         lst.__delitem__(self.n)
-        assert lst == [5, 6, 8, 9, 10, 11]
+        self.assertEqual(lst, [5, 6, 8, 9, 10, 11])

     def test_inplace_repeat(self):
         self.o.ind = 2
         self.n.ind = 3
         lst = [6, 4]
         lst *= self.o
-        assert lst == [6, 4, 6, 4]
+        self.assertEqual(lst, [6, 4, 6, 4])
         lst *= self.n
-        assert lst == [6, 4, 6, 4] * 3
+        self.assertEqual(lst, [6, 4, 6, 4] * 3)

         lst = [5, 6, 7, 8, 9, 11]
         l2 = lst.__imul__(self.n)
-        assert l2 is lst
-        assert lst == [5, 6, 7, 8, 9, 11] * 3
+        self.assert_(l2 is lst)
+        self.assertEqual(lst, [5, 6, 7, 8, 9, 11] * 3)


-class TupleTestCase(BaseTestCase):
+class TupleTestCase(SeqTestCase):
     seq = (0,10,20,30,40,50)

-class StringTestCase(BaseTestCase):
+class StringTestCase(SeqTestCase):
     seq = "this is a test"

-class UnicodeTestCase(BaseTestCase):
+class UnicodeTestCase(SeqTestCase):
     seq = u"this is a test"


@@ -120,17 +168,47 @@
     def test_xrange(self):
         n = newstyle()
         n.ind = 5
-        assert xrange(1, 20)[n] == 6
-        assert xrange(1, 20).__getitem__(n) == 6
+        self.assertEqual(xrange(1, 20)[n], 6)
+        self.assertEqual(xrange(1, 20).__getitem__(n), 6)
+
+class OverflowTestCase(unittest.TestCase):
+
+    def setUp(self):
+        self.pos = 2**100
+        self.neg = -self.pos
+
+    def test_large_longs(self):
+        self.assertEqual(self.pos.__index__(), self.pos)
+        self.assertEqual(self.neg.__index__(), self.neg)
+
+    def test_getitem(self):
+        class GetItem(object):
+            def __len__(self):
+                return maxint
+            def __getitem__(self, key):
+                return key
+            def __getslice__(self, i, j):
+                return i, j
+        x = GetItem()
+        self.assertEqual(x[self.pos], self.pos)
+        self.assertEqual(x[self.neg], self.neg)
+        self.assertEqual(x[self.neg:self.pos], (-1, maxint))
+        self.assertEqual(x[self.neg:self.pos:1].indices(maxint), (0,
maxint, 1))
+
+    def test_sequence_repeat(self):
+        self.failUnlessRaises(OverflowError, lambda: "a" * self.pos)
+        self.failUnlessRaises(OverflowError, lambda: "a" * self.neg)


 def test_main():
     test_support.run_unittest(
+        BaseTestCase,
         ListTestCase,
         TupleTestCase,
         StringTestCase,
         UnicodeTestCase,
         XRangeTestCase,
+        OverflowTestCase,
     )

 if __name__ == "__main__":

Modified: python/trunk/Misc/NEWS
==============================================================================
--- python/trunk/Misc/NEWS      (original)
+++ python/trunk/Misc/NEWS      Sat Aug 12 19:03:09 2006
@@ -12,6 +12,11 @@
 Core and builtins
 -----------------

+- Patch #1538606, Fix __index__() clipping.  There were some problems
+  discovered with the API and how integers that didn't fit into Py_ssize_t
+  were handled.  This patch attempts to provide enough alternatives
+  to effectively use __index__.
+
 - Bug #1536021: __hash__ may now return long int; the final hash
   value is obtained by invoking hash on the long int.


Modified: python/trunk/Modules/arraymodule.c
==============================================================================
--- python/trunk/Modules/arraymodule.c  (original)
+++ python/trunk/Modules/arraymodule.c  Sat Aug 12 19:03:09 2006
@@ -1572,14 +1572,11 @@
        return s;
 }

-#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX)
-
 static PyObject*
 array_subscr(arrayobject* self, PyObject* item)
 {
-       PyNumberMethods *nb = item->ob_type->tp_as_number;
-       if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) {
-               Py_ssize_t i = nb->nb_index(item);
+       if (PyIndex_Check(item)) {
+               Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
                if (i==-1 && PyErr_Occurred()) {
                        return NULL;
                }
@@ -1627,9 +1624,8 @@
 static int
 array_ass_subscr(arrayobject* self, PyObject* item, PyObject* value)
 {
-       PyNumberMethods *nb = item->ob_type->tp_as_number;
-       if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) {
-               Py_ssize_t i = nb->nb_index(item);
+       if (PyIndex_Check(item)) {
+               Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
                if (i==-1 && PyErr_Occurred())
                        return -1;
                if (i < 0)

Modified: python/trunk/Modules/mmapmodule.c
==============================================================================
--- python/trunk/Modules/mmapmodule.c   (original)
+++ python/trunk/Modules/mmapmodule.c   Sat Aug 12 19:03:09 2006
@@ -808,8 +808,6 @@
 };


-#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX)
-
 /* extract the map size from the given PyObject

    Returns -1 on error, with an appropriate Python exception raised. On
@@ -817,31 +815,19 @@
 static Py_ssize_t
 _GetMapSize(PyObject *o)
 {
-       PyNumberMethods *nb = o->ob_type->tp_as_number;
-       if (nb != NULL && HASINDEX(o) && nb->nb_index != NULL) {
-               Py_ssize_t i = nb->nb_index(o);
+       if (PyIndex_Check(o)) {
+               Py_ssize_t i = PyNumber_AsSsize_t(o, PyExc_OverflowError);
                if (i==-1 && PyErr_Occurred())
                        return -1;
-               if (i < 0)
-                       goto onnegoverflow;
-               if (i==PY_SSIZE_T_MAX)
-                       goto onposoverflow;
+               if (i < 0) {
+                       PyErr_SetString(PyExc_OverflowError,
+                                       "memory mapped size must be positive");
+                       return -1;
+               }
                return i;
        }
-       else {
-               PyErr_SetString(PyExc_TypeError,
-                               "map size must be an integral value");
-               return -1;
-       }
-
-  onnegoverflow:
-       PyErr_SetString(PyExc_OverflowError,
-                       "memory mapped size must be positive");
-       return -1;

-  onposoverflow:
-       PyErr_SetString(PyExc_OverflowError,
-                       "memory mapped size is too large (limited by C int)");
+       PyErr_SetString(PyExc_TypeError, "map size must be an integral value");
        return -1;
 }


Modified: python/trunk/Modules/operator.c
==============================================================================
--- python/trunk/Modules/operator.c     (original)
+++ python/trunk/Modules/operator.c     Sat Aug 12 19:03:09 2006
@@ -139,15 +139,7 @@
 static PyObject *
 op_index(PyObject *s, PyObject *a)
 {
-       Py_ssize_t i;
-       PyObject *a1;
-       if (!PyArg_UnpackTuple(a,"index", 1, 1, &a1))
-               return NULL;
-       i = PyNumber_Index(a1);
-       if (i == -1 && PyErr_Occurred())
-               return NULL;
-       else
-               return PyInt_FromSsize_t(i);
+       return PyNumber_Index(a);
 }

 static PyObject*
@@ -249,7 +241,7 @@

 spam1(is_, "is_(a, b) -- Same as a is b.")
 spam1(is_not, "is_not(a, b) -- Same as a is not b.")
-spam2(index, __index__, "index(a) -- Same as a.__index__()")
+spam2o(index, __index__, "index(a) -- Same as a.__index__()")
 spam2(add,__add__, "add(a, b) -- Same as a + b.")
 spam2(sub,__sub__, "sub(a, b) -- Same as a - b.")
 spam2(mul,__mul__, "mul(a, b) -- Same as a * b.")

Modified: python/trunk/Objects/abstract.c
==============================================================================
--- python/trunk/Objects/abstract.c     (original)
+++ python/trunk/Objects/abstract.c     Sat Aug 12 19:03:09 2006
@@ -8,8 +8,6 @@
 #define NEW_STYLE_NUMBER(o) PyType_HasFeature((o)->ob_type, \
                                Py_TPFLAGS_CHECKTYPES)

-#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX)
-

 /* Shorthands to return certain errors */

@@ -122,9 +120,9 @@
                return m->mp_subscript(o, key);

        if (o->ob_type->tp_as_sequence) {
-               PyNumberMethods *nb = key->ob_type->tp_as_number;
-               if (nb != NULL && HASINDEX(key) && nb->nb_index != NULL) {
-                       Py_ssize_t key_value = nb->nb_index(key);
+               if (PyIndex_Check(key)) {
+                       Py_ssize_t key_value;
+                       key_value = PyNumber_AsSsize_t(key, PyExc_IndexError);
                        if (key_value == -1 && PyErr_Occurred())
                                return NULL;
                        return PySequence_GetItem(o, key_value);
@@ -151,9 +149,9 @@
                return m->mp_ass_subscript(o, key, value);

        if (o->ob_type->tp_as_sequence) {
-               PyNumberMethods *nb = key->ob_type->tp_as_number;
-               if (nb != NULL && HASINDEX(key) && nb->nb_index != NULL) {
-                       Py_ssize_t key_value = nb->nb_index(key);
+               if (PyIndex_Check(key)) {
+                       Py_ssize_t key_value;
+                       key_value = PyNumber_AsSsize_t(key, PyExc_IndexError);
                        if (key_value == -1 && PyErr_Occurred())
                                return -1;
                        return PySequence_SetItem(o, key_value, value);
@@ -183,9 +181,9 @@
                return m->mp_ass_subscript(o, key, (PyObject*)NULL);

        if (o->ob_type->tp_as_sequence) {
-               PyNumberMethods *nb = key->ob_type->tp_as_number;
-               if (nb != NULL && HASINDEX(key) && nb->nb_index != NULL) {
-                       Py_ssize_t key_value = nb->nb_index(key);
+               if (PyIndex_Check(key)) {
+                       Py_ssize_t key_value;
+                       key_value = PyNumber_AsSsize_t(key, PyExc_IndexError);
                        if (key_value == -1 && PyErr_Occurred())
                                return -1;
                        return PySequence_DelItem(o, key_value);
@@ -653,9 +651,8 @@
 sequence_repeat(ssizeargfunc repeatfunc, PyObject *seq, PyObject *n)
 {
        Py_ssize_t count;
-       PyNumberMethods *nb = n->ob_type->tp_as_number;
-       if (nb != NULL && HASINDEX(n) && nb->nb_index != NULL) {
-               count = nb->nb_index(n);
+       if (PyIndex_Check(n)) {
+               count = PyNumber_AsSsize_t(n, PyExc_OverflowError);
                if (count == -1 && PyErr_Occurred())
                        return NULL;
        }
@@ -938,23 +935,89 @@
        return x;
 }

-/* Return a Py_ssize_t integer from the object item */
-Py_ssize_t
+/* Return a Python Int or Long from the object item
+   Raise TypeError if the result is not an int-or-long
+   or if the object cannot be interpreted as an index.
+*/
+PyObject *
 PyNumber_Index(PyObject *item)
 {
-       Py_ssize_t value = -1;
-       PyNumberMethods *nb = item->ob_type->tp_as_number;
-       if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) {
-               value = nb->nb_index(item);
+       PyObject *result = NULL;
+       if (item == NULL)
+               return null_error();
+       /* XXX(nnorwitz): should these be CheckExact?  Aren't subclasses ok? */
+       if (PyInt_CheckExact(item) || PyLong_CheckExact(item)) {
+               Py_INCREF(item);
+               return item;
+       }
+       if (PyIndex_Check(item)) {
+               result = item->ob_type->tp_as_number->nb_index(item);
+               /* XXX(nnorwitz): Aren't subclasses ok here too? */
+               if (result &&
+                   !PyInt_CheckExact(result) && !PyLong_CheckExact(result)) {
+                       PyErr_Format(PyExc_TypeError,
+                                    "__index__ returned non-(int,long) " \
+                                    "(type %.200s)",
+                                    result->ob_type->tp_name);
+                       Py_DECREF(result);
+                       return NULL;
+               }
        }
        else {
                PyErr_Format(PyExc_TypeError,
                             "'%.200s' object cannot be interpreted "
                             "as an index", item->ob_type->tp_name);
        }
-       return value;
+       return result;
 }

+/* Return an error on Overflow only if err is not NULL*/
+
+Py_ssize_t
+PyNumber_AsSsize_t(PyObject *item, PyObject *err)
+{
+       Py_ssize_t result;
+       PyObject *runerr;
+       PyObject *value = PyNumber_Index(item);
+       if (value == NULL)
+               return -1;
+
+       /* We're done if PyInt_AsSsize_t() returns without error. */
+       result = PyInt_AsSsize_t(value);
+       if (result != -1 || !(runerr = PyErr_Occurred()))
+               goto finish;
+
+       /* Error handling code -- only manage OverflowError differently */
+       if (!PyErr_GivenExceptionMatches(runerr, PyExc_OverflowError))
+               goto finish;
+
+       PyErr_Clear();
+       /* If no error-handling desired then the default clipping
+          is sufficient.
+        */
+       if (!err) {
+               assert(PyLong_Check(value));
+               /* Whether or not it is less than or equal to
+                  zero is determined by the sign of ob_size
+               */
+               if (_PyLong_Sign(value) < 0)
+                       result = PY_SSIZE_T_MIN;
+               else
+                       result = PY_SSIZE_T_MAX;
+       }
+       else {
+               /* Otherwise replace the error with caller's error object. */
+               PyErr_Format(err,
+                            "cannot fit '%.200s' into an index-sized integer",
+                            item->ob_type->tp_name);
+       }
+
+ finish:
+       Py_DECREF(value);
+       return result;
+}
+
+
 PyObject *
 PyNumber_Int(PyObject *o)
 {

Modified: python/trunk/Objects/classobject.c
==============================================================================
--- python/trunk/Objects/classobject.c  (original)
+++ python/trunk/Objects/classobject.c  Sat Aug 12 19:03:09 2006
@@ -1670,40 +1670,28 @@
        return outcome > 0;
 }

-static Py_ssize_t
+static PyObject *
 instance_index(PyInstanceObject *self)
 {
        PyObject *func, *res;
-       Py_ssize_t outcome;
        static PyObject *indexstr = NULL;

        if (indexstr == NULL) {
                indexstr = PyString_InternFromString("__index__");
                if (indexstr == NULL)
-                       return -1;
+                       return NULL;
        }
        if ((func = instance_getattr(self, indexstr)) == NULL) {
                if (!PyErr_ExceptionMatches(PyExc_AttributeError))
-                       return -1;
+                       return NULL;
                PyErr_Clear();
                PyErr_SetString(PyExc_TypeError,
                                "object cannot be interpreted as an index");
-               return -1;
+               return NULL;
        }
        res = PyEval_CallObject(func, (PyObject *)NULL);
        Py_DECREF(func);
-       if (res == NULL)
-               return -1;
-       if (PyInt_Check(res) || PyLong_Check(res)) {
-               outcome = res->ob_type->tp_as_number->nb_index(res);
-       }
-       else {
-               PyErr_SetString(PyExc_TypeError,
-                               "__index__ must return an int or a long");
-               outcome = -1;
-       }
-       Py_DECREF(res);
-       return outcome;
+       return res;
 }


@@ -2026,7 +2014,7 @@
        instance_truediv,               /* nb_true_divide */
        instance_ifloordiv,             /* nb_inplace_floor_divide */
        instance_itruediv,              /* nb_inplace_true_divide */
-       (lenfunc)instance_index,        /* nb_index */
+       (unaryfunc)instance_index,      /* nb_index */
 };

 PyTypeObject PyInstance_Type = {

Modified: python/trunk/Objects/intobject.c
==============================================================================
--- python/trunk/Objects/intobject.c    (original)
+++ python/trunk/Objects/intobject.c    Sat Aug 12 19:03:09 2006
@@ -193,16 +193,21 @@
        PyIntObject *io;
        Py_ssize_t val;
 #endif
-       if (op && !PyInt_CheckExact(op) && PyLong_Check(op))
+
+       if (op == NULL) {
+               PyErr_SetString(PyExc_TypeError, "an integer is required");
+               return -1;
+       }
+
+       if (PyInt_Check(op))
+               return PyInt_AS_LONG((PyIntObject*) op);
+       if (PyLong_Check(op))
                return _PyLong_AsSsize_t(op);
 #if SIZEOF_SIZE_T == SIZEOF_LONG
        return PyInt_AsLong(op);
 #else

-       if (op && PyInt_Check(op))
-               return PyInt_AS_LONG((PyIntObject*) op);
-
-       if (op == NULL || (nb = op->ob_type->tp_as_number) == NULL ||
+       if ((nb = op->ob_type->tp_as_number) == NULL ||
            (nb->nb_int == NULL && nb->nb_long == 0)) {
                PyErr_SetString(PyExc_TypeError, "an integer is required");
                return -1;
@@ -1079,7 +1084,7 @@
        int_true_divide,        /* nb_true_divide */
        0,                      /* nb_inplace_floor_divide */
        0,                      /* nb_inplace_true_divide */
-       PyInt_AsSsize_t,        /* nb_index */
+       (unaryfunc)int_int,     /* nb_index */
 };

 PyTypeObject PyInt_Type = {

Modified: python/trunk/Objects/listobject.c
==============================================================================
--- python/trunk/Objects/listobject.c   (original)
+++ python/trunk/Objects/listobject.c   Sat Aug 12 19:03:09 2006
@@ -2450,14 +2450,13 @@
 "list() -> new list\n"
 "list(sequence) -> new list initialized from sequence's items");

-#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX)

 static PyObject *
 list_subscript(PyListObject* self, PyObject* item)
 {
-       PyNumberMethods *nb = item->ob_type->tp_as_number;
-       if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) {
-               Py_ssize_t i = nb->nb_index(item);
+       if (PyIndex_Check(item)) {
+               Py_ssize_t i;
+               i = PyNumber_AsSsize_t(item, PyExc_IndexError);
                if (i == -1 && PyErr_Occurred())
                        return NULL;
                if (i < 0)
@@ -2504,9 +2503,8 @@
 static int
 list_ass_subscript(PyListObject* self, PyObject* item, PyObject* value)
 {
-       PyNumberMethods *nb = item->ob_type->tp_as_number;
-       if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) {
-               Py_ssize_t i = nb->nb_index(item);
+       if (PyIndex_Check(item)) {
+               Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
                if (i == -1 && PyErr_Occurred())
                        return -1;
                if (i < 0)

Modified: python/trunk/Objects/longobject.c
==============================================================================
--- python/trunk/Objects/longobject.c   (original)
+++ python/trunk/Objects/longobject.c   Sat Aug 12 19:03:09 2006
@@ -240,8 +240,11 @@
        return -1;
 }

-static Py_ssize_t
-_long_as_ssize_t(PyObject *vv) {
+/* Get a Py_ssize_t from a long int object.
+   Returns -1 and sets an error condition if overflow occurs. */
+
+Py_ssize_t
+_PyLong_AsSsize_t(PyObject *vv) {
        register PyLongObject *v;
        size_t x, prev;
        Py_ssize_t i;
@@ -277,45 +280,7 @@
  overflow:
        PyErr_SetString(PyExc_OverflowError,
                        "long int too large to convert to int");
-       if (sign > 0)
-               return PY_SSIZE_T_MAX;
-       else
-               return PY_SSIZE_T_MIN;
-}
-
-/* Get a Py_ssize_t from a long int object.
-   Returns -1 and sets an error condition if overflow occurs. */
-
-Py_ssize_t
-_PyLong_AsSsize_t(PyObject *vv)
-{
-       Py_ssize_t x;
-
-       x = _long_as_ssize_t(vv);
-       if (PyErr_Occurred()) return -1;
-       return x;
-}
-
-
-/* Get a Py_ssize_t from a long int object.
-   Silently reduce values larger than PY_SSIZE_T_MAX to PY_SSIZE_T_MAX,
-   and silently boost values less than -PY_SSIZE_T_MAX-1 to -PY_SSIZE_T_MAX-1.
-   On error, return -1 with an exception set.
-*/
-
-static Py_ssize_t
-long_index(PyObject *vv)
-{
-       Py_ssize_t x;
-
-       x = _long_as_ssize_t(vv);
-       if (PyErr_Occurred()) {
-               /* If overflow error, ignore the error */
-               if (x != -1) {
-                       PyErr_Clear();
-               }
-       }
-       return x;
+       return -1;
 }

 /* Get a C unsigned long int from a long int object.
@@ -3405,7 +3370,7 @@
        long_true_divide,               /* nb_true_divide */
        0,                              /* nb_inplace_floor_divide */
        0,                              /* nb_inplace_true_divide */
-       long_index,                     /* nb_index */
+       long_long,                      /* nb_index */
 };

 PyTypeObject PyLong_Type = {

Modified: python/trunk/Objects/sliceobject.c
==============================================================================
--- python/trunk/Objects/sliceobject.c  (original)
+++ python/trunk/Objects/sliceobject.c  Sat Aug 12 19:03:09 2006
@@ -252,7 +252,7 @@
 {
        Py_ssize_t ilen, start, stop, step, slicelength;

-       ilen = PyInt_AsSsize_t(len);
+       ilen = PyNumber_AsSsize_t(len, PyExc_OverflowError);

        if (ilen == -1 && PyErr_Occurred()) {
                return NULL;

Modified: python/trunk/Objects/stringobject.c
==============================================================================
--- python/trunk/Objects/stringobject.c (original)
+++ python/trunk/Objects/stringobject.c Sat Aug 12 19:03:09 2006
@@ -1184,14 +1184,11 @@
        return x;
 }

-#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX)
-
 static PyObject*
 string_subscript(PyStringObject* self, PyObject* item)
 {
-       PyNumberMethods *nb = item->ob_type->tp_as_number;
-       if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) {
-               Py_ssize_t i = nb->nb_index(item);
+       if (PyIndex_Check(item)) {
+               Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
                if (i == -1 && PyErr_Occurred())
                        return NULL;
                if (i < 0)

Modified: python/trunk/Objects/tupleobject.c
==============================================================================
--- python/trunk/Objects/tupleobject.c  (original)
+++ python/trunk/Objects/tupleobject.c  Sat Aug 12 19:03:09 2006
@@ -577,14 +577,11 @@
        (objobjproc)tuplecontains,              /* sq_contains */
 };

-#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX)
-
 static PyObject*
 tuplesubscript(PyTupleObject* self, PyObject* item)
 {
-       PyNumberMethods *nb = item->ob_type->tp_as_number;
-       if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) {
-               Py_ssize_t i = nb->nb_index(item);
+       if (PyIndex_Check(item)) {
+               Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
                if (i == -1 && PyErr_Occurred())
                        return NULL;
                if (i < 0)

Modified: python/trunk/Objects/typeobject.c
==============================================================================
--- python/trunk/Objects/typeobject.c   (original)
+++ python/trunk/Objects/typeobject.c   Sat Aug 12 19:03:09 2006
@@ -3527,7 +3527,7 @@

        if (!PyArg_UnpackTuple(args, "", 1, 1, &o))
                return NULL;
-       i = PyNumber_Index(o);
+       i = PyNumber_AsSsize_t(o, PyExc_OverflowError);
        if (i == -1 && PyErr_Occurred())
                return NULL;
        return (*func)(self, i);
@@ -3538,7 +3538,7 @@
 {
        Py_ssize_t i;

-       i = PyNumber_Index(arg);
+       i = PyNumber_AsSsize_t(arg, PyExc_OverflowError);
        if (i == -1 && PyErr_Occurred())
                return -1;
        if (i < 0) {
@@ -4344,26 +4344,11 @@
 }


-static Py_ssize_t
+static PyObject *
 slot_nb_index(PyObject *self)
 {
        static PyObject *index_str;
-       PyObject *temp = call_method(self, "__index__", &index_str, "()");
-       Py_ssize_t result;
-
-       if (temp == NULL)
-               return -1;
-       if (PyInt_CheckExact(temp) || PyLong_CheckExact(temp)) {
-               result = temp->ob_type->tp_as_number->nb_index(temp);
-       }
-       else {
-               PyErr_Format(PyExc_TypeError,
-                            "__index__ must return an int or a long, "
-                            "not '%.200s'", temp->ob_type->tp_name);
-               result = -1;
-       }
-       Py_DECREF(temp);
-       return result;
+       return call_method(self, "__index__", &index_str, "()");
 }


@@ -5109,7 +5094,7 @@
               "oct(x)"),
        UNSLOT("__hex__", nb_hex, slot_nb_hex, wrap_unaryfunc,
               "hex(x)"),
-       NBSLOT("__index__", nb_index, slot_nb_index, wrap_lenfunc,
+       NBSLOT("__index__", nb_index, slot_nb_index, wrap_unaryfunc,
               "x[y:z] <==> x[y.__index__():z.__index__()]"),
        IBSLOT("__iadd__", nb_inplace_add, slot_nb_inplace_add,
               wrap_binaryfunc, "+"),

Modified: python/trunk/Objects/unicodeobject.c
==============================================================================
--- python/trunk/Objects/unicodeobject.c        (original)
+++ python/trunk/Objects/unicodeobject.c        Sat Aug 12 19:03:09 2006
@@ -6985,14 +6985,11 @@
     PyUnicode_Contains,                /* sq_contains */
 };

-#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX)
-
 static PyObject*
 unicode_subscript(PyUnicodeObject* self, PyObject* item)
 {
-    PyNumberMethods *nb = item->ob_type->tp_as_number;
-    if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) {
-        Py_ssize_t i = nb->nb_index(item);
+    if (PyIndex_Check(item)) {
+        Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
         if (i == -1 && PyErr_Occurred())
             return NULL;
         if (i < 0)

Modified: python/trunk/Python/ceval.c
==============================================================================
--- python/trunk/Python/ceval.c (original)
+++ python/trunk/Python/ceval.c Sat Aug 12 19:03:09 2006
@@ -3866,12 +3866,14 @@
        if (v != NULL) {
                Py_ssize_t x;
                if (PyInt_Check(v)) {
-                       x = PyInt_AsSsize_t(v);
+                       /* XXX(nnorwitz): I think PyInt_AS_LONG is correct,
+                          however, it looks like it should be AsSsize_t.
+                          There should be a comment here explaining why.
+                       */
+                       x = PyInt_AS_LONG(v);
                }
-               else if (v->ob_type->tp_as_number &&
-                        PyType_HasFeature(v->ob_type, Py_TPFLAGS_HAVE_INDEX)
-                        && v->ob_type->tp_as_number->nb_index) {
-                       x = v->ob_type->tp_as_number->nb_index(v);
+               else if (PyIndex_Check(v)) {
+                       x = PyNumber_AsSsize_t(v, NULL);
                        if (x == -1 && PyErr_Occurred())
                                return 0;
                }
@@ -3887,10 +3889,8 @@
 }

 #undef ISINDEX
-#define ISINDEX(x) ((x) == NULL || PyInt_Check(x) || PyLong_Check(x) || \
-                   ((x)->ob_type->tp_as_number && \
-                     PyType_HasFeature((x)->ob_type, Py_TPFLAGS_HAVE_INDEX) \
-                    && (x)->ob_type->tp_as_number->nb_index))
+#define ISINDEX(x) ((x) == NULL || \
+                   PyInt_Check(x) || PyLong_Check(x) || PyIndex_Check(x))

 static PyObject *
 apply_slice(PyObject *u, PyObject *v, PyObject *w) /* return u[v:w] */
_______________________________________________
Python-checkins mailing list
Python-checkins at python.org
http://mail.python.org/mailman/listinfo/python-checkins

From g.brandl at gmx.net  Sat Aug 12 20:10:54 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 12 Aug 2006 20:10:54 +0200
Subject: [Python-Dev] dict containment annoyance
In-Reply-To: <1d85506f0608120957r7d856211g5f382a0ab796c74c@mail.gmail.com>
References: <1d85506f0608120957r7d856211g5f382a0ab796c74c@mail.gmail.com>
Message-ID: <ebl5je$ikv$1@sea.gmane.org>

tomer filiba wrote:
> [Aahz]
>> -1
>>
>> This is seriously no different from an attempt to do
>>
>> >>> a = {}
>> >>> a[ [] ] = 1
> 
> how so? i'm not trying to modify/store anything in a dict.
> i'm only testing if a certain object is contained in the dict.
> that's totally different.
> imagine this:
>>>> x = 6
>>>> x < 2
> am i trying to modify 'x' in any way? i'm only testing if x is
> less than 2.

The correct analogon is

 >>> x = []
 >>> x < 2

which currently works in Python, but won't in 3.0.

Georg


From oliphant.travis at ieee.org  Sat Aug 12 21:56:27 2006
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Sat, 12 Aug 2006 13:56:27 -0600
Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk:
 Doc/api/abstract.tex Include/abstract.h Include/object.h
 Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c
 Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/
In-Reply-To: <ee2a432c0608121013q5a3a3043od63325c332b1d0ba@mail.gmail.com>
References: <ee2a432c0608121013q5a3a3043od63325c332b1d0ba@mail.gmail.com>
Message-ID: <eblbpc$2d5$1@sea.gmane.org>

Neal Norwitz wrote:
> I checked in this fix for the __index__ clipping issue that's been
> discussed.  This patch is an improvement, but still needs some work.
> 

> +/* Return a Python Int or Long from the object item
> +   Raise TypeError if the result is not an int-or-long
> +   or if the object cannot be interpreted as an index.
> +*/
> +PyObject *
>  PyNumber_Index(PyObject *item)
>  {
> -       Py_ssize_t value = -1;
> -       PyNumberMethods *nb = item->ob_type->tp_as_number;
> -       if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) {
> -               value = nb->nb_index(item);
> +       PyObject *result = NULL;
> +       if (item == NULL)
> +               return null_error();
> +       /* XXX(nnorwitz): should these be CheckExact?  Aren't subclasses ok? */

The idea is that the __index__() method should return an exact int or 
exact long or this call will raise an error.  The restriction is present 
to remove the possibility of infinite recursion (though I'm not sure 
where that would occur exactly).


> Modified: python/trunk/Python/ceval.c
> ==============================================================================
> --- python/trunk/Python/ceval.c (original)
> +++ python/trunk/Python/ceval.c Sat Aug 12 19:03:09 2006
> @@ -3866,12 +3866,14 @@
>         if (v != NULL) {
>                 Py_ssize_t x;
>                 if (PyInt_Check(v)) {
> -                       x = PyInt_AsSsize_t(v);
> +                       /* XXX(nnorwitz): I think PyInt_AS_LONG is correct,
> +                          however, it looks like it should be AsSsize_t.
> +                          There should be a comment here explaining why.
> +                       */
> +                       x = PyInt_AS_LONG(v);

Right now throughout the Python code it is assumed that 
sizeof(Py_ssize_t) <= sizeof(long).  Because this code is an 
optimization for integers (or their sub-classes), it seems prudent to 
truly make it fast rather than make a function call that will just go 
through a series of checks to eventually make this very same call.


-Travis


From oliphant.travis at ieee.org  Sat Aug 12 22:16:23 2006
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Sat, 12 Aug 2006 14:16:23 -0600
Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk:
 Doc/api/abstract.tex Include/abstract.h Include/object.h
 Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c
 Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/
In-Reply-To: <ee2a432c0608121013q5a3a3043od63325c332b1d0ba@mail.gmail.com>
References: <ee2a432c0608121013q5a3a3043od63325c332b1d0ba@mail.gmail.com>
Message-ID: <eblcuo$5ea$1@sea.gmane.org>

Neal Norwitz wrote:
> I checked in this fix for the __index__ clipping issue that's been
> discussed.  This patch is an improvement, but still needs some work.
> Please pull out any parts you have an issue with and suggest a patch
> to address your concern.
> 

For me the only remaining concern is that quite often in the code we do this

if (PyIndex_Check(obj)) {
...
	key = PyNumber_Index(obj);
or
  	key_value = PyNumber_AsSize_t(obj, ...)
}
else {remaining checks}


Internally PyNumber_AsSize_t makes a call to PyNumber_Index, and 
PyNumber_Index also calls the PyIndex_Check as well .  So, basically we 
end up calling PyIndex_Check(obj) 2 times when only one check should be 
necessary.

This code could be re-written to move any other type checks first and 
replace the PyIndex_Check(obj) code with PyNumber_Index(obj) and error 
handling but I'm not sure if that's the right way to go or if it's worth 
it.

-Travis Oliphant




From martin at v.loewis.de  Sun Aug 13 00:24:31 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 13 Aug 2006 00:24:31 +0200
Subject: [Python-Dev] What is the status of file.readinto?
In-Reply-To: <d38f5330608111507s1407a1damacf9f229fa40b6db@mail.gmail.com>
References: <d38f5330608111507s1407a1damacf9f229fa40b6db@mail.gmail.com>
Message-ID: <44DE551F.8050609@v.loewis.de>

Alexander Belopolsky schrieb:
> I know that it is a subject of an annual discussion, but since I could
> not find any mention of it in the last year, let me ask this question
> again: why is file.readinto "Undocumented"? It is quite useful,
> particularly with numpy.
> 
> Would a patch adding "readinto" to StringIO be acceptable?

You mean, whether it would be accepted? Depends on the patch.
Or are you asking whether it would be accepted now? Definitely
not; no new features are allowed for Python 2.5.

readinto was documented as undocumented here


r23251 | tim_one | 2001-09-20 09:55:22 +0200 (Do, 20 Sep 2001) | 3 lines

SF bug [#463093] File methods need doc strings.
Now they don't.

I can only guess why it may go away; my guess it will go away when
the buffer interface is removed from Python (then it becomes
unimplementable).

Regards,
Martin

From tim.peters at gmail.com  Sun Aug 13 00:54:58 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 12 Aug 2006 18:54:58 -0400
Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk:
	Doc/api/abstract.tex Include/abstract.h Include/object.h
	Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c
	Modules/mmapmodule.c Modules/operator.c Objects/abstract.c
	Objects/class
Message-ID: <1f7befae0608121554p48ca2255s5179fd22ef34b978@mail.gmail.com>

[Travis E. Oliphant]
> Right now throughout the Python code it is assumed that
> sizeof(Py_ssize_t) <= sizeof(long).

If you find any code like that, it's a bug.  Win64 is a platform where
it's false.

From oliphant.travis at ieee.org  Sun Aug 13 02:05:21 2006
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Sat, 12 Aug 2006 18:05:21 -0600
Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk:
 Doc/api/abstract.tex Include/abstract.h Include/object.h
 Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c
 Modules/operator.c Objects/abstract.c Objects/class
In-Reply-To: <1f7befae0608121554p48ca2255s5179fd22ef34b978@mail.gmail.com>
References: <1f7befae0608121554p48ca2255s5179fd22ef34b978@mail.gmail.com>
Message-ID: <eblqc1$2uf$1@sea.gmane.org>

Tim Peters wrote:
> [Travis E. Oliphant]
>> Right now throughout the Python code it is assumed that
>> sizeof(Py_ssize_t) <= sizeof(long).
> 
> If you find any code like that, it's a bug.  Win64 is a platform where
> it's false.

Sorry for my confusion.  I meant to say that it is assumed that

sizeof(long) <= sizeof(Py_ssize_t)

because the assumption is that a Python Int (stored as long) will 
*always* fit into  into a Py_ssize_t.

I think this is true on all platforms.

-Travis


From guido at python.org  Sun Aug 13 04:28:44 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 12 Aug 2006 19:28:44 -0700
Subject: [Python-Dev] What is the status of file.readinto?
In-Reply-To: <44DE551F.8050609@v.loewis.de>
References: <d38f5330608111507s1407a1damacf9f229fa40b6db@mail.gmail.com>
	<44DE551F.8050609@v.loewis.de>
Message-ID: <ca471dc20608121928r2695f1b9s19d9159927be936@mail.gmail.com>

On 8/12/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I can only guess why it may go away; my guess it will go away when
> the buffer interface is removed from Python (then it becomes
> unimplementable).

In Py3k, the I/O APIs will be redesigned, especially the binary ones.
My current idea is to have read() on a binary file return a bytes
object. If readinto() continues to be necessary, please make sure the
Py3k list (python-3000 at python.org) knows about your use case. We
aren't quite writing up the I/O APIs in PEP-form, but when we do, that
would be the right time to speak up.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Aug 13 04:33:17 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 12 Aug 2006 19:33:17 -0700
Subject: [Python-Dev] 2.6 idea: a 'function' builtin to parallel
	classmethod and staticmethod
In-Reply-To: <44DD412E.9060109@iinet.net.au>
References: <44DD412E.9060109@iinet.net.au>
Message-ID: <ca471dc20608121933u77f4843cxbcf8681b9b79ffab@mail.gmail.com>

It ought to be called @instancemethod for a better analogy.

PS Nick how's the book coming along? :-)

--Guido

On 8/11/06, Nick Coghlan <ncoghlan at iinet.net.au> wrote:
> It's sometimes useful to be able to use an existing callable as a method of a
> new class. If the callable is a real function, this is easy. You just
> including the following line in the class definition:
>
>    method = some_callable
>
> However, callable objects without a function-like __get__ method can't be used
> that way. So, to avoid a dependency on an implementation detail of
> some_callable (i.e. whether or not it is a true function object), you have to
> write:
>
>
>    def method():
>        return some_callable()
>
> (and you can lose useful metadata in the process!)
>
> However, if you're adding a callable as a class method or static method, there
> is OOWTDI:
>
>    method = classmethod(some_callable)
>    method = staticmethod(some_callable)
>
> It would be nice if there was a similar mechanism for normal instance methods
> as well:
>
>    method = function(some_callable)
>
> This came up during the PEP 343 implementation - "context =
> contextlib.closing" is a tempting thing to write in order to provide a
> "x.context()" method for use in a with statement, but it doesn't actually work
> properly (because closing is a class, not a function).
>
> Similarly, you can't create a method simply by applying functools.partial to
> an existing function - the result won't have a __get__ method, so it will be
> treated like a normal attribute instead of as an instance method.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>              http://www.boredomandlaziness.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From alexander.belopolsky at gmail.com  Sun Aug 13 05:19:27 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sat, 12 Aug 2006 23:19:27 -0400
Subject: [Python-Dev] What is the status of file.readinto?
In-Reply-To: <44DE551F.8050609@v.loewis.de>
References: <d38f5330608111507s1407a1damacf9f229fa40b6db@mail.gmail.com>
	<44DE551F.8050609@v.loewis.de>
Message-ID: <d38f5330608122019r6c59a6bdl650cb8c5cec40429@mail.gmail.com>

On 8/12/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Alexander Belopolsky schrieb:
> > Would a patch adding "readinto" to StringIO be acceptable?
>
> You mean, whether it would be accepted? Depends on the patch.

Here is the patch:
https://sourceforge.net/tracker/index.php?func=detail&aid=1539381&group_id=5470&atid=305470

From ncoghlan at gmail.com  Sun Aug 13 08:31:06 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 13 Aug 2006 16:31:06 +1000
Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk:
 Doc/api/abstract.tex Include/abstract.h Include/object.h
 Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c
 Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/
In-Reply-To: <eblcuo$5ea$1@sea.gmane.org>
References: <ee2a432c0608121013q5a3a3043od63325c332b1d0ba@mail.gmail.com>
	<eblcuo$5ea$1@sea.gmane.org>
Message-ID: <44DEC72A.7020007@gmail.com>

Travis E. Oliphant wrote:
> Internally PyNumber_AsSize_t makes a call to PyNumber_Index, and 
> PyNumber_Index also calls the PyIndex_Check as well .  So, basically we 
> end up calling PyIndex_Check(obj) 2 times when only one check should be 
> necessary.
> 
> This code could be re-written to move any other type checks first and 
> replace the PyIndex_Check(obj) code with PyNumber_Index(obj) and error 
> handling but I'm not sure if that's the right way to go or if it's worth 
> it.

This concern was what lead me up the garden path with the more complicated C 
API in my patch. I've since become convinced that compared to everything else 
going on, the repetition of the 3 not-NULL checks performed by that macro 
really isn't worth worrying about (particularly at this stage of the release 
cycle).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From scott+python-dev at scottdial.com  Mon Aug 14 00:46:27 2006
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Sun, 13 Aug 2006 18:46:27 -0400
Subject: [Python-Dev] dict containment annoyance
In-Reply-To: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm>
References: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm>
Message-ID: <44DFABC3.2010201@scottdial.com>

Jean-Paul Calderone wrote:
> def blacklisted(o):
>     try:
>         # Is the object contained in the blacklist set?
>         return o in _blacklistset
>     except TypeError:
>         # If it *cannot* be contained in the blacklist set,
>         # then it probably isn't.
>         return False

I feel confident that the parent could've wrote this snippet himself. 
His point was to discuss the semantic logic of "x in y" throwing and 
exception when x can't be in y (because of the TypeError you are 
catching in your snippet).

FWIW, I think the logic of swallowing the TypeError is completely 
reasonable. It is surprising in a way that you are presented with an 
exception. However, I can't say I have ever ran into this.

For my money, it would make the most sense to have your own blacklistset 
type.

class BlacklistSet(dict):
     def __contains__(self, x):
         try:
             hash(x)
         except TypeError:
             return False
         return super(BlacklistSet, self).__contains__(x)

It's however unfortunate that you cannot use the handy curly-braces anymore.

-- 
Scott Dial
scott at scottdial.com
scodial at indiana.edu

From nnorwitz at gmail.com  Mon Aug 14 02:48:20 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sun, 13 Aug 2006 17:48:20 -0700
Subject: [Python-Dev] test_socketserver failure on cygwin
Message-ID: <ee2a432c0608131748l10aed501jefff93b53898d3f5@mail.gmail.com>

I'm not sure if this is a race condition in the test, the test using
SocketServer inappropriately, or if the failure is exposing a problem
in SocketServer.

The issue is that when the second (UDP) server is setup, there is
still a child process from the first (TCP) server.  So when the UDP
server calls collect_children(), it gets back the pid from the TCP
server, which is not in the UDP server's list and an exception is
raised.  In general, this is a problem, if program is using a
ForkingServer and spawns it's own children, this problem can occur.
This is because os.waitpid() is called with a pid of 0, rather than
the individual pids in the list.  I suppose that would fix the
problem.  Is this worth fixing?  (It's a pain to handle properly
because there is also use of a blocking wait.)

http://www.python.org/dev/buildbot/all/x86%20cygwin%20trunk/builds/1107/step-test/0

Add this patch corrects the problem (see below for more details):

Index: Lib/test/test_socketserver.py
===================================================================
--- Lib/test/test_socketserver.py       (revision 51260)
+++ Lib/test/test_socketserver.py       (working copy)
@@ -183,7 +183,11 @@

 def testall():
     testloop(socket.AF_INET, tcpservers, MyStreamHandler, teststream)
+    time.sleep(.1)
+    reap_children()
     testloop(socket.AF_INET, udpservers, MyDatagramHandler, testdgram)
+    time.sleep(.1)
+    reap_children()
     if hasattr(socket, 'AF_UNIX'):
         testloop(socket.AF_UNIX, streamservers, MyStreamHandler, teststream)
         # Alas, on Linux (at least) recvfrom() doesn't return a meaningful

So modifying SocketServer like this:
-bash-3.1$ svn diff Lib/SocketServer.py Lib/test/test_socketserver.py
Index: Lib/SocketServer.py
===================================================================
--- Lib/SocketServer.py (revision 51260)
+++ Lib/SocketServer.py (working copy)
@@ -420,6 +420,7 @@
             except os.error:
                 pid = None
             if not pid: break
+            print >> sys.__stdout__, 'pid', pid, 'active',
self.active_children             self.active_children.remove(pid)

     def process_request(self, request, client_address):
@@ -428,6 +429,7 @@
         pid = os.fork()
         if pid:
             # Parent process
+            print >> sys.__stdout__, 'in parent, adding', pid,
self.active_children
             if self.active_children is None:
                 self.active_children = []
             self.active_children.append(pid)

We see this on error (prior to the test patch above):

test_socketserver
in parent, adding 3084 None
pid 3084 active [3084]
in parent, adding 2408 []
pid 2408 active [2408]
in parent, adding 3908 []
in parent, adding 532 None
pid 3908 active [532]
Exception in thread Thread-12:
Traceback (most recent call last):
 ...
 File "/home/neal/build/python/trunk/Lib/SocketServer.py", line 424,
in collect_children
    self.active_children.remove(pid)
ValueError: list.remove(x): x not in list

From guido at python.org  Mon Aug 14 03:49:29 2006
From: guido at python.org (Guido van Rossum)
Date: Sun, 13 Aug 2006 18:49:29 -0700
Subject: [Python-Dev] dict containment annoyance
In-Reply-To: <44DFABC3.2010201@scottdial.com>
References: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm>
	<44DFABC3.2010201@scottdial.com>
Message-ID: <ca471dc20608131849m1bc2b4f1jf0b54bc2d2d075ed@mail.gmail.com>

On 8/13/06, Scott Dial <scott+python-dev at scottdial.com> wrote:
> FWIW, I think the logic of swallowing the TypeError is completely
> reasonable.

Then you haven't debugged enough Python programs.

Swallowing an exception of *any* kind is always a trap waiting to shut
when you least expect it, because you have no control over what other
operations might cause an exception.

E.g. when you write

try:
  hash(x)
except TypeError:
  # apparently x is not hashable

then you're also swallowing any type errors in the computation of a
legitimate hash function.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Mon Aug 14 04:26:14 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 14 Aug 2006 14:26:14 +1200
Subject: [Python-Dev] NotHashableError? (Re: dict containment annoyance)
In-Reply-To: <ca471dc20608131849m1bc2b4f1jf0b54bc2d2d075ed@mail.gmail.com>
References: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm>
	<44DFABC3.2010201@scottdial.com>
	<ca471dc20608131849m1bc2b4f1jf0b54bc2d2d075ed@mail.gmail.com>
Message-ID: <44DFDF46.8060404@canterbury.ac.nz>

Guido van Rossum wrote:

> try:
>   hash(x)
> except TypeError:
>   # apparently x is not hashable
> 
> then you're also swallowing any type errors in the computation of a
> legitimate hash function.

Maybe it would help if there were a specific exception,
such as NotHashableError, that hash functions were
expected to raise in this situation instead of a
generic TypeError.

For backwards compatibility it could be a subclass
of TypeError.

--
Greg

From scott+python-dev at scottdial.com  Mon Aug 14 05:49:09 2006
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Sun, 13 Aug 2006 23:49:09 -0400
Subject: [Python-Dev] NotHashableError? (Re: dict containment annoyance)
In-Reply-To: <44DFDF46.8060404@canterbury.ac.nz>
References: <20060812170752.1717.1940170045.divmod.quotient.20817@ohm>	<44DFABC3.2010201@scottdial.com>	<ca471dc20608131849m1bc2b4f1jf0b54bc2d2d075ed@mail.gmail.com>
	<44DFDF46.8060404@canterbury.ac.nz>
Message-ID: <44DFF2B5.4050902@scottdial.com>

Greg Ewing wrote:
> Guido van Rossum wrote:
>> try:
>>   hash(x)
>> except TypeError:
>>   # apparently x is not hashable
>>
>> then you're also swallowing any type errors in the computation of a
>> legitimate hash function.
> 
> Maybe it would help if there were a specific exception,
> such as NotHashableError, that hash functions were
> expected to raise in this situation instead of a
> generic TypeError.

I was going to suggest this, but then I realized that what is really 
desirable is a check for hashability. But since hashability is a deep 
property, it is difficult to check for.. seems like an very specific 
exception type would be a simple solution. FYI, I appreciate that 
swallowing all TypeErrors is bad form, so it would be nice if it was 
possible to differentiate ;-)

Some brief googling on the subject yield a discussion on py3k about this 
very subject.

http://mail.python.org/pipermail/python-3000/2006-July/002697.html

Guido wrote:
 > Personally, I'm not sure this problem needs solving; I don't recall
 > ever needing to know whether something is hashable. So perhaps it's of
 > purely theoretical importance? That would suit me fine given the above
 > dilemma...
 >
 > --Guido


-- 
Scott Dial
scott at scottdial.com
scodial at indiana.edu

From ncoghlan at gmail.com  Mon Aug 14 11:10:20 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 14 Aug 2006 19:10:20 +1000
Subject: [Python-Dev] 2.6 idea: a 'function' builtin to parallel
 classmethod and staticmethod
In-Reply-To: <ca471dc20608121933u77f4843cxbcf8681b9b79ffab@mail.gmail.com>
References: <44DD412E.9060109@iinet.net.au>
	<ca471dc20608121933u77f4843cxbcf8681b9b79ffab@mail.gmail.com>
Message-ID: <44E03DFC.2010600@gmail.com>

Guido van Rossum wrote:
> It ought to be called @instancemethod for a better analogy.

That's actually what I was originally going to suggest.

And then I changed my mind after wondering how I would answer the question 
"Why don't I need to decorate functions with @instancemethod in order to use 
them as instance methods, but I need to decorate them with @classmethod and 
@staticmethod to use them as class and static methods?".

However, now that I've thought about it some more, the explanation would 
simply be that functions behave like instancemethod instances by default, 
while other callables typically behave like staticmethod instances.

So instancemethod(f) would return the passed in object for an actual function, 
and a wrapper with an appropriate __get__ method for anything else (stealing 
__doc__ etc from the original callable).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mal at egenix.com  Mon Aug 14 12:58:48 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 14 Aug 2006 12:58:48 +0200
Subject: [Python-Dev] Dict suppressing exceptions
In-Reply-To: <44DCF26E.6090003@egenix.com>
References: <ca471dc20608091430g3e9ba573t824dd76f664fac1c@mail.gmail.com>	<44DAF23A.103@egenix.com>	<ca471dc20608100736h3b6693d8pd7397ab105d719ca@mail.gmail.com>	<44DB84EE.8020606@v.loewis.de>	<20060810211254.GA24181@code0.codespeak.net>	<ca471dc20608101436n1a63f74sfe1c3f4d9e090059@mail.gmail.com>	<20060811092757.GA27929@code0.codespeak.net>	<ca471dc20608110756v497aa93aodb292a23abdf6a8@mail.gmail.com>	<44DCABF7.6040709@egenix.com>
	<44DCF26E.6090003@egenix.com>
Message-ID: <44E05768.2070207@egenix.com>

M.-A. Lemburg wrote:
> M.-A. Lemburg wrote:
>> Guido van Rossum wrote:
>>> Marc-Andre, how's the patch coming along?
>> I'm working on it.
>>
>> Since we only want equal compares to generate the warning,
>> I have to add a rich compare function to Unicode objects.
> 
> Here's an initial version:
> 
> http://sourceforge.net/tracker/index.php?func=detail&aid=1538956&group_id=5470&atid=305470
> 
> The title of the patch is slightly incorrect - SF doesn't allow
> more descriptive titles... :-(

FYI: I've checked in the patch as revision 51276.

Please give it some serious testing.

All tests pass, but there may still be some corner cases left
which the tests don't catch.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 14 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From fredrik at pythonware.com  Mon Aug 14 13:51:48 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 14 Aug 2006 13:51:48 +0200
Subject: [Python-Dev] Elementtree and Namespaces in 2.5
References: <beb7688e0608110815k44369b92tdd4dbcecd00b1a8@mail.gmail.com>
Message-ID: <ebpo4f$jd8$1@sea.gmane.org>

"Chris S" wrote:

> and while most users and the w3 spec
> (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting)
> agree this feature is actually a bug

ET's not a canonicalization library, and doesn't claim to be one, so that reference isn't
very relevant.  And "most users" know that ET uses a simplified infoset; it's not exactly
a secret.

I think you need to work on your FUD skills.

> at least patch the stdlib's version of Elementtree to produce an output
> more in line with the w3 standard.

And that patch would look exactly how ?

</F> 




From fredrik at pythonware.com  Mon Aug 14 13:58:40 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 14 Aug 2006 13:58:40 +0200
Subject: [Python-Dev] Elementtree and Namespaces in 2.5
References: <beb7688e0608110815k44369b92tdd4dbcecd00b1a8@mail.gmail.com>
	<44DCDC18.3000701@v.loewis.de>
Message-ID: <ebpoha$kir$1@sea.gmane.org>

Martin v. Löwis wrote:

> That is not enough reason. Yes, it makes certain applications
> impossible, e.g. when namespace prefixes are inside attribute
> values. It just means you can't use it for that application,
> then. XML has many other applications, and so does ElementTree.

there are ways to deal with this in ET, though; the "iterparse" interface gives you
access to namespace prefixes, the semi-official _namespace_map dictionary gives
you better control over how the default serializer maps URI:s to prefixes, and the
"QName" wrapper allows you to use qualified names for individual attribute values.

dealing with namespace abusers such as SOAP and similar XML dialects are per-
fectly possible, and usually not very hard.

(ET 1.3 will probably improve things further, but that won't be out before 2.5 final.)

</F> 




From fredrik at pythonware.com  Mon Aug 14 14:06:08 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 14 Aug 2006 14:06:08 +0200
Subject: [Python-Dev] SimpleXMLWriter missing from elementtree
References: <ebfa81$3kp$1@sea.gmane.org>
Message-ID: <ebpov9$m3u$1@sea.gmane.org>

Ralf Schmitt wrote:

> any chance to get SimpleXMLWriter (and other modules maybe??) getting
> included into xml.etree? Otherwise people might have to stick to the
> original elementtree, which doesn't really make sense, since most of
> elementtree already is included.

the original decision was to start with three fundamental modules, and
leave out less popular and potentially less stable modules.

leaving out SimpleXMLWriter (and perhaps the HTML stuff) might have
been a mistake, but with 2.5 in final beta, that's how things are going to be
in 2.5.

</F> 




From noamraph at gmail.com  Mon Aug 14 21:18:45 2006
From: noamraph at gmail.com (Noam Raphael)
Date: Mon, 14 Aug 2006 22:18:45 +0300
Subject: [Python-Dev] ANN: byteplay - a bytecode assembler/disassembler
Message-ID: <b348a0850608141218g265c3212qf811a1775876c801@mail.gmail.com>

Hello,

I wanted to tell you that I wrote a Python bytecode
assembler/disassembler, and would be happy if people tried it and said
what they think.

I send this message to this list because this module deals with pretty
low-level Python, so I thought it might interest the audience here. If
I was wrong - please forgive me.

The idea is to define an object which is equivalent to Python's code
object, but which is easy to work with. To explain what I mean, I'll
show a quick example. We can define this stupid function:

>>> def f(a, b):
...     print (a, b)

We can convert it to an equivalent object, and see how it stores the byte code:

>>> from byteplay import *
>>> c = Code.from_code(f.func_code)
>>> from pprint import pprint; pprint(c.code)
[(SetLineno, 2),
 (LOAD_FAST, 'a'),
 (LOAD_FAST, 'b'),
 (BUILD_TUPLE, 2),
 (PRINT_ITEM, None),
 (PRINT_NEWLINE, None),
 (LOAD_CONST, None),
 (RETURN_VALUE, None)]

We can change the bytecode easily, and see what happens:

>>> c.code[3:3] = [(ROT_TWO, None)]
>>> f.func_code = c.to_code()
>>> f(3, 5)
(5, 3)

The idea is basically the same as that of Michael Hudson's
bytecodehacks, but this one works with Python 2.4 and 2.5. I also
think that it's simpler to use. I borrowed some code from Phillip J.
Eby's peak.util.assembler - the main difference between his package
and mine is that mine lets you play with existing bytecode, not only
create new code objects.

I learned a lot about Python's bytecode from writing this, and I think
that other may learn from it as well - I think it's much easier to
understand how code objects work by understanding equivalent objects
which were meant to be as simple as possible, instead of as fast as
possible.

I think it got pretty good testing - I patched __import__ so that
after a module is imported, all function objects (found by the gc
module) were disassembled and assembled again. I then managed to get
the complete test suite to pass!

You can download the module from
http://byteplay.googlecode.com/svn/trunk/byteplay.py . I wrote a
documentation (which goes into some length in purpose of explaining
how bytecode works). It's on the wiki:
http://wiki.python.org/moin/ByteplayDoc .

I even thought that it might get to the standard library, because it
seemed to me to be a pretty good documentation of bytecode details,
and because it has a pretty straightforward interface. It also makes
meddling with bytecode much less dangerous - for example, you can see
Raymond's original recipe for binding constants at compile time (I
reposted it at http://python.pastebin.com/768312) and how using
byteplay makes it simple (posted at http://python.pastebin.com/768318
so you can view the diff). But, of course - it's up to you. I will be
entirely satisfied if people simply find it useful.

Have a good day,
Noam

From amk at amk.ca  Mon Aug 14 23:51:06 2006
From: amk at amk.ca (A.M. Kuchling)
Date: Mon, 14 Aug 2006 17:51:06 -0400
Subject: [Python-Dev] Arlington VA sprint on Sept. 23
Message-ID: <20060814215106.GA851@Siri.local>

The CanDo group continues to have sprints in Arlington, so we may as
well continue to piggyback on their space.  The next one will be
Saturday Sept. 23; sign up at
<http://wiki.python.org/moin/ArlingtonSprint>.

If the PEP 356 plan holds, Python 2.5 will have been released a week
and a half before.  I plan to work on the bug/patch backlog (try
searching for the Python 2.3 bugs on SF and see how many open bugs
there are).

--amk


From g.brandl at gmx.net  Tue Aug 15 00:21:34 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Aug 2006 00:21:34 +0200
Subject: [Python-Dev] Arlington VA sprint on Sept. 23
In-Reply-To: <20060814215106.GA851@Siri.local>
References: <20060814215106.GA851@Siri.local>
Message-ID: <ebqt1f$dqh$1@sea.gmane.org>

A.M. Kuchling wrote:
> The CanDo group continues to have sprints in Arlington, so we may as
> well continue to piggyback on their space.  The next one will be
> Saturday Sept. 23; sign up at
> <http://wiki.python.org/moin/ArlingtonSprint>.
> 
> If the PEP 356 plan holds, Python 2.5 will have been released a week
> and a half before.  I plan to work on the bug/patch backlog (try
> searching for the Python 2.3 bugs on SF and see how many open bugs
> there are).

Once we do have a new tracker system, it would be a Good Thing[tm]
to go through all the bugs, patches and RFE, and try to get them
properly categorized: we'd have to

* merge duplicates (also bug + patch duplicates caused by the current
   split trackers)
* assign a realistic priority
* assign a realistic milestone
* flag RFE patches as RFE ("patch" shouldn't be a category on its own)
* try to reproduce if bug is quite old
* close old bugs which can't be reproduced or refer to obscure platforms
* close RFEs and patches which have no chance of going in (there are many
   patches on SF having one or two "-1" comments, is anyone ever going to
   check them in without a python-dev discussion?)


Quite a lot of work, but with a fair amount of volunteers it should be
doable and cut the number of open issues to under 1000.

Georg


From oliphant.travis at ieee.org  Tue Aug 15 00:28:45 2006
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Mon, 14 Aug 2006 16:28:45 -0600
Subject: [Python-Dev] Fwd: [Python-checkins] r51236 - in python/trunk:
 Doc/api/abstract.tex Include/abstract.h Include/object.h
 Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c
 Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/
In-Reply-To: <eblbpc$2d5$1@sea.gmane.org>
References: <ee2a432c0608121013q5a3a3043od63325c332b1d0ba@mail.gmail.com>
	<eblbpc$2d5$1@sea.gmane.org>
Message-ID: <44E0F91D.1090206@ieee.org>

Travis E. Oliphant wrote:
> 
> The idea is that the __index__() method should return an exact int or 
> exact long or this call will raise an error.  The restriction is present 
> to remove the possibility of infinite recursion (though I'm not sure 
> where that would occur exactly).
> 

I just realized that returning a sub-class of Int and/or Long would not 
be a problem here.  The recursion problem that I was guarding against 
only arose when the possibility of returning any object with an 
.__index__() method was suggested.

Therefore, I think these exact int or exact long checks can and should 
be replaced with PyInt_Check() and PyLong_Check().

-Travis




From fdrake at acm.org  Tue Aug 15 00:34:08 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon, 14 Aug 2006 18:34:08 -0400
Subject: [Python-Dev] Arlington VA sprint on Sept. 23
In-Reply-To: <ebqt1f$dqh$1@sea.gmane.org>
References: <20060814215106.GA851@Siri.local> <ebqt1f$dqh$1@sea.gmane.org>
Message-ID: <200608141834.08899.fdrake@acm.org>

On Monday 14 August 2006 18:21, Georg Brandl wrote:
 > * flag RFE patches as RFE ("patch" shouldn't be a category on its own)

This is something Martin and I have disagreed over in the past.  Martin has 
indicated that he'd rather see the patches as separate artifacts rather than 
as attachments to a bug report, while I'd rather see them attached to the 
relevant bug report or feature request.

My thought is that it's easier to deal with fewer items in the tracker.  
Keeping the candidate patches with the bug report or feature request makes 
them readily accessible to a reviewer.  It's not the only way.

I can guess at Martin's thinking, but I'd rather let him speak for himself, 
since I'm not a trained channeller.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From alexander.belopolsky at gmail.com  Tue Aug 15 01:08:18 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 14 Aug 2006 19:08:18 -0400
Subject: [Python-Dev] Type of range object members
Message-ID: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>

The range object is currently defined in Objects/rangeobject.c as

typedef struct {
	PyObject_HEAD
	long	start;
	long	step;
	long	len;
} rangeobject;

Is this consistent with PEP 353, or should Py_ssize_t be used instead of long?

It looks like some of the code in rangeobject.c is already Py_ssize_t
aware (see range_item and range_length), but it assumes that it is
safe to cast long to ssize_t and back.

From guido at python.org  Tue Aug 15 01:32:26 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Aug 2006 16:32:26 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
Message-ID: <ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>

Methinks that as long as PyIntObject uses long (see intobject.h)
there's no point in changing this to long.

--Guido

On 8/14/06, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
> The range object is currently defined in Objects/rangeobject.c as
>
> typedef struct {
>         PyObject_HEAD
>         long    start;
>         long    step;
>         long    len;
> } rangeobject;
>
> Is this consistent with PEP 353, or should Py_ssize_t be used instead of long?
>
> It looks like some of the code in rangeobject.c is already Py_ssize_t
> aware (see range_item and range_length), but it assumes that it is
> safe to cast long to ssize_t and back.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From amk at amk.ca  Tue Aug 15 01:48:52 2006
From: amk at amk.ca (A.M. Kuchling)
Date: Mon, 14 Aug 2006 19:48:52 -0400
Subject: [Python-Dev] Arlington VA sprint on Sept. 23
In-Reply-To: <ebqt1f$dqh$1@sea.gmane.org>
References: <20060814215106.GA851@Siri.local> <ebqt1f$dqh$1@sea.gmane.org>
Message-ID: <20060814234852.GB17594@Andrew-iBook2.local>

On Tue, Aug 15, 2006 at 12:21:34AM +0200, Georg Brandl wrote:
> * close RFEs and patches which have no chance of going in (there are many
>   patches on SF having one or two "-1" comments, is anyone ever going to
>   check them in without a python-dev discussion?)

In the past, someone (I think Raymond) has argued that we shouldn't
close old bug reports because even if no one is interested in
processing them *now*, someone might come along someday who wants to
update that module and fix those bugs or apply those enhancements.

Perhaps we need a 'not dead, not actively pursued' state for patches
and RFEs.  

--amk


From ncoghlan at gmail.com  Tue Aug 15 03:54:03 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Aug 2006 11:54:03 +1000
Subject: [Python-Dev] Arlington VA sprint on Sept. 23
In-Reply-To: <20060814234852.GB17594@Andrew-iBook2.local>
References: <20060814215106.GA851@Siri.local> <ebqt1f$dqh$1@sea.gmane.org>
	<20060814234852.GB17594@Andrew-iBook2.local>
Message-ID: <44E1293B.2000607@gmail.com>

A.M. Kuchling wrote:
> On Tue, Aug 15, 2006 at 12:21:34AM +0200, Georg Brandl wrote:
>> * close RFEs and patches which have no chance of going in (there are many
>>   patches on SF having one or two "-1" comments, is anyone ever going to
>>   check them in without a python-dev discussion?)
> 
> In the past, someone (I think Raymond) has argued that we shouldn't
> close old bug reports because even if no one is interested in
> processing them *now*, someone might come along someday who wants to
> update that module and fix those bugs or apply those enhancements.
> 
> Perhaps we need a 'not dead, not actively pursued' state for patches
> and RFEs.  

Open with priority 0?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Tue Aug 15 03:59:46 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Aug 2006 11:59:46 +1000
Subject: [Python-Dev] Arlington VA sprint on Sept. 23
In-Reply-To: <200608141834.08899.fdrake@acm.org>
References: <20060814215106.GA851@Siri.local> <ebqt1f$dqh$1@sea.gmane.org>
	<200608141834.08899.fdrake@acm.org>
Message-ID: <44E12A92.7090208@gmail.com>

Fred L. Drake, Jr. wrote:
> On Monday 14 August 2006 18:21, Georg Brandl wrote:
>  > * flag RFE patches as RFE ("patch" shouldn't be a category on its own)
> 
> This is something Martin and I have disagreed over in the past.  Martin has 
> indicated that he'd rather see the patches as separate artifacts rather than 
> as attachments to a bug report, while I'd rather see them attached to the 
> relevant bug report or feature request.
> 
> My thought is that it's easier to deal with fewer items in the tracker.  
> Keeping the candidate patches with the bug report or feature request makes 
> them readily accessible to a reviewer.  It's not the only way.

I'd rather see this as a "has patch" flag, similar to "is RFE", rather than 
completely separate trackers. A patch will be aimed at addressing either an 
RFE or a bug fix - so long as people can filter out the tracker items with 
patches attached to find things to review, it makes sense to me to keep the 
patch with the tracker item it addresses.

Another useful categorization would be "what skills are needed to address this 
item" - Python coding, C coding or documentation writing (although this is 
covered to some degree by the existing categories, it may not always be obvious).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Tue Aug 15 04:01:46 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 15 Aug 2006 12:01:46 +1000
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>
Message-ID: <44E12B0A.9020907@gmail.com>

Guido van Rossum wrote:
> Methinks that as long as PyIntObject uses long (see intobject.h)
> there's no point in changing this to long.

Those fields are going to have to change to Py_Object* eventually if xrange() 
is going to become the range() replacement in Py3k. . .

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From alexander.belopolsky at gmail.com  Tue Aug 15 04:08:19 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 14 Aug 2006 22:08:19 -0400
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>
Message-ID: <04EA6FD8-CECD-4766-93B0-BE7137316669@local>


On Aug 14, 2006, at 7:32 PM, Guido van Rossum wrote:

> Methinks that as long as PyIntObject uses long (see intobject.h)
> there's no point in changing this to long.

I guess you meant "changing this to Py_ssize_t ".  I don't understand  
why the type used by PyIntObject is relevant here.  Range object's  
"start" is logically  an index, but int object's "ob_ival" is not.    
Since PyIntObject's is definition is exposed by Python.h, changing  
the type of ob_ival will probably break a lot of code.  This  
reasoning does not apply to the range object.

Since on most platforms ssize_t is the same as long, the choice  
between the two is just a matter of self-documenting code.  Speaking  
of which, I find it unfortunate that the name Py_ssize_t was selected  
for the typedef.  I would prefer Py_index_t.  The first time I saw  
Py_ssize_t, I did not notice the double 's' and thought it was an  
unsigned type.  On the second look, I've realized that it is signed  
and started wondering why not ptrdiff_t.  I understand that ssize_t  
is defined by POSIX as the return type of functions such as "read"  
that can return either size or -1 for error.  I don't think POSIX  
mandates sizeof(size_t) == sizeof(ssize_t), but I may be wrong.   I  
would agree that ptrdiff_t, although standard C, is not a very  
intuitive name, but ssize_t is even less clear. 

From guido at python.org  Tue Aug 15 04:56:55 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Aug 2006 19:56:55 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <04EA6FD8-CECD-4766-93B0-BE7137316669@local>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>
	<04EA6FD8-CECD-4766-93B0-BE7137316669@local>
Message-ID: <ca471dc20608141956i59ce1453l2268e8e4959997c5@mail.gmail.com>

On 8/14/06, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
>
> On Aug 14, 2006, at 7:32 PM, Guido van Rossum wrote:
>
> > Methinks that as long as PyIntObject uses long (see intobject.h)
> > there's no point in changing this to long.
>
> I guess you meant "changing this to Py_ssize_t ".

Yup, sorry.

> I don't understand
> why the type used by PyIntObject is relevant here.

Because the only way to create one (in 2.5 or before) is by passing it
a Python int.

> Range object's
> "start" is logically  an index, but int object's "ob_ival" is not.
> Since PyIntObject's is definition is exposed by Python.h, changing
> the type of ob_ival will probably break a lot of code.  This
> reasoning does not apply to the range object.

But since the start and end come from a Python int, what advantage
would it have to use Py_ssize_t instead? We know sizeof(long) <=
sizeof(Py_ssize_t).

> Since on most platforms ssize_t is the same as long, the choice
> between the two is just a matter of self-documenting code.  Speaking
> of which, I find it unfortunate that the name Py_ssize_t was selected
> for the typedef.  I would prefer Py_index_t.  The first time I saw
> Py_ssize_t, I did not notice the double 's' and thought it was an
> unsigned type.

Blame the C standards committee -- they introduced ssize_t in C99.

> On the second look, I've realized that it is signed
> and started wondering why not ptrdiff_t.

Because it is not used for the difference of pointers?

Frankly, you're about 6 months too late with naming concerns. It's not
going to change now. The PEP has been discussed and the code reviewed
ad infinitum. We're only looking for bugs now, and so far the issue
you've brought up decidedly falls in the category "non-bug".

> I understand that ssize_t
> is defined by POSIX as the return type of functions such as "read"
> that can return either size or -1 for error.  I don't think POSIX
> mandates sizeof(size_t) == sizeof(ssize_t), but I may be wrong.   I
> would agree that ptrdiff_t, although standard C, is not a very
> intuitive name, but ssize_t is even less clear.

Water under the bridge.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From alexander.belopolsky at gmail.com  Tue Aug 15 05:58:49 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 14 Aug 2006 23:58:49 -0400
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608141956i59ce1453l2268e8e4959997c5@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>
	<04EA6FD8-CECD-4766-93B0-BE7137316669@local>
	<ca471dc20608141956i59ce1453l2268e8e4959997c5@mail.gmail.com>
Message-ID: <80BC7ADD-01E0-41E5-B5B5-021765C6C865@local>


On Aug 14, 2006, at 10:56 PM, Guido van Rossum wrote:

>
> Because the only way to create one (in 2.5 or before) is by passing it
> a Python int.
>

Is that true?

Python 2.4.2 (#2, Jan 13 2006, 12:00:38)
 >>> xrange(long(2))
xrange(2)


>
> But since the start and end come from a Python int, what advantage
> would it have to use Py_ssize_t instead? We know sizeof(long) <=
> sizeof(Py_ssize_t).

They don't have to come from a python int, they can come from a  
long.    I guess range_new would have to be changed to the 'n'  
conversion code instead of 'l' to do the proper conversion.    
Similarly, rangeiter_next will need to use PyInt_FromSsize_t .


>
> Blame the C standards committee -- they introduced ssize_t in C99.

I've checked in the Wiley's 2003 edition of "The C Standard" and it  
does not have ssize_t .  I could not find anything suggesting that it  
is in the C standard on google either.  Are you sure it is in C99?


>
>> On the second look, I've realized that it is signed
>> and started wondering why not ptrdiff_t.
>
> Because it is not used for the difference of pointers?

After being exposed to C++, ptrdiff_t (or more generally  
difference_type) is what I expect as an argument of a [] operator  
that allows negative indices (such as operator[] of C++ iterators).

>
> Frankly, you're about 6 months too late with naming concerns.

I know :-(  I'll shut up now.


From guido at python.org  Tue Aug 15 06:14:09 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 14 Aug 2006 21:14:09 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <80BC7ADD-01E0-41E5-B5B5-021765C6C865@local>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>
	<04EA6FD8-CECD-4766-93B0-BE7137316669@local>
	<ca471dc20608141956i59ce1453l2268e8e4959997c5@mail.gmail.com>
	<80BC7ADD-01E0-41E5-B5B5-021765C6C865@local>
Message-ID: <ca471dc20608142114hb5a7220sa5e4a45b184cc77d@mail.gmail.com>

On 8/14/06, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
>
> On Aug 14, 2006, at 10:56 PM, Guido van Rossum wrote:
>
> > Because the only way to create one (in 2.5 or before) is by passing it
> > a Python int.
>
> Is that true?
>
> Python 2.4.2 (#2, Jan 13 2006, 12:00:38)
>  >>> xrange(long(2))
> xrange(2)

But a long with a value larger than sys.maxint is never accepted right?

> > But since the start and end come from a Python int, what advantage
> > would it have to use Py_ssize_t instead? We know sizeof(long) <=
> > sizeof(Py_ssize_t).
>
> They don't have to come from a python int, they can come from a
> long.    I guess range_new would have to be changed to the 'n'
> conversion code instead of 'l' to do the proper conversion.
> Similarly, rangeiter_next will need to use PyInt_FromSsize_t .

Feel free to submit a patch for Python 2.6. I expect the 2.5 release
managers to reject this on the basis of being a new feature.

> > Blame the C standards committee -- they introduced ssize_t in C99.
>
> I've checked in the Wiley's 2003 edition of "The C Standard" and it
> does not have ssize_t .  I could not find anything suggesting that it
> is in the C standard on google either.  Are you sure it is in C99?

To be honest I have no idea how/why Martin or Tim picked this name.
Perhaps it is in POSIX? I doubt they made it up; ssize_t is used all
over the standard headers in the 4-year-old Linux distro (Red Hat 7.3)
running on my ancient home desktop.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nnorwitz at gmail.com  Tue Aug 15 06:31:16 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 14 Aug 2006 21:31:16 -0700
Subject: [Python-Dev] no remaining issues blocking 2.5 release
Message-ID: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>

I just updated the PEP to remove all references to issues blocking
release of 2.5.
I don't know of any.  I haven't heard of any issues with the fixes
that have been checked in.

If you have issues, respond ASAP!  The release candidate is planned to
be cut this Thursday/Friday.  There are only a few more days before
code freeze.  A branch will be made when the release candidate is cut.

n

From anthony at interlink.com.au  Tue Aug 15 07:32:09 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue, 15 Aug 2006 15:32:09 +1000
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608142114hb5a7220sa5e4a45b184cc77d@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<80BC7ADD-01E0-41E5-B5B5-021765C6C865@local>
	<ca471dc20608142114hb5a7220sa5e4a45b184cc77d@mail.gmail.com>
Message-ID: <200608151532.11881.anthony@interlink.com.au>

On Tuesday 15 August 2006 14:14, Guido van Rossum wrote:
> > They don't have to come from a python int, they can come from a
> > long.    I guess range_new would have to be changed to the 'n'
> > conversion code instead of 'l' to do the proper conversion.
> > Similarly, rangeiter_next will need to use PyInt_FromSsize_t .
>
> Feel free to submit a patch for Python 2.6. I expect the 2.5 release
> managers to reject this on the basis of being a new feature.

Absolutely correct. At this point in the release process, there's going to be 
a massive resistance to changes.


Anthony

-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From anthony at interlink.com.au  Tue Aug 15 07:36:46 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue, 15 Aug 2006 15:36:46 +1000
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
Message-ID: <200608151536.48043.anthony@interlink.com.au>

On Tuesday 15 August 2006 14:31, Neal Norwitz wrote:
> I just updated the PEP to remove all references to issues blocking
> release of 2.5.
> I don't know of any.  I haven't heard of any issues with the fixes
> that have been checked in.
>
> If you have issues, respond ASAP!  The release candidate is planned to
> be cut this Thursday/Friday.  There are only a few more days before
> code freeze.  A branch will be made when the release candidate is cut.

More information here: I plan to branch release25-maint on Thursday morning, 
as the first step in the release process for 2.5c1. This will be done after 
the repository is updated to identify itself as '2.5c1'. Once this is done, I 
plan on doing all further 2.5 releases from the release25-maint branch. I 
will no longer care about the trunk's stability :-) 

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From fredrik at pythonware.com  Tue Aug 15 08:17:09 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 15 Aug 2006 08:17:09 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608142114hb5a7220sa5e4a45b184cc77d@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	<ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>	<04EA6FD8-CECD-4766-93B0-BE7137316669@local>	<ca471dc20608141956i59ce1453l2268e8e4959997c5@mail.gmail.com>	<80BC7ADD-01E0-41E5-B5B5-021765C6C865@local>
	<ca471dc20608142114hb5a7220sa5e4a45b184cc77d@mail.gmail.com>
Message-ID: <ebross$emq$1@sea.gmane.org>

Guido van Rossum wrote:

> To be honest I have no idea how/why Martin or Tim picked this name.
> Perhaps it is in POSIX?

it's from sys/types.h, which is a posix thing, afaik:

    http://www.opengroup.org/onlinepubs/007908799/xsh/systypes.h.html

</F>


From martin at v.loewis.de  Tue Aug 15 09:16:59 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Aug 2006 09:16:59 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
Message-ID: <44E174EB.1080305@v.loewis.de>

Alexander Belopolsky wrote:
> The range object is currently defined in Objects/rangeobject.c as
> 
> typedef struct {
> 	PyObject_HEAD
> 	long	start;
> 	long	step;
> 	long	len;
> } rangeobject;
> 
> Is this consistent with PEP 353, or should Py_ssize_t be used instead of long?

As others have said: no. The range object produces ints. People have
asked to make it produce arbitrary longs instead, but none of these
patches ever got successfully reviewed.

> It looks like some of the code in rangeobject.c is already Py_ssize_t
> aware (see range_item and range_length), but it assumes that it is
> safe to cast long to ssize_t and back.

Where does it assume that it is safe to case ssize_t -> long?
That would be a bug.

Regards,
Martin

From martin at v.loewis.de  Tue Aug 15 09:25:35 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Aug 2006 09:25:35 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <04EA6FD8-CECD-4766-93B0-BE7137316669@local>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	<ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>
	<04EA6FD8-CECD-4766-93B0-BE7137316669@local>
Message-ID: <44E176EF.2080707@v.loewis.de>

Alexander Belopolsky wrote:
> Since on most platforms ssize_t is the same as long, the choice  
> between the two is just a matter of self-documenting code.

No, it would be an actual change: on Win64,
sizeof(Py_ssize_t)>sizeof(long).

> Speaking  
> of which, I find it unfortunate that the name Py_ssize_t was selected  
> for the typedef.  I would prefer Py_index_t.  The first time I saw  
> Py_ssize_t, I did not notice the double 's' and thought it was an  
> unsigned type.

Hmm. That you fail to read it correctly can hardly be an argument
against it.

> On the second look, I've realized that it is signed  
> and started wondering why not ptrdiff_t.  I understand that ssize_t  
> is defined by POSIX as the return type of functions such as "read"  
> that can return either size or -1 for error.  I don't think POSIX  
> mandates sizeof(size_t) == sizeof(ssize_t), but I may be wrong.

In the rationale (XRAT) they say

"This is intended to be a signed analog of size_t."

They don't mandate it to have the same size, but it is the expectation
that implementations typically will.

> I would agree that ptrdiff_t, although standard C, is not a very  
> intuitive name, but ssize_t is even less clear. 

In the discussion, ptrdiff_t was never brought up as an alternative
(intptr_t was; see the PEP for why it is unsuitable). ssize_t seemed
most natural to me since size_t is the C "number of bytes" type,
and we need a signed version of it.

Regards,
Martin

From martin at v.loewis.de  Tue Aug 15 09:30:12 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Aug 2006 09:30:12 +0200
Subject: [Python-Dev] Arlington VA sprint on Sept. 23
In-Reply-To: <200608141834.08899.fdrake@acm.org>
References: <20060814215106.GA851@Siri.local> <ebqt1f$dqh$1@sea.gmane.org>
	<200608141834.08899.fdrake@acm.org>
Message-ID: <44E17804.204@v.loewis.de>

Fred L. Drake, Jr. wrote:
> I can guess at Martin's thinking, but I'd rather let him speak for himself, 
> since I'm not a trained channeller.  ;-)

Essentially, I want to give patches more attention, since they are
larger contributions. I don't care if bug reports get unnoticed for
years; I do worry if patches get unnoticed/unprocessed for years.

Regards,
Martin

From g.brandl at gmx.net  Tue Aug 15 10:18:07 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 15 Aug 2006 10:18:07 +0200
Subject: [Python-Dev] Arlington VA sprint on Sept. 23
In-Reply-To: <44E17804.204@v.loewis.de>
References: <20060814215106.GA851@Siri.local>
	<ebqt1f$dqh$1@sea.gmane.org>	<200608141834.08899.fdrake@acm.org>
	<44E17804.204@v.loewis.de>
Message-ID: <ebrvvv$1l3$1@sea.gmane.org>

Martin v. L?wis wrote:
> Fred L. Drake, Jr. wrote:
>> I can guess at Martin's thinking, but I'd rather let him speak for himself, 
>> since I'm not a trained channeller.  ;-)
> 
> Essentially, I want to give patches more attention, since they are
> larger contributions. I don't care if bug reports get unnoticed for
> years; I do worry if patches get unnoticed/unprocessed for years.

Then a "has-patch" flag would be okay, wouldn't it, along with assigning
a higher priority?

Georg


From martin at v.loewis.de  Tue Aug 15 10:43:41 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Aug 2006 10:43:41 +0200
Subject: [Python-Dev] Arlington VA sprint on Sept. 23
In-Reply-To: <ebrvvv$1l3$1@sea.gmane.org>
References: <20060814215106.GA851@Siri.local>	<ebqt1f$dqh$1@sea.gmane.org>	<200608141834.08899.fdrake@acm.org>	<44E17804.204@v.loewis.de>
	<ebrvvv$1l3$1@sea.gmane.org>
Message-ID: <44E1893D.7030402@v.loewis.de>

Georg Brandl wrote:
>>> I can guess at Martin's thinking, but I'd rather let him speak for himself, 
>>> since I'm not a trained channeller.  ;-)
>> Essentially, I want to give patches more attention, since they are
>> larger contributions. I don't care if bug reports get unnoticed for
>> years; I do worry if patches get unnoticed/unprocessed for years.
> 
> Then a "has-patch" flag would be okay, wouldn't it, along with assigning
> a higher priority?

Sure, that would work as well.

Regards,
Martin

From theller at python.net  Tue Aug 15 14:57:36 2006
From: theller at python.net (Thomas Heller)
Date: Tue, 15 Aug 2006 14:57:36 +0200
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
Message-ID: <ebsgc0$jkk$1@sea.gmane.org>

Neal Norwitz schrieb:
> I just updated the PEP to remove all references to issues blocking
> release of 2.5.
> I don't know of any.  I haven't heard of any issues with the fixes
> that have been checked in.
> 
> If you have issues, respond ASAP!  The release candidate is planned to
> be cut this Thursday/Friday.  There are only a few more days before
> code freeze.  A branch will be made when the release candidate is cut.

What is the policy for documentation changes?  Am I allowed to check in
changes/additions to the ctypes docs without release manager permission
after the release candidate is out?  I'l always make sure that the html
docs can be built.

Thanks,
Thomas


From mal at egenix.com  Tue Aug 15 16:06:18 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Aug 2006 16:06:18 +0200
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
Message-ID: <44E1D4DA.2080002@egenix.com>

Neal Norwitz wrote:
> I just updated the PEP to remove all references to issues blocking
> release of 2.5.
> I don't know of any.  I haven't heard of any issues with the fixes
> that have been checked in.
> 
> If you have issues, respond ASAP!  The release candidate is planned to
> be cut this Thursday/Friday.  There are only a few more days before
> code freeze.  A branch will be made when the release candidate is cut.

The distutils version number should be changed back to a static
string literal.

It's currently setup to get its version number
from the Python version running it which pretty much defeats
the whole purpose of having a version number and makes using the
SVN distutils package with other Python versions problematic.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 15 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From alexander.belopolsky at gmail.com  Tue Aug 15 16:32:45 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 15 Aug 2006 10:32:45 -0400
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608142114hb5a7220sa5e4a45b184cc77d@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>
	<04EA6FD8-CECD-4766-93B0-BE7137316669@local>
	<ca471dc20608141956i59ce1453l2268e8e4959997c5@mail.gmail.com>
	<80BC7ADD-01E0-41E5-B5B5-021765C6C865@local>
	<ca471dc20608142114hb5a7220sa5e4a45b184cc77d@mail.gmail.com>
Message-ID: <25F486F7-B449-4FE1-8C11-DCEFDDE4CBE2@local>


On Aug 15, 2006, at 12:14 AM, Guido van Rossum wrote:

>
> Feel free to submit a patch for Python 2.6.

Please see http://sourceforge.net/tracker/index.php? 
func=detail&aid=1540617&group_id=5470&atid=305470


> Perhaps it is in POSIX?

Yes, "ssize_t - Used for a count of bytes or an error indication.",  
defined in sys/types.h .

I promised to shut up, but let me respond to Martin:

On Aug 15, 2006, at 3:25 AM, Martin v. L?wis wrote:
> Alexander Belopolsky wrote:
>> [snip] I did not notice the double 's' and thought it was an
>> unsigned type.
>
> Hmm. That you fail to read it correctly can hardly be an argument
> against it.

Google tells me I am not the only one: <http://dbforums.com/ 
showthread.php?t=1430615>.
What happened to "readability counts"?


>
> In the rationale (XRAT) they say
>
> "This is intended to be a signed analog of size_t."

That's POSIX rationale <http://www.opengroup.org/onlinepubs/009695399/ 
xrat/xsh_chap02.html>. PEP 353 says: "A new type Py_ssize_t is  
introduced, which has the same size as the compiler's size_t type,  
but is signed."

>
> They don't mandate it to have the same size, but it is the expectation
> that implementations typically will.

That's one of the reasons why POSIX' ssize_t is a wrong analogy.   
Another reason is that POSIX interprets negative values of ssize_t as  
"an error indication", not as an offset from the end. A better POSIX  
analogy would be off_t (as used in lseek).

>
> In the discussion, ptrdiff_t was never brought up as an alternative

Yes, it was, by you <http://mail.python.org/pipermail/python-dev/2006- 
January/059562.html> :-).

I was not suggesting Py_ptrdiff_t.  My suggestion was "Py_index_t",  
but I am six month late :-(.


From guido at python.org  Tue Aug 15 17:21:59 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Aug 2006 08:21:59 -0700
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <44E1D4DA.2080002@egenix.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
	<44E1D4DA.2080002@egenix.com>
Message-ID: <ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>

On 8/15/06, M.-A. Lemburg <mal at egenix.com> wrote:
> The distutils version number should be changed back to a static
> string literal.
>
> It's currently setup to get its version number
> from the Python version running it which pretty much defeats
> the whole purpose of having a version number and makes using the
> SVN distutils package with other Python versions problematic.

I am sympathetic to this case. Is there any advantage to the *users*
of distutils of the dynamic version number? If it's only done for the
benefit of the release managers (with whose plight I also
sympathesize) I think it must be rolled back, at least as long as
distutils is officially listed as a package that needs to support
older versions of Python, which pretty much implies that it's okay to
extract it from the 2.5 release and distribute it separately for use
with older Python versions.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Aug 15 17:27:17 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Aug 2006 08:27:17 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
Message-ID: <ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>

FWIW, I propose that in py3k, the int type uses Py_ssize_t instead of long.

Already decided is that in py3k, the (x)range object will support
arbitrary integer sizes, so that e.g. range(10**10, 10**10+10) is
valid (it currently is, but for different reasons, it isn't with
xrange, which will replace range in py3k).

For 2.6 and onwards, I propose to let the issue rest *or* eventually
backport the py3k xrange implementation; "fixing" xrange to use
Py_ssize_t doesn't seem worth the churn.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mal at egenix.com  Tue Aug 15 17:45:37 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Aug 2006 17:45:37 +0200
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>	<44E1D4DA.2080002@egenix.com>
	<ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>
Message-ID: <44E1EC21.2080905@egenix.com>

Guido van Rossum wrote:
> On 8/15/06, M.-A. Lemburg <mal at egenix.com> wrote:
>> The distutils version number should be changed back to a static
>> string literal.
>>
>> It's currently setup to get its version number
>> from the Python version running it which pretty much defeats
>> the whole purpose of having a version number and makes using the
>> SVN distutils package with other Python versions problematic.
> 
> I am sympathetic to this case. Is there any advantage to the *users*
> of distutils of the dynamic version number? If it's only done for the
> benefit of the release managers (with whose plight I also
> sympathesize)

FWIW, I've already volunteered to do the version bumping to take
that burden off the release managers.

> I think it must be rolled back, at least as long as
> distutils is officially listed as a package that needs to support
> older versions of Python, which pretty much implies that it's okay to
> extract it from the 2.5 release and distribute it separately for use
> with older Python versions.

Well, that's another point we should discuss after 2.5 is
out the door: distutils was delisted from PEP 291 (without
public discussion).

I find it important to maintain distutils compatibility with
a few Python versions back. Even if I can't volunteer to
maintain distutils, like Martin suggested, due to lack of time,
I don't really see the requirement to use the latest and greatest
Python features in distutils, so preserving compatibility
with the commonly used Python versions wouldn't hurt.

It makes the life of Python extension writers a tad easier
and Python users benefit as a result of being able to download
pre-built binaries without being forced to upgrade (this is
esp. true for Python users on Windows).

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 15 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From martin at v.loewis.de  Tue Aug 15 17:51:38 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Aug 2006 17:51:38 +0200
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>	<44E1D4DA.2080002@egenix.com>
	<ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>
Message-ID: <44E1ED8A.8080308@v.loewis.de>

Guido van Rossum schrieb:
> I am sympathetic to this case. Is there any advantage to the *users*
> of distutils of the dynamic version number?

This series of commits was triggered by a user who wondered why
Python 2.4.3 ships with distutils 2.4.1, yet Python 2.5bsomething
ships with the older 2.4.0; he requested that the newer release
of distutils should be included in Python 2.5.

So the advantage to the user is that the distutils version numbers
become less confusing, and more "correct" (as long as the packaged
distutils is used).

> If it's only done for the
> benefit of the release managers (with whose plight I also
> sympathesize)

In principle, the release manager shouldn't be the one updating
the distutils release number. Instead, whoever modifies distutils
needs to decide whether this change is a bug fix (subminor change)
or feature change (minor number changes).

> I think it must be rolled back, at least as long as
> distutils is officially listed as a package that needs to support
> older versions of Python, which pretty much implies that it's okay to
> extract it from the 2.5 release and distribute it separately for use
> with older Python versions.

See, it isn't listed as such anymore, not since AMK removed it from
PEP 291.

That said, I'm not terribly opposed to it being changed back (*) - but
*I* won't change it back (having it changed already twice in two
days). I just wish I could then delegate all bug reports "distutils
doesn't work with older Python releases" to MAL. As far as I'm
concerned, this isn't a distutils feature anymore.

Regards,
Martin

(*) where "back" means 2.5.0, not 2.4.0.

From martin at v.loewis.de  Tue Aug 15 17:58:24 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Aug 2006 17:58:24 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <25F486F7-B449-4FE1-8C11-DCEFDDE4CBE2@local>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	<ca471dc20608141632r173e8d60o20af3787b7efbc9d@mail.gmail.com>	<04EA6FD8-CECD-4766-93B0-BE7137316669@local>	<ca471dc20608141956i59ce1453l2268e8e4959997c5@mail.gmail.com>	<80BC7ADD-01E0-41E5-B5B5-021765C6C865@local>	<ca471dc20608142114hb5a7220sa5e4a45b184cc77d@mail.gmail.com>
	<25F486F7-B449-4FE1-8C11-DCEFDDE4CBE2@local>
Message-ID: <44E1EF20.2010204@v.loewis.de>

Alexander Belopolsky schrieb:
> Another reason is that POSIX interprets negative values of ssize_t as  
> "an error indication", not as an offset from the end. A better POSIX  
> analogy would be off_t (as used in lseek).

That's subtle. By this reasoning, ptrdiff_t would be wrong, as well,
since it really means "negative index", not "count from the end".

>> In the discussion, ptrdiff_t was never brought up as an alternative
> 
> Yes, it was, by you <http://mail.python.org/pipermail/python-dev/2006- 
> January/059562.html> :-).

Ah, that's why it isn't ptrdiff_t: Tim Peters said he liked ssize_t

> I was not suggesting Py_ptrdiff_t.  My suggestion was "Py_index_t",  
> but I am six month late :-(.

Indeed, it is too late for such a change.

Regards,
Martin

From martin at v.loewis.de  Tue Aug 15 18:01:51 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Aug 2006 18:01:51 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
Message-ID: <44E1EFEF.20305@v.loewis.de>

Guido van Rossum schrieb:
> FWIW, I propose that in py3k, the int type uses Py_ssize_t instead of long.

This is really a py3k issue, but: shouldn't the int and long types
really get unified into a single type in Py3k?

I suppose the "i" parameter to PyArg_ParseTuple would continue to parse
int?

Regards,
Martin

From martin at v.loewis.de  Tue Aug 15 18:04:33 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 15 Aug 2006 18:04:33 +0200
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <44E1EC21.2080905@egenix.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>	<44E1D4DA.2080002@egenix.com>	<ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>
	<44E1EC21.2080905@egenix.com>
Message-ID: <44E1F091.60306@v.loewis.de>

M.-A. Lemburg schrieb:
> I find it important to maintain distutils compatibility with
> a few Python versions back. Even if I can't volunteer to
> maintain distutils, like Martin suggested, due to lack of time,
> I don't really see the requirement to use the latest and greatest
> Python features in distutils, so preserving compatibility
> with the commonly used Python versions wouldn't hurt.

Actually, it would. bdist_msi uses extension modules which
are only available in 2.5. Keeping support for the
compiler-recognizing code in msvccompiler also hurts.

It's either an official feature, with somebody maintaining it,
or people should expect to break it anytime.

Regards,
Martin

From mal at egenix.com  Tue Aug 15 18:09:28 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Aug 2006 18:09:28 +0200
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <44E1ED8A.8080308@v.loewis.de>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>	<44E1D4DA.2080002@egenix.com>	<ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>
	<44E1ED8A.8080308@v.loewis.de>
Message-ID: <44E1F1B8.2070603@egenix.com>

Martin v. L?wis wrote:
> Guido van Rossum schrieb:
>> I think it must be rolled back, at least as long as
>> distutils is officially listed as a package that needs to support
>> older versions of Python, which pretty much implies that it's okay to
>> extract it from the 2.5 release and distribute it separately for use
>> with older Python versions.
> 
> See, it isn't listed as such anymore, not since AMK removed it from
> PEP 291.
> 
> That said, I'm not terribly opposed to it being changed back (*) - but
> *I* won't change it back (having it changed already twice in two
> days). I just wish I could then delegate all bug reports "distutils
> doesn't work with older Python releases" to MAL. As far as I'm
> concerned, this isn't a distutils feature anymore.

How many of those are there ?

We're running SVN distutils regularly for Python 2.4 and use
the one that shipped with Python 2.3 (the last to support 1.5.2)
for all versions between 1.5.2 and 2.3.

We haven't hit any bugs related to distutils not being
compatible to those Python versions anymore. The few I found
over the years, I corrected directly in the repository.

I just checked: SVN distutils (at least the parts that we use)
still works with Python 2.2. It no longer works with Python 2.1
due to the use of "True" and probably a few other things.

> Regards,
> Martin
> 
> (*) where "back" means 2.5.0, not 2.4.0.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 15 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From guido at python.org  Tue Aug 15 18:10:12 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Aug 2006 09:10:12 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E1EFEF.20305@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
Message-ID: <ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>

On 8/15/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Guido van Rossum schrieb:
> > FWIW, I propose that in py3k, the int type uses Py_ssize_t instead of long.
>
> This is really a py3k issue, but: shouldn't the int and long types
> really get unified into a single type in Py3k?

>From the Python *user*'s perspective, yes, as much as possible. But
I'm still playing with the thought of having two implementation types,
since otherwise we'd have to devote 4 bytes (8 on a 64-bit platform)
to the single *bit* telling the difference between the two internal
representations. I haven't decided whether to make 'int' an abstract
base type and have 'short' and 'long' subtypes (perhaps using other
names), or whether to make 'int' the base type and 'short' the
subtype, or whether to cheat and hack type() so that type() of any
integer always returns int.  But in any case, this would mean that at
the C level the distinction continues to exist.

Maybe we can discuss this at the sprint next week?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Aug 15 18:14:49 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Aug 2006 09:14:49 -0700
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <44E1F1B8.2070603@egenix.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
	<44E1D4DA.2080002@egenix.com>
	<ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>
	<44E1ED8A.8080308@v.loewis.de> <44E1F1B8.2070603@egenix.com>
Message-ID: <ca471dc20608150914n7613fc60q5be564b6aaea9903@mail.gmail.com>

Marc-Andre,

I think the release managers might let you change this back if you
volunteered, not to maintain all of distutils (I wouldn't wish that on
my worst enemies :-) but at least to keep the version number up to
date and to do the occasional work to keep it backwards compatible in
the way you want it.

I think it's fine of the new distutils contains new features that
won't work with older Pythons, as long as the old features still work
(and have perhaps updated functionality in some other way). How
exactly to reconcile this with the black/white notion of PPE 291 I
don't know; it seems a compromise is in order.

--Guido

On 8/15/06, M.-A. Lemburg <mal at egenix.com> wrote:
> Martin v. L?wis wrote:
> > Guido van Rossum schrieb:
> >> I think it must be rolled back, at least as long as
> >> distutils is officially listed as a package that needs to support
> >> older versions of Python, which pretty much implies that it's okay to
> >> extract it from the 2.5 release and distribute it separately for use
> >> with older Python versions.
> >
> > See, it isn't listed as such anymore, not since AMK removed it from
> > PEP 291.
> >
> > That said, I'm not terribly opposed to it being changed back (*) - but
> > *I* won't change it back (having it changed already twice in two
> > days). I just wish I could then delegate all bug reports "distutils
> > doesn't work with older Python releases" to MAL. As far as I'm
> > concerned, this isn't a distutils feature anymore.
>
> How many of those are there ?
>
> We're running SVN distutils regularly for Python 2.4 and use
> the one that shipped with Python 2.3 (the last to support 1.5.2)
> for all versions between 1.5.2 and 2.3.
>
> We haven't hit any bugs related to distutils not being
> compatible to those Python versions anymore. The few I found
> over the years, I corrected directly in the repository.
>
> I just checked: SVN distutils (at least the parts that we use)
> still works with Python 2.2. It no longer works with Python 2.1
> due to the use of "True" and probably a few other things.
>
> > Regards,
> > Martin
> >
> > (*) where "back" means 2.5.0, not 2.4.0.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Aug 15 2006)
> >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
> >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mal at egenix.com  Tue Aug 15 18:17:29 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Aug 2006 18:17:29 +0200
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <44E1F091.60306@v.loewis.de>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>	<44E1D4DA.2080002@egenix.com>	<ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>	<44E1EC21.2080905@egenix.com>
	<44E1F091.60306@v.loewis.de>
Message-ID: <44E1F399.5080908@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg schrieb:
>> I find it important to maintain distutils compatibility with
>> a few Python versions back. Even if I can't volunteer to
>> maintain distutils, like Martin suggested, due to lack of time,
>> I don't really see the requirement to use the latest and greatest
>> Python features in distutils, so preserving compatibility
>> with the commonly used Python versions wouldn't hurt.
> 
> Actually, it would. bdist_msi uses extension modules which
> are only available in 2.5. 

bdist_msi is only a small part of distutils. While it would
be great to have that distribution format for 2.3 and 2.4 as well,
I don't think it's vital.

It's simply a feature that is only available when using
Python 2.5.

> Keeping support for the
> compiler-recognizing code in msvccompiler also hurts.

I guess that's something to tell Microsoft ;-)

However, I don't see that as argument for dropping the
distutils support for Python versions prior to 2.5.

> It's either an official feature, with somebody maintaining it,
> or people should expect to break it anytime.

I'll let you know when things break - is that good enough ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 15 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Tue Aug 15 18:24:01 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 15 Aug 2006 18:24:01 +0200
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <ca471dc20608150914n7613fc60q5be564b6aaea9903@mail.gmail.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>	<44E1D4DA.2080002@egenix.com>	<ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>	<44E1ED8A.8080308@v.loewis.de>
	<44E1F1B8.2070603@egenix.com>
	<ca471dc20608150914n7613fc60q5be564b6aaea9903@mail.gmail.com>
Message-ID: <44E1F521.2010908@egenix.com>

Guido van Rossum wrote:
> Marc-Andre,
> 
> I think the release managers might let you change this back if you
> volunteered, not to maintain all of distutils (I wouldn't wish that on
> my worst enemies :-) but at least to keep the version number up to
> date and to do the occasional work to keep it backwards compatible in
> the way you want it.

I guess it's Anthony's turn then... the patch is there and
tested, just waiting in its shell window to get checked
in ;-)

I'll add a distutils checkin message filter to keep me
informed of any changes to distutils.

> I think it's fine of the new distutils contains new features that
> won't work with older Pythons, as long as the old features still work
> (and have perhaps updated functionality in some other way). How
> exactly to reconcile this with the black/white notion of PPE 291 I
> don't know; it seems a compromise is in order.

+1

> --Guido
> 
> On 8/15/06, M.-A. Lemburg <mal at egenix.com> wrote:
>> Martin v. L?wis wrote:
>> > Guido van Rossum schrieb:
>> >> I think it must be rolled back, at least as long as
>> >> distutils is officially listed as a package that needs to support
>> >> older versions of Python, which pretty much implies that it's okay to
>> >> extract it from the 2.5 release and distribute it separately for use
>> >> with older Python versions.
>> >
>> > See, it isn't listed as such anymore, not since AMK removed it from
>> > PEP 291.
>> >
>> > That said, I'm not terribly opposed to it being changed back (*) - but
>> > *I* won't change it back (having it changed already twice in two
>> > days). I just wish I could then delegate all bug reports "distutils
>> > doesn't work with older Python releases" to MAL. As far as I'm
>> > concerned, this isn't a distutils feature anymore.
>>
>> How many of those are there ?
>>
>> We're running SVN distutils regularly for Python 2.4 and use
>> the one that shipped with Python 2.3 (the last to support 1.5.2)
>> for all versions between 1.5.2 and 2.3.
>>
>> We haven't hit any bugs related to distutils not being
>> compatible to those Python versions anymore. The few I found
>> over the years, I corrected directly in the repository.
>>
>> I just checked: SVN distutils (at least the parts that we use)
>> still works with Python 2.2. It no longer works with Python 2.1
>> due to the use of "True" and probably a few other things.
>>
>> > Regards,
>> > Martin
>> >
>> > (*) where "back" means 2.5.0, not 2.4.0.
>>
>> -- 
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Source  (#1, Aug 15 2006)
>> >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>> >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>> ________________________________________________________________________
>>
>> ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
>>
> 
> 

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 15 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From gustavo at niemeyer.net  Tue Aug 15 18:57:09 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Tue, 15 Aug 2006 13:57:09 -0300
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
Message-ID: <20060815165709.GA29060@niemeyer.net>

> If you have issues, respond ASAP!  The release candidate is planned to
> be cut this Thursday/Friday.  There are only a few more days before
> code freeze.  A branch will be made when the release candidate is cut.

I'd like to see problem #1531862 fixed.  The bug is clear and the
fix should be trivial.  I can commit a fix tonight, if the subprocess
module author/maintainer is unavailable to check it out.

-- 
Gustavo Niemeyer
http://niemeyer.net

From faassen at infrae.com  Tue Aug 15 19:39:51 2006
From: faassen at infrae.com (Martijn Faassen)
Date: Tue, 15 Aug 2006 19:39:51 +0200
Subject: [Python-Dev] Elementtree and Namespaces in 2.5
In-Reply-To: <ebpo4f$jd8$1@sea.gmane.org>
References: <beb7688e0608110815k44369b92tdd4dbcecd00b1a8@mail.gmail.com>
	<ebpo4f$jd8$1@sea.gmane.org>
Message-ID: <44E206E7.4040403@infrae.com>

Fredrik Lundh wrote:
> "Chris S" wrote:
> 
>> and while most users and the w3 spec
>> (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting)
>> agree this feature is actually a bug
> 
> ET's not a canonicalization library, and doesn't claim to be one, so that reference isn't
> very relevant.  And "most users" know that ET uses a simplified infoset; it's not exactly
> a secret.

And if you need an XML library with ET's API that does retain namespace 
prefixes and has support for canonicalization and a boatload of other 
features, you could try lxml.

http://codespeak.net/lxml

Regards,

Martijn

From jimjjewett at gmail.com  Tue Aug 15 22:19:43 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 15 Aug 2006 16:19:43 -0400
Subject: [Python-Dev] IDLE patches - bugfix or not?
Message-ID: <fb6fbf560608151319w21753f3ek575e7709b8ba3521@mail.gmail.com>

I just uploaded a series of IDLE patches, but I'm not quite sure how
to classify them on the feature/bugfix scale now that the last beta is
out.

>From most to least buggish:

python.org/sf/1540892 -- honor the new quit() behavior.  On the other
hand, it was documented that this didn't work in IDLE, and it is
*possible* that someone was counting on this.

python.org/sf/1540851 -- with is now a blockopener, and should be
counted as such -- I *think* this one would be safe, but I know that
changing a parser can be surprising, and I suppose it *could* wait
until with no longer requires a future statement.

python.org/sf/1540874 -- broken shortcut keys.  On windows, only one
entry per menu can be reached with the same shortcut letter, so
advertising others is just an attractive nuisance.  I'm not sure that
other systems wouldn't be able to use the hidden shortcuts.

python.org/sf/1540869 -- GUI fix.  The current code puts in a
separator using a magic number (and has XXX comments about it.)  This
changes the magic number so that the separator is more visible, but
I'm not sure the old behavior rose to a bug, or that it wasn't
platform dependent.

python.org/sf/1540849 -- except too broad.  I wouldn't suggest
applying this late in the release cycle, except that it seems sort of
like the memory errors that are still being patched.

-jJ

From martin at v.loewis.de  Wed Aug 16 00:20:23 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 00:20:23 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>	
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
Message-ID: <44E248A7.4080107@v.loewis.de>

Guido van Rossum schrieb:
> From the Python *user*'s perspective, yes, as much as possible. But
> I'm still playing with the thought of having two implementation types,
> since otherwise we'd have to devote 4 bytes (8 on a 64-bit platform)
> to the single *bit* telling the difference between the two internal
> representations.

We had this discussion before; if you use ob_size==0 to indicate
that it's an int, this space isn't needed in a long int. On a 32-bit
platform, the size of an int would go up from 12 to 16; if we stop
using a special-cased allocator (which we should (*)), there isn't
any space increase on such a platform. On a 64-bit platform, the
size of an int would go up from 24 bytes to 32 bytes.

Regards,
Martin

(*) people have complained that the memory allocated for a large
number of ints isn't ever reused. They consumed it by passing
range() some large argument.

From guido at python.org  Wed Aug 16 00:29:13 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Aug 2006 15:29:13 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E248A7.4080107@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
Message-ID: <ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>

On 8/15/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Guido van Rossum schrieb:
> > From the Python *user*'s perspective, yes, as much as possible. But
> > I'm still playing with the thought of having two implementation types,
> > since otherwise we'd have to devote 4 bytes (8 on a 64-bit platform)
> > to the single *bit* telling the difference between the two internal
> > representations.
>
> We had this discussion before; if you use ob_size==0 to indicate
> that it's an int, this space isn't needed in a long int. On a 32-bit
> platform, the size of an int would go up from 12 to 16; if we stop
> using a special-cased allocator (which we should (*)), there isn't
> any space increase on such a platform. On a 64-bit platform, the
> size of an int would go up from 24 bytes to 32 bytes.
>
> Regards,
> Martin
>
> (*) people have complained that the memory allocated for a large
> number of ints isn't ever reused. They consumed it by passing
> range() some large argument.

Since range() won't return a real list any more, this no longer is the
case. I worry that dropping the special allocator will be too slow.
And the space increase on 64-bit platforms is too much IMO. But
clearly these issues can only be addressed by careful benchmarking.
Perhaps we should do some of that to settle the issue.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Wed Aug 16 00:30:18 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 00:30:18 +0200
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <44E1F399.5080908@egenix.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>	<44E1D4DA.2080002@egenix.com>	<ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>	<44E1EC21.2080905@egenix.com>
	<44E1F091.60306@v.loewis.de> <44E1F399.5080908@egenix.com>
Message-ID: <44E24AFA.1090102@v.loewis.de>

M.-A. Lemburg schrieb:
>> It's either an official feature, with somebody maintaining it,
>> or people should expect to break it anytime.
> 
> I'll let you know when things break - is that good enough ?

That can't be an official policy; you seem to define "breaks"
as "breaks in my (your) personal usage of distutils". While
this is fine as an end-user point of view, it isn't useful
as a maintenance guideline.

In case you don't see what I mean: I said that something
is already broken (bdist_msi), where "broken" means
"doesn't work", and you said "it's only a small part".
So if it's fine that only small parts break, I wonder
where this gets us over time.

For another example, you found that 2.1 compatibility is
now broken (due to usage of True and False). When distutils
was still listed in PEP 291, 2.1 compatibility was mentioned
as a goal. Now, it doesn't seem to bother you very much
that 2.1 compatibility is gone.

Regards,
Martin


From foom at fuhm.net  Wed Aug 16 00:34:43 2006
From: foom at fuhm.net (James Y Knight)
Date: Tue, 15 Aug 2006 18:34:43 -0400
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E248A7.4080107@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>	
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
Message-ID: <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>


On Aug 15, 2006, at 6:20 PM, Martin v. L?wis wrote:
> Guido van Rossum schrieb:
>> From the Python *user*'s perspective, yes, as much as possible. But
>> I'm still playing with the thought of having two implementation  
>> types,
>> since otherwise we'd have to devote 4 bytes (8 on a 64-bit platform)
>> to the single *bit* telling the difference between the two internal
>> representations.
>
> We had this discussion before; if you use ob_size==0 to indicate
> that it's an int, this space isn't needed in a long int. On a 32-bit
> platform, the size of an int would go up from 12 to 16; if we stop
> using a special-cased allocator (which we should (*)), there isn't
> any space increase on such a platform. On a 64-bit platform, the
> size of an int would go up from 24 bytes to 32 bytes.

But it's the short int that you probably really want to make size  
efficient. Which is of course also doable via something like:

typedef struct {
     PyObject_HEAD
     long ob_islong : 1;
     long ob_ival_or_size : LONG_BITS - 1;
     long ob_digit[0];
} PyIntObject;

There's no particular reason that a short int must be able to store  
the entire range of C "long", so, as many bits can be stolen from it  
as desired.

James

From mal at egenix.com  Wed Aug 16 00:55:45 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 16 Aug 2006 00:55:45 +0200
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <44E24AFA.1090102@v.loewis.de>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>	<44E1D4DA.2080002@egenix.com>	<ca471dc20608150821h13a72619yf83b264f6e04a103@mail.gmail.com>	<44E1EC21.2080905@egenix.com>	<44E1F091.60306@v.loewis.de>
	<44E1F399.5080908@egenix.com> <44E24AFA.1090102@v.loewis.de>
Message-ID: <44E250F1.40309@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg schrieb:
>>> It's either an official feature, with somebody maintaining it,
>>> or people should expect to break it anytime.
>> I'll let you know when things break - is that good enough ?
> 
> That can't be an official policy; you seem to define "breaks"
> as "breaks in my (your) personal usage of distutils". While
> this is fine as an end-user point of view, it isn't useful
> as a maintenance guideline.

It's all I can offer.

We do use quite a few distutils
features and also extend it in various ways, so I suppose
that our build system covers quite a bit of what distutils
has to offer. Think of it as a test-bot that runs a real-life
test on the distutils code.

> In case you don't see what I mean: I said that something
> is already broken (bdist_msi), where "broken" means
> "doesn't work", and you said "it's only a small part".
> So if it's fine that only small parts break, I wonder
> where this gets us over time.

distutils is composed of many components. bdist_msi is
one of them. Because it's new it can hardly "break"
distutils in any way because it's a feature that didn't
exist in the distutils version shipped with
Python prior to 2.5.

As a result, that distribution format cannot be used with
older Python versions, but the rest of distutils continues
to work as usual.

I wouldn't call that breakage of distutils. It's just a
feature that has external requirements, much like you
can't run bdist_rpm without having the RPM tool installed.

In general, if you have to rely on new features in Python
such as extension modules which are not available in older
Python versions, then that's fine - but again, if not
really necessary, I'd rather see an approach being used
that also works with older Python versions.

What I do consider breakage, is if you use Python 2.5
features just to beef up some part of distutils core
code.

> For another example, you found that 2.1 compatibility is
> now broken (due to usage of True and False). When distutils
> was still listed in PEP 291, 2.1 compatibility was mentioned
> as a goal. Now, it doesn't seem to bother you very much
> that 2.1 compatibility is gone.

Actually, it does bother me, but I understand that not
many people use Python 2.1 anymore, so don't insist keeping
that version requirement.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 16 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From guido at python.org  Wed Aug 16 01:06:52 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Aug 2006 16:06:52 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
Message-ID: <ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>

On 8/15/06, James Y Knight <foom at fuhm.net> wrote:
>
> On Aug 15, 2006, at 6:20 PM, Martin v. L?wis wrote:
> > Guido van Rossum schrieb:
> >> From the Python *user*'s perspective, yes, as much as possible. But
> >> I'm still playing with the thought of having two implementation
> >> types,
> >> since otherwise we'd have to devote 4 bytes (8 on a 64-bit platform)
> >> to the single *bit* telling the difference between the two internal
> >> representations.
> >
> > We had this discussion before; if you use ob_size==0 to indicate
> > that it's an int, this space isn't needed in a long int. On a 32-bit
> > platform, the size of an int would go up from 12 to 16; if we stop
> > using a special-cased allocator (which we should (*)), there isn't
> > any space increase on such a platform. On a 64-bit platform, the
> > size of an int would go up from 24 bytes to 32 bytes.
>
> But it's the short int that you probably really want to make size
> efficient. Which is of course also doable via something like:
>
> typedef struct {
>      PyObject_HEAD
>      long ob_islong : 1;
>      long ob_ival_or_size : LONG_BITS - 1;
>      long ob_digit[0];
> } PyIntObject;
>
> There's no particular reason that a short int must be able to store
> the entire range of C "long", so, as many bits can be stolen from it
> as desired.

There isn't? Actually a lot of APIs currently assumen that.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Wed Aug 16 01:20:39 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 01:20:39 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>		<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>		<44E1EFEF.20305@v.loewis.de>	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
Message-ID: <44E256C7.5060808@v.loewis.de>

James Y Knight schrieb:
> But it's the short int that you probably really want to make size  
> efficient.

Only if you have many of them. And if you do, you have the problem
of the special-cased allocator: when the many ints go away, Python
will hold onto their memory forever.

Regards,
Martin

From guido at python.org  Wed Aug 16 01:28:09 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Aug 2006 16:28:09 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E256C7.5060808@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
	<44E256C7.5060808@v.loewis.de>
Message-ID: <ca471dc20608151628w244dbf50xcae3bdb31d6c7f2e@mail.gmail.com>

On 8/15/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> James Y Knight schrieb:
> > But it's the short int that you probably really want to make size
> > efficient.
>
> Only if you have many of them. And if you do, you have the problem
> of the special-cased allocator: when the many ints go away, Python
> will hold onto their memory forever.

But that's a bit of a corner case. There are plenty of cases where
ints are allocated and deallocated at a fast rate without allocating
tons of them at once, or where there's no need to reuse the same
memory for something else later. I wonder if we could have a smarter
int allocator that allocates some ints in a special array but switches
to the standard allocator if too many are being allocated?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From foom at fuhm.net  Wed Aug 16 01:33:12 2006
From: foom at fuhm.net (James Y Knight)
Date: Tue, 15 Aug 2006 19:33:12 -0400
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
Message-ID: <0EC0B3AA-422B-4D88-BA28-F3BD06E40F6F@fuhm.net>

On Aug 15, 2006, at 7:06 PM, Guido van Rossum wrote:
>> There's no particular reason that a short int must be able to store
>> the entire range of C "long", so, as many bits can be stolen from it
>> as desired.
>
> There isn't? Actually a lot of APIs currently assumen that.

I thought we were talking about Py3k. *IF* the idea is to integrate  
both short/long ints into a single type, with only an internal  
distinction (which is what is being discussed), all those APIs are  
broken already. The particular internal division of the new int  
object between short and long doesn't matter. Which is all I was saying.

If combining all integers into a single type isn't actually desired,  
then neither my message nor Martin's is relevant.

James

From nas at arctrix.com  Wed Aug 16 01:33:47 2006
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue, 15 Aug 2006 23:33:47 +0000 (UTC)
Subject: [Python-Dev] no remaining issues blocking 2.5 release
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
Message-ID: <ebtlkr$joi$1@sea.gmane.org>

Neal Norwitz <nnorwitz at gmail.com> wrote:
> I don't know of any.  I haven't heard of any issues with the fixes
> that have been checked in.

It would be nice if someone could bytecompile Lib using
Tools/compiler/compile.py and then run the test suite.  I'd do it
myself but can't spare the time at the moment (I started but ran
into what seems to be a gcc bug along the way).

  Neil


From guido at python.org  Wed Aug 16 01:38:31 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 15 Aug 2006 16:38:31 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <0EC0B3AA-422B-4D88-BA28-F3BD06E40F6F@fuhm.net>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
	<0EC0B3AA-422B-4D88-BA28-F3BD06E40F6F@fuhm.net>
Message-ID: <ca471dc20608151638q6b5044e2y4266dec04b48fa42@mail.gmail.com>

On 8/15/06, James Y Knight <foom at fuhm.net> wrote:
> On Aug 15, 2006, at 7:06 PM, Guido van Rossum wrote:
> >> There's no particular reason that a short int must be able to store
> >> the entire range of C "long", so, as many bits can be stolen from it
> >> as desired.
> >
> > There isn't? Actually a lot of APIs currently assumen that.
>
> I thought we were talking about Py3k. *IF* the idea is to integrate
> both short/long ints into a single type, with only an internal
> distinction (which is what is being discussed), all those APIs are
> broken already. The particular internal division of the new int
> object between short and long doesn't matter. Which is all I was saying.
>
> If combining all integers into a single type isn't actually desired,
> then neither my message nor Martin's is relevant.

As I said before, at the C level I expect the distinction between int
and long to be alive and well. Changing it so that ints don't have the
full range of the corresponding C type would be painful IMO.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Wed Aug 16 03:40:38 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Aug 2006 13:40:38 +1200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E248A7.4080107@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
Message-ID: <44E27796.20101@canterbury.ac.nz>

Martin v. L?wis wrote:
> We had this discussion before; if you use ob_size==0 to indicate
> that it's an int, this space isn't needed in a long int.

What about int subclasses?

--
Greg

From greg.ewing at canterbury.ac.nz  Wed Aug 16 03:42:56 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Aug 2006 13:42:56 +1200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>
Message-ID: <44E27820.40205@canterbury.ac.nz>

Guido van Rossum wrote:
> I worry that dropping the special allocator will be too slow.

Surely there's some compromise that would allow
recently-used ints to be kept around, but reclaimed
if memory becomes low?

--
Greg

From anthony at interlink.com.au  Wed Aug 16 03:45:41 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 16 Aug 2006 11:45:41 +1000
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <44E1F521.2010908@egenix.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
	<ca471dc20608150914n7613fc60q5be564b6aaea9903@mail.gmail.com>
	<44E1F521.2010908@egenix.com>
Message-ID: <200608161145.44938.anthony@interlink.com.au>

I really don't care any more about this. My initial concern (and why I 
requested the change) was that there are no more official separate distutils 
releases. I don't see how keeping a bunch of version numbers in the stdlib 
that just track the main version number is a sane use of developer time - 
particularly when it's only being used for someone's private releases. If 
you're cutting releases out of the stdlib, it seems to me that the 
maintenance load should be on you (where 'you' is 'anyone doing something 
like this', not MAL specifically <wink>)

I'd also like to see idle's separate version number go away and have it start 
using the Python version number - maybe as of 2.6? This would then also mean 
we could pull idle's separate NEWS file into the master NEWS file. Right now, 
the "release notes" on the webpage miss all IDLE release notes, since it's 
just the main NEWS file.

Since it's provoked so many complaints, change it back if you want. I won't be 
bothering to check that it's correct from now on, nor will I be updating it. 
Any user complaints will be ignored, and an incorrect version number will not 
be considered a reason for a bugfix release.

Anthony

From greg.ewing at canterbury.ac.nz  Wed Aug 16 03:47:37 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 16 Aug 2006 13:47:37 +1200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
Message-ID: <44E27939.9060505@canterbury.ac.nz>

Guido van Rossum wrote:

> On 8/15/06, James Y Knight <foom at fuhm.net> wrote:
 >
>>There's no particular reason that a short int must be able to store
>>the entire range of C "long", so, as many bits can be stolen from it
>>as desired.
> 
> There isn't? Actually a lot of APIs currently assumen that.

Also it means you'd pay a penalty every time you
access it, whereas presumably short ints are the
case we want to optimise for speed as well.

--
Greg


From anthony at interlink.com.au  Wed Aug 16 03:49:20 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 16 Aug 2006 11:49:20 +1000
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <ebsgc0$jkk$1@sea.gmane.org>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
	<ebsgc0$jkk$1@sea.gmane.org>
Message-ID: <200608161149.22013.anthony@interlink.com.au>

On Tuesday 15 August 2006 22:57, Thomas Heller wrote:
> What is the policy for documentation changes?  Am I allowed to check in
> changes/additions to the ctypes docs without release manager permission
> after the release candidate is out?  I'l always make sure that the html
> docs can be built.

So long as you don't break anything, doc changes are fine. Remember that post 
RC1 you'll need to checkin to both release25-maint and trunk.

Anthony

-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From anthony at interlink.com.au  Wed Aug 16 04:10:43 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 16 Aug 2006 12:10:43 +1000
Subject: [Python-Dev] IDLE patches - bugfix or not?
In-Reply-To: <fb6fbf560608151319w21753f3ek575e7709b8ba3521@mail.gmail.com>
References: <fb6fbf560608151319w21753f3ek575e7709b8ba3521@mail.gmail.com>
Message-ID: <200608161210.46100.anthony@interlink.com.au>

On Wednesday 16 August 2006 06:19, Jim Jewett wrote:
> I just uploaded a series of IDLE patches, but I'm not quite sure how
> to classify them on the feature/bugfix scale now that the last beta is
> out.
>
> >From most to least buggish:
>
> python.org/sf/1540892 -- honor the new quit() behavior.  On the other
> hand, it was documented that this didn't work in IDLE, and it is
> *possible* that someone was counting on this.

This seems like a sensible thing to add.

> python.org/sf/1540851 -- with is now a blockopener, and should be
> counted as such -- I *think* this one would be safe, but I know that
> changing a parser can be surprising, and I suppose it *could* wait
> until with no longer requires a future statement.

If this can be done safely, it should be done. Preferably before RC1, so that 
we have time to fix any issues it shows up.

> python.org/sf/1540874 -- broken shortcut keys.  On windows, only one
> entry per menu can be reached with the same shortcut letter, so
> advertising others is just an attractive nuisance.  I'm not sure that
> other systems wouldn't be able to use the hidden shortcuts.

Tough call. I guess it depends on the other systems - I will try this on Linux 
at least, and see if it works there. If it's broken everywhere, then changing 
it would seem the least offensive choice.

> python.org/sf/1540869 -- GUI fix.  The current code puts in a
> separator using a magic number (and has XXX comments about it.)  This
> changes the magic number so that the separator is more visible, but
> I'm not sure the old behavior rose to a bug, or that it wasn't
> platform dependent.

Let's leave this one for 2.6.

> python.org/sf/1540849 -- except too broad.  I wouldn't suggest
> applying this late in the release cycle, except that it seems sort of
> like the memory errors that are still being patched.

I'd be concerned that this might cause other obscure behaviour changes, and so 
I'd prefer to leave this to 2.6.


Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From anthony at interlink.com.au  Wed Aug 16 04:37:04 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 16 Aug 2006 12:37:04 +1000
Subject: [Python-Dev] TRUNK FREEZE for 2.5c1, 00:00 UTC, Thursday 17th August
Message-ID: <200608161237.06556.anthony@interlink.com.au>

Ok, here we go... I'm declaring the TRUNK FROZEN as of 00:00 UTC on Thursday 
the 17th of August. This is in about 22 hours time. At this time, I'll be 
cutting the release25-maint branch and 2.5c1 (and all future 2.5 releases) 
will be from that branch. I'll send another email once the release is done - 
at that point, the trunk is then available for 2.6 checkins. 

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From kbk at shore.net  Wed Aug 16 04:44:40 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue, 15 Aug 2006 22:44:40 -0400
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <200608161145.44938.anthony@interlink.com.au> (Anthony Baxter's
	message of "Wed, 16 Aug 2006 11:45:41 +1000")
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
	<ca471dc20608150914n7613fc60q5be564b6aaea9903@mail.gmail.com>
	<44E1F521.2010908@egenix.com>
	<200608161145.44938.anthony@interlink.com.au>
Message-ID: <87mza5csbr.fsf@hydra.bayview.thirdcreek.com>

Anthony Baxter <anthony at interlink.com.au> writes:

> I'd also like to see idle's separate version number go away and have
> it start using the Python version number - maybe as of 2.6?

+1

When we merged IDLEfork the consensus was to keep the versioning
separate.  But it seems confusing and is extra work for the release
managers.

> This would then also mean we could pull idle's separate NEWS file into
> the master NEWS file. Right now, the "release notes" on the webpage
> miss all IDLE release notes, since it's just the main NEWS file.

-1

Right now IDLE has a button in the "About IDLE" dialog which accesses
the IDLE NEWS.txt file, and another which accesses Python's NEWS.
IDLE users may not want to grovel through all the Python changes to
find out what changed in the IDE.

It would be nice if the key IDLE changes could make it to the "What's New
in Python X.X".  If Andrew is interested, I could draft something for him.

-- 
KBK

From kbk at shore.net  Wed Aug 16 04:53:55 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue, 15 Aug 2006 22:53:55 -0400
Subject: [Python-Dev] IDLE patches - bugfix or not?
In-Reply-To: <200608161210.46100.anthony@interlink.com.au> (Anthony Baxter's
	message of "Wed, 16 Aug 2006 12:10:43 +1000")
References: <fb6fbf560608151319w21753f3ek575e7709b8ba3521@mail.gmail.com>
	<200608161210.46100.anthony@interlink.com.au>
Message-ID: <87irktcrwc.fsf@hydra.bayview.thirdcreek.com>

Anthony Baxter <anthony at interlink.com.au> writes:

> On Wednesday 16 August 2006 06:19, Jim Jewett wrote:
>> I just uploaded a series of IDLE patches, but I'm not quite sure how
>> to classify them on the feature/bugfix scale now that the last beta is
>> out.
>>
>> >From most to least buggish:
>>
>> python.org/sf/1540892 -- honor the new quit() behavior.  On the other
>> hand, it was documented that this didn't work in IDLE, and it is
>> *possible* that someone was counting on this.
>
> This seems like a sensible thing to add.
>
>> python.org/sf/1540851 -- with is now a blockopener, and should be
>> counted as such -- I *think* this one would be safe, but I know that
>> changing a parser can be surprising, and I suppose it *could* wait
>> until with no longer requires a future statement.
>
> If this can be done safely, it should be done. Preferably before RC1, so that 
> we have time to fix any issues it shows up.

I will look at these two.  RC1 seems reasonable.

>> python.org/sf/1540874 -- broken shortcut keys.  On windows, only one
>> entry per menu can be reached with the same shortcut letter, so
>> advertising others is just an attractive nuisance.  I'm not sure that
>> other systems wouldn't be able to use the hidden shortcuts.
>
> Tough call. I guess it depends on the other systems - I will try this on Linux 
> at least, and see if it works there. If it's broken everywhere, then changing 
> it would seem the least offensive choice.

I would have been inclined to make the other choice, i.e. 'p' as the
hotkey for print, rather than the rarely used "save copy as".

>> python.org/sf/1540869 -- GUI fix.  The current code puts in a
>> separator using a magic number (and has XXX comments about it.)  This
>> changes the magic number so that the separator is more visible, but
>> I'm not sure the old behavior rose to a bug, or that it wasn't
>> platform dependent.
>
> Let's leave this one for 2.6.
>
>> python.org/sf/1540849 -- except too broad.  I wouldn't suggest
>> applying this late in the release cycle, except that it seems sort of
>> like the memory errors that are still being patched.
>
> I'd be concerned that this might cause other obscure behaviour changes, and so 
> I'd prefer to leave this to 2.6.

Yes, 2.6 for these two.

-- 
KBK

From brett at python.org  Tue Aug  8 02:27:45 2006
From: brett at python.org (Brett Cannon)
Date: Mon, 7 Aug 2006 17:27:45 -0700
Subject: [Python-Dev] Four issue trackers submitted for Infrastructue
	Committee's tracker search
Message-ID: <bbaeab100608071727t64c1064dwdb2c69861c50d303@mail.gmail.com>

Back in June, the Python Software Foundation's Infrastructure Committee
asked for help in the search for a new issue tracker to replace SourceForge
(see http://wiki.python.org/moin/CallForTrackers for details). We asked
people who wished to help with the search to install their favourite issue
tracker and import a data dump of our issues from SourceForge. We placed a
deadline of August 7th to have the installation up and going. We realized
that this was a large request to make of the community, but we needed to
make sure that we got to evaluate any candidate trackers with the amount of
issues that Python has so as to get a real-world feel.

Not surprisingly, the community stepped up to the challenge with four
different test tracker installations by the deadline! We have one for JIRA,
Roundup, Trac, and Launchpad. The current schedule calls for the
Infrastructure Committee to spend the next month evaluating the four
trackers and making a decision on which one to go with. This might slip,
though, since this overlaps with traditional American and European vacation
time. But the slippage will not go beyond October 1.

On behalf of the Infrastructure committee I thank the people who took the
time and effort to set up a test tracker to help improve the development of
Python.

- Brett Cannon
  Chairman, PSF Infrastructure Committee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060807/dd080e69/attachment.htm 

From mal at lemburg.com  Tue Aug  8 09:22:01 2006
From: mal at lemburg.com (mal)
Date: Tue, 08 Aug 2006 09:22:01 +0200
Subject: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and
 unicode	asdictionarykeys
In-Reply-To: <44D73964.3050402@v.loewis.de>
References: <20060804052652.rbdrz5qw5tcsswkk@login.werra.lunarpages.com>	<2m3bccwopj.fsf@starship.python.net>	<eavofa$n2f$1@sea.gmane.org>	<44D36B36.8030202@egenix.com>
	<44D73964.3050402@v.loewis.de>
Message-ID: <44D83B99.2070603@lemburg.com>

Martin v. L?wis wrote:
> M.-A. Lemburg schrieb:
>> Python just doesn't know the encoding of the 8-bit string, so can't
>> make any assumptions on it. As result, it raises an exception to inform
>> the programmer.
> 
> Oh, Python does make an assumption what the encoding is: it assumes
> it is the system encoding (i.e. "ascii"). Then invoking the ascii
> codec raises an exception, because the string clearly isn't ascii.

Right, and as consequence, Python raises an exception to let the
programmer correct the problem.

The subsequent solution to the problem may result in the
string being decoded into Unicode and the two resulting Unicode
objects being unequal, or it may also result in them being equal.
Python doesn't have this knowledge, so always returning false
is clearly wrong.

Hiding programmer errors is not making life easier in the
long run, so I'm -1 on having the equality comparison return
False.

Instead we should generate a warning in Python 2.5 and introduce
the exception in Python 2.6.

>> Note that you do have to interpret the string as characters
>> > if you compare it to Unicode and there's nothing wrong with
>> > that.
> 
> Consider this:
> py> int(3+4j)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: can't convert complex to int; use int(abs(z))
> py> 3 == 3+4j
> False
>
> So even though the conversion raises an exception, the
> values are determined to be not equal. Again, because int
> is a nearly true subset of complex, the conversion goes
> the other way, but *if* it would use the complex->int
> conversion, then the TypeError should be taken as
> a guarantee that the objects don't compare equal.

In the above example, you clearly know that the two are
unequal due to the relationship between complex numbers
having an imaginary part and integers..

The same is true for the overflow case:

>>> 2**10000 == 1.23
False
>>> float(2**10000)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
OverflowError: long int too large to convert to float

(Note that in Python 2.3 this used to raise an exception as well.)

However, this is not the case for 8-bit string vs. Unicode,
since you cannot use such extra knowledge if you find that ASCII
encoding assumption obviously doesn't match the string
in question.

> Expanding this view to Unicode should mean that a unicode
> string U equals a byte string B if
> U.encode(system_encode) == B or B.decode(system_encoding) == U,
> and that they don't equal otherwise 

Agreed.

Note that Python always coerces to the "bigger" type. As a result,
the second option is what is actually implemented in Python.

> (e.g. if the conversion
> fails with a "not convertible" exception). 

I disagree with this part.

Failure to decode a string doesn't imply inequality. It implies
that the programmer needs to step in and correct the problem by
making an explicit and conscious decision.

The alternative would be to decide that equal comparisons should never
be allowed to raise exceptions and instead have the equal comparison
return False. In which case, we'd have the revert the dict patch
altogether and instead silence all exceptions that
are generated during the equal comparison (not only in the dict
implementation), replacing them with a False return value.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 07 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::


From oren at hishome.net  Fri Aug  4 20:26:09 2006
From: oren at hishome.net (Oren Tirosh)
Date: Fri, 4 Aug 2006 21:26:09 +0300
Subject: [Python-Dev] uuid module - byte order issue
In-Reply-To: <Pine.LNX.4.58.0608041217290.32166@server1.LFW.org>
References: <7168d65a0608031311g3479824ds18a9ede4a33cae7e@mail.gmail.com>
	<Pine.LNX.4.58.0608041217290.32166@server1.LFW.org>
Message-ID: <7168d65a0608041126y6f02f127of48530b752afef74@mail.gmail.com>

On 04/08/06, Ka-Ping Yee <python-dev at zesty.ca> wrote:
> On Thu, 3 Aug 2006, Oren Tirosh wrote:
> > The UUID module uses network byte order, regardless of the platform
> > byte order. On little-endian platforms like Windows the ".bytes"
> > property of UUID objects is not compatible with the memory layout
>
> RFC 4122 says:
>
>     In the absence of explicit application or presentation protocol
>     specification to the contrary, a UUID is encoded as a 128-bit
>     object, as follows:
>
>     The fields are encoded as 16 octets, with the sizes and order of
>     the fields defined above, and with each field encoded with the
>     Most Significant Byte first (known as network byte order).

RFC 4122 defines a canonical byte order for UUIDs but also makes
explicit reference to the fact that UUIDs are stored locally in native
byte order. The final step in the RFC 4122 UUID generation algorithm
is:

>   o  Convert the resulting UUID to local byte order.

So this is not another case of the
Microsoft-implements-RFC-incorrectly syndrome. After all, they are one
of the co-authors of the document.

Compatibility with Windows "GUIDs" may be one of the most important
use cases for the UUID module. It's important to resolve this or users
will have unpleasant surprises. I did.

alternatives:

1. Default is big endian byte order. Little endian is explicit.
2. Default is native byte order. Little endian and big endian are explicit.
3. No default. Little endian and big endian are both explicit.

All three are relevant for both the constructor and retrieving the
byte representation.

  Oren

From tim.peters at gmail.com  Wed Aug 16 07:22:42 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 16 Aug 2006 01:22:42 -0400
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E27820.40205@canterbury.ac.nz>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>
	<44E27820.40205@canterbury.ac.nz>
Message-ID: <1f7befae0608152222n7403360ane3ece2174f7d7cb8@mail.gmail.com>

[Greg Ewing]
> Surely there's some compromise that would allow
> recently-used ints to be kept around, but reclaimed
> if memory becomes low?

Not without losing /something/ we currently enjoy.  The current int
scheme has theoretically optimal memory use too, consuming 12 bytes
per int object on a 32-bit box.  That's minimal, because the object's
type pointer, refcount, and integer value each require 4 bytes on a
32-bit box.

It does this by allocating "big blocks" and carving them into 12-byte
slices itself.  But as with any "big block" scheme, a live integer
anywhere in a block prevents the entire block from being freed, and
objects can't /move/ in CPython either (so, e.g., if a single live
integer is keeping a big block alive, we can't move it into some other
block).  That's the same kind of problem obmalloc has with its "very
big block" arenas, and for which only heuristic help was added (with
considerable increase in complexity) for 2.5 (before 2.5, obmalloc
never freed an arena, same as ints and floats never free their
versions of big blocks).

From martin at v.loewis.de  Wed Aug 16 08:23:53 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 08:23:53 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E27796.20101@canterbury.ac.nz>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de> <44E27796.20101@canterbury.ac.nz>
Message-ID: <44E2B9F9.3020004@v.loewis.de>

Greg Ewing schrieb:
> Martin v. L?wis wrote:
>> We had this discussion before; if you use ob_size==0 to indicate
>> that it's an int, this space isn't needed in a long int.
> 
> What about int subclasses?

It's what Guido proposes.

It would still leave two types (perhaps three) at the C level,
so C code might have to continue making conditional code depending
on which of these it is. Also, Python code that dispatches by type
still needs to make the distinction.

Regards,
Martin

From martin at v.loewis.de  Wed Aug 16 08:25:32 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 08:25:32 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E27820.40205@canterbury.ac.nz>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>
	<44E27820.40205@canterbury.ac.nz>
Message-ID: <44E2BA5C.7010405@v.loewis.de>

Greg Ewing schrieb:
> Guido van Rossum wrote:
>> I worry that dropping the special allocator will be too slow.
> 
> Surely there's some compromise that would allow
> recently-used ints to be kept around, but reclaimed
> if memory becomes low?

Hardly. The efficiency of the special-case allocator also
comes from the fact that it doesn't ever have to release
memory. Just try changing it to see what I mean.

Regards,
Martin

From martin at v.loewis.de  Wed Aug 16 08:28:13 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 08:28:13 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E27939.9060505@canterbury.ac.nz>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
	<44E27939.9060505@canterbury.ac.nz>
Message-ID: <44E2BAFD.50105@v.loewis.de>

Greg Ewing schrieb:
>> There isn't? Actually a lot of APIs currently assumen that.
> 
> Also it means you'd pay a penalty every time you
> access it, whereas presumably short ints are the
> case we want to optimise for speed as well.

That penalty is already paid today. Much code dealing with
ints has a type test whether it's an int or a long. If
int and long become subtypes of each other or of some abstract
type, performance will decrease even more because a subtype
test is quite expensive if the object is neither int nor
long (it has to traverse the entire base type hierarchy to
find out its not inherited from int).

Regards,
Martin

From nnorwitz at gmail.com  Wed Aug 16 08:46:26 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 15 Aug 2006 23:46:26 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E2BAFD.50105@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
	<44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de>
Message-ID: <ee2a432c0608152346m6a0adb54m135355bf4beeb462@mail.gmail.com>

On 8/15/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> That penalty is already paid today. Much code dealing with
> ints has a type test whether it's an int or a long. If
> int and long become subtypes of each other or of some abstract
> type, performance will decrease even more because a subtype
> test is quite expensive if the object is neither int nor
> long (it has to traverse the entire base type hierarchy to
> find out its not inherited from int).

I was playing around with a little patch to avoid that penalty.  It
doesn't take any additional memory, just a handful of bits we aren't
using. :-)

For the more common builtin types, it stores whether it's a subclass
in tp_flags, so there's no function call necessary and it's a constant
time operation.  It was faster when doing simple stuff.  Haven't
thought much whether this is really worthwhile or not.

n
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fast-sub.diff
Type: text/x-diff
Size: 6669 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060815/4a362cc4/attachment.bin 

From nnorwitz at gmail.com  Wed Aug 16 09:06:39 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 16 Aug 2006 00:06:39 -0700
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <ebtlkr$joi$1@sea.gmane.org>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
	<ebtlkr$joi$1@sea.gmane.org>
Message-ID: <ee2a432c0608160006s3ebc47c8w6112036028802361@mail.gmail.com>

On 8/15/06, Neil Schemenauer <nas at arctrix.com> wrote:
>
> It would be nice if someone could bytecompile Lib using
> Tools/compiler/compile.py and then run the test suite.  I'd do it
> myself but can't spare the time at the moment (I started but ran
> into what seems to be a gcc bug along the way).

Has this been done before?

# This code causes python to segfault
def foo(S):
  all(x > 42 for x in S)

# around Python/ceval.c 2167:  x = (*v->ob_type->tp_iternext)(v);
# tp_iternext is NULL

I added the changes below to Lib/compiler/pycodegen.py which are
clearly wrong.  It just crashes in a diff place.  I think the changes
to genxpr inner may be close to correct.  The changes to _makeClosure
and visitGenExpr are clearly wrong.  I was just wondering how far it
would go.  There are a bunch of differences. Some are the bytecode
optimizations or different ordering, but others are things dealing
with co_names, co_varnames.

Hopefully someone has time to look into this.  Otherwise, it will have
to wait for 2.5.1

n
--
Index: Lib/compiler/pycodegen.py
===================================================================
--- Lib/compiler/pycodegen.py   (revision 51305)
+++ Lib/compiler/pycodegen.py   (working copy)
@@ -628,9 +628,9 @@
         self.newBlock()
         self.emit('POP_TOP')

-    def _makeClosure(self, gen, args):
+    def _makeClosure(self, gen, args, gen_outer=False):
         frees = gen.scope.get_free_vars()
-        if frees:
+        if frees and not gen_outer:
             for name in frees:
                 self.emit('LOAD_CLOSURE', name)
             self.emit('BUILD_TUPLE', len(frees))
@@ -646,7 +646,7 @@
         walk(node.code, gen)
         gen.finish()
         self.set_lineno(node)
-        self._makeClosure(gen, 0)
+        self._makeClosure(gen, 0, True)
         # precomputation of outmost iterable
         self.visit(node.code.quals[0].iter)
         self.emit('GET_ITER')
@@ -655,6 +655,11 @@
     def visitGenExprInner(self, node):
         self.set_lineno(node)
         # setup list
+        after = self.newBlock()
+        start = self.newBlock()
+        self.setups.push((LOOP, start))
+        self.emit('SETUP_LOOP', after)
+        self.nextBlock(start)

         stack = []
         for i, for_ in zip(range(len(node.quals)), node.quals):
@@ -676,8 +681,12 @@
                 self.startBlock(cont)
                 self.emit('POP_TOP')
                 self.nextBlock(skip_one)
+            self.emit('POP_TOP')
             self.emit('JUMP_ABSOLUTE', start)
             self.startBlock(anchor)
+        self.emit('POP_BLOCK')
+        self.setups.pop()
+        self.nextBlock(after)
         self.emit('LOAD_CONST', None)

     def visitGenExprFor(self, node):

From kbk at shore.net  Wed Aug 16 09:16:58 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed, 16 Aug 2006 03:16:58 -0400 (EDT)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200608160716.k7G7GwcP015299@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  404 open ( +2) /  3376 closed (+16) /  3780 total (+18)
Bugs    :  860 open ( -1) /  6131 closed (+17) /  6991 total (+16)
RFE     :  229 open ( +1) /   235 closed ( +1) /   464 total ( +2)

New / Reopened Patches
______________________

option to leave tempfile.NamedTemporaryFile around on close  (2006-08-10)
       http://python.org/sf/1537850  opened by  djmdjm

Patch to fix __index__() clipping  (2006-08-11)
CLOSED http://python.org/sf/1538606  opened by  Travis Oliphant

Patch cElementTree to export CurrentLineNumber  (2006-08-11)
       http://python.org/sf/1538691  opened by  Robin Bryce

Replace unicode.__eq__ exceptions with a warning  (2006-08-11)
CLOSED http://python.org/sf/1538956  opened by  M.-A. Lemburg

Add readinto method to StringIO and cStringIO  (2006-08-12)
       http://python.org/sf/1539381  opened by  Alexander Belopolsky

Backports from trunk to release24-maint  (2006-08-15)
       http://python.org/sf/1540329  opened by  flub

tarfile __slots__ addition  (2006-08-14)
       http://python.org/sf/1540385  opened by  Brian Harring

Patches for OpenBSD 4.0  (2006-08-15)
       http://python.org/sf/1540470  opened by  djmdjm

Use Py_ssize_t for rangeobject members  (2006-08-15)
       http://python.org/sf/1540617  opened by  Alexander Belopolsky

except too broad  (2006-08-15)
       http://python.org/sf/1540849  opened by  Jim Jewett

with is now a blockopener  (2006-08-15)
CLOSED http://python.org/sf/1540851  opened by  Jim Jewett

IOBinding overly broad except (2nd try)  (2006-08-15)
CLOSED http://python.org/sf/1540856  opened by  Jim Jewett

except too broad  (2006-08-15)
CLOSED http://python.org/sf/1540857  opened by  Jim Jewett

IOBinding broad except - try 3  (2006-08-15)
CLOSED http://python.org/sf/1540859  opened by  Jim Jewett

CodeContext visibility   (2006-08-15)
       http://python.org/sf/1540869  opened by  Jim Jewett

broken shortcut keys  (2006-08-15)
       http://python.org/sf/1540874  opened by  Jim Jewett

make IDLE honor the quit() builtin  (2006-08-15)
CLOSED http://python.org/sf/1540892  opened by  Jim Jewett

Patches Closed
______________

Add notes on locale module changes to whatsnew25.tex  (2006-08-03)
       http://python.org/sf/1534027  closed by  akuchling

Build ctypes on OpenBSD x86_64  (2006-08-08)
       http://python.org/sf/1536908  closed by  theller

Fix __index__() clipping really big numbers  (2006-07-29)
       http://python.org/sf/1530738  closed by  ncoghlan

PyShell.recall - fix indentation logic  (2006-07-25)
       http://python.org/sf/1528468  closed by  kbk

Patch to fix __index__() clipping  (2006-08-11)
       http://python.org/sf/1538606  closed by  nnorwitz

Replace unicode.__eq__ exceptions with a warning  (2006-08-11)
       http://python.org/sf/1538956  closed by  lemburg

Replace the ctypes internal '_as_parameter_' mechanism  (2006-08-02)
       http://python.org/sf/1532975  closed by  theller

writelines() in bz2 module does not raise check for errors  (2006-08-06)
       http://python.org/sf/1535500  closed by  gbrandl

trace.py on win32 has problems with lowercase drive names  (2006-08-07)
       http://python.org/sf/1536071  closed by  gbrandl

Give Cookie.py its own _idmap  (2006-07-13)
       http://python.org/sf/1521882  closed by  gbrandl

socket.gethostbyaddr fix for machines with incomplete DNS  (2006-06-23)
       http://python.org/sf/1511317  closed by  gbrandl

with is now a blockopener  (2006-08-15)
       http://python.org/sf/1540851  closed by  kbk

IOBinding overly broad except (2nd try)  (2006-08-15)
       http://python.org/sf/1540856  closed by  jimjjewett

except too broad  (2006-08-15)
       http://python.org/sf/1540857  closed by  jimjjewett

IOBinding broad except - try 3  (2006-08-15)
       http://python.org/sf/1540859  closed by  jimjjewett

make IDLE honor the quit() builtin  (2006-08-15)
       http://python.org/sf/1540892  closed by  kbk

New / Reopened Bugs
___________________

2nd issue with need for speed patch  (2006-08-09)
       http://python.org/sf/1537167  opened by  Robin Bryce

Missing platform information in subprocess documentation  (2006-08-09)
CLOSED http://python.org/sf/1537195  opened by  Aaron Bingham

urllib2 httplib _read_chunked timeout  (2006-08-09)
       http://python.org/sf/1537445  opened by  devloop

Installation on Windows Longhorn  (2006-08-10)
       http://python.org/sf/1537601  opened by  O.R.Senthil Kumaran

import on cElementTree on Windows  (2006-08-09)
       http://python.org/sf/1537685  reopened by  thomasbhickey

import on cElementTree on Windows  (2006-08-09)
       http://python.org/sf/1537685  opened by  Thomas B Hickey

indent changes when copying command  (2006-08-10)
CLOSED http://python.org/sf/1538445  opened by  mjd__

PyThreadState_SetAsyncExc bug  (2006-08-11)
       http://python.org/sf/1538556  opened by  ganges master

pyo's are not overwritten by different optimization levels  (2006-08-11)
       http://python.org/sf/1538778  opened by  Toshio Kuratomi

Long command lines don't work on Windows  (2006-08-12)
       http://python.org/sf/1539295  opened by  Albert Strasheim

distutils example code missing imports  (2006-08-13)
CLOSED http://python.org/sf/1539336  opened by  Albert Strasheim

Identifiers begining with __ renamed  (2006-08-14)
CLOSED http://python.org/sf/1539847  opened by  W Barnes

warnings in interactive sessions  (2006-08-14)
       http://python.org/sf/1539925  opened by  M.-A. Lemburg

Closing a pipe can lead to uncompleted command  (2006-08-14)
       http://python.org/sf/1539954  opened by  Stefan Sonnenberg

Insane format string should cause ValueError  (2006-08-14)
CLOSED http://python.org/sf/1539955  opened by  Nick Coghlan

SocketServer.ForkingMixIn.collect_children() waits on pid 0  (2006-08-14)
       http://python.org/sf/1540386  opened by  Neal Norwitz

cgi.py error on parsing/handling content-disposition   (2006-08-15)
       http://python.org/sf/1540529  opened by  Dirk Holtwick

Bugs Closed
___________

PyArg_ParseTupleAndKeywords potential core dump  (2006-07-17)
       http://python.org/sf/1523610  closed by  gbrandl

Missing platform information in subprocess documentation  (2006-08-09)
       http://python.org/sf/1537195  closed by  gbrandl

Concatenation on a long string breaks  (2006-07-21)
       http://python.org/sf/1526585  closed by  arigo

import on cElementTree on Windows  (2006-08-09)
       http://python.org/sf/1537685  closed by  gbrandl

cgi.FieldStorage memory usage can spike in line-oriented ops  (2005-01-30)
       http://python.org/sf/1112549  closed by  gvanrossum

tokenize module does not detect inconsistent dedents  (2005-06-21)
       http://python.org/sf/1224621  closed by  gbrandl

fix inplace assignment for immutable sequences  (2006-02-21)
       http://python.org/sf/1436226  closed by  gbrandl

BROWSER path with capital letters  (2006-07-27)
       http://python.org/sf/1529655  closed by  sf-robot

struct.pack raises TypeError where it used to convert  (2006-07-28)
       http://python.org/sf/1530559  closed by  nnorwitz

indent changes when copying command  (2006-08-11)
       http://python.org/sf/1538445  closed by  kbk

distutils example code missing imports  (2006-08-12)
       http://python.org/sf/1539336  closed by  nnorwitz

Segmentation fault when importing expat from xml.parser  (2005-07-27)
       http://python.org/sf/1246405  closed by  sf-robot

CSV regression in 2.5a1: multi-line cells  (2006-04-05)
       http://python.org/sf/1465014  closed by  sf-robot

Installing documentation doesn't make it show up  (2003-07-06)
       http://python.org/sf/766842  closed by  sf-robot

infinite __str__ recursion in thread causes seg fault  (2003-07-31)
       http://python.org/sf/780714  closed by  sf-robot

Identifiers begining with __ renamed  (2006-08-14)
       http://python.org/sf/1539847  closed by  gbrandl

Insane format string should cause ValueError  (2006-08-14)
       http://python.org/sf/1539955  closed by  gbrandl

CTypes _as_parameter_ not working as documented  (2006-08-03)
       http://python.org/sf/1533481  closed by  theller

PythonLauncher uses incorrect working directory  (2006-07-23)
       http://python.org/sf/1527397  closed by  sf-robot

Quitter object masked  (2006-05-01)
       http://python.org/sf/1479785  closed by  kbk

New / Reopened RFE
__________________

csv module: add header row to DictWriter  (2006-08-10)
       http://python.org/sf/1537721  opened by  ed_abraham

Allow choice of copy function in shutil.copytree  (2006-08-14)
       http://python.org/sf/1540112  opened by  Tony


From thomas at python.org  Wed Aug 16 13:37:47 2006
From: thomas at python.org (Thomas Wouters)
Date: Wed, 16 Aug 2006 13:37:47 +0200
Subject: [Python-Dev] Making 'python -t' the default.
Message-ID: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com>

Can I suggest making 'python -t' the default, in 2.6? It makes python warn
about mixing tabs and spaces in indentation. In Py3k, '-tt' (the
error-raising version) will be the default, instead. I see too many newsbies
with problems they can't figure out because they mix tabs and spaces
(usually not even consciously, but just because their editor isn't playing
along.) There probably should be an option to turn the warning off again
(although I won't fight for it ;) -- but I don't know what that option
should be.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060816/8e9559f9/attachment.html 

From g.brandl at gmx.net  Wed Aug 16 14:19:55 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Aug 2006 14:19:55 +0200
Subject: [Python-Dev] Making 'python -t' the default.
In-Reply-To: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com>
References: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com>
Message-ID: <ebv2hb$9cc$1@sea.gmane.org>

Thomas Wouters wrote:
> 
> Can I suggest making 'python -t' the default, in 2.6? It makes python 
> warn about mixing tabs and spaces in indentation.

+1.


> In Py3k, '-tt' (the 
> error-raising version) will be the default, instead.

Or disallow tabs altogether.

> I see too many 
> newsbies with problems they can't figure out because they mix tabs and 
> spaces (usually not even consciously, but just because their editor 
> isn't playing along.) There probably should be an option to turn the 
> warning off again (although I won't fight for it ;) -- but I don't know 
> what that option should be.

Perhaps -T.

Georg


From amk at amk.ca  Wed Aug 16 14:43:09 2006
From: amk at amk.ca (A.M. Kuchling)
Date: Wed, 16 Aug 2006 08:43:09 -0400
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <87mza5csbr.fsf@hydra.bayview.thirdcreek.com>
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
	<ca471dc20608150914n7613fc60q5be564b6aaea9903@mail.gmail.com>
	<44E1F521.2010908@egenix.com>
	<200608161145.44938.anthony@interlink.com.au>
	<87mza5csbr.fsf@hydra.bayview.thirdcreek.com>
Message-ID: <20060816124309.GA4841@localhost.localdomain>

On Tue, Aug 15, 2006 at 10:44:40PM -0400, Kurt B. Kaiser wrote:
> It would be nice if the key IDLE changes could make it to the "What's New
> in Python X.X".  If Andrew is interested, I could draft something for him.

Sure!  I can try to look through the IDLE NEWS file, but you'd
certainly have a better idea of which changes are most significant to
IDLE users. 

(You can commit your changes to whatsnew25.tex, module Anthony's trunk
freeze in 12 hours or so, or just write plain text and let me add the
LaTeX markup.)

--amk


From barry at python.org  Wed Aug 16 14:48:17 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 16 Aug 2006 08:48:17 -0400
Subject: [Python-Dev] Making 'python -t' the default.
In-Reply-To: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com>
References: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com>
Message-ID: <580957E2-1635-44AB-97EF-9649008E241D@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Aug 16, 2006, at 7:37 AM, Thomas Wouters wrote:

>
> Can I suggest making 'python -t' the default, in 2.6? It makes  
> python warn about mixing tabs and spaces in indentation. In Py3k, '- 
> tt' (the error-raising version) will be the default,

+1.  We even have Subversion hooks in our repository to reject any  
commits that have tabs in them (except for Makefiles and a few  
whitelisted 3rd-party apps that we don't want to reformat).

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBROMUEXEjvBPtnXfVAQJSuQQAlU/FqiDhVUXKiIIaH91s0iyyd8/NgHMn
fuA5fO06vcpAWxDvZtpmF8xAqGbN9AUh87iISDEEYUdpe1xrs1hly2QEJ5xIBnmB
rF+3FfDZI+YKCkCHzHhG01jJBH6j/4pW0ZxUs4DAKz5os0o1xor7Yiy1sI/t1cVR
Ra0m8ByDssE=
=D4hn
-----END PGP SIGNATURE-----

From skip at pobox.com  Wed Aug 16 16:02:27 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 16 Aug 2006 09:02:27 -0500
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E2BA5C.7010405@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>
	<44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de>
Message-ID: <17635.9587.937810.163966@montanaro.dyndns.org>

    Guido> I worry that dropping the special allocator will be too slow.

    Greg> Surely there's some compromise that would allow recently-used ints
    Greg> to be kept around, but reclaimed if memory becomes low?

    Martin> Hardly. The efficiency of the special-case allocator also comes
    Martin> from the fact that it doesn't ever have to release memory. Just
    Martin> try changing it to see what I mean.

Wouldn't use of obmalloc offset much of that?  Before obmalloc was
available, the int free list was a huge win.  Is it likely to be such a huge
win today?

Skip

From guido at python.org  Wed Aug 16 16:51:16 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Aug 2006 07:51:16 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E2B9F9.3020004@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de> <44E27796.20101@canterbury.ac.nz>
	<44E2B9F9.3020004@v.loewis.de>
Message-ID: <ca471dc20608160751v4011e77by7adcac1a464a66d5@mail.gmail.com>

On 8/15/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Greg Ewing schrieb:
> > Martin v. L?wis wrote:
> >> We had this discussion before; if you use ob_size==0 to indicate
> >> that it's an int, this space isn't needed in a long int.
> >
> > What about int subclasses?
>
> It's what Guido proposes.
>
> It would still leave two types (perhaps three) at the C level,
> so C code might have to continue making conditional code depending
> on which of these it is. Also, Python code that dispatches by type
> still needs to make the distinction.

I'm not sure that subclassing ints gives us much. We could make int
and long "final" types, and then all we have to do is tweak type() and
__class__ so that they always return the 'int' type.

Alternatively, yes, there would be some minimal awareness of the two
types in Python -- but nothing like we currently have; dispatching on
exact type (which we discourage anyway) would be the only case. Would
that be so bad?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Wed Aug 16 17:25:20 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Aug 2006 17:25:20 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ee2a432c0608152346m6a0adb54m135355bf4beeb462@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>	<44E1EFEF.20305@v.loewis.de>	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>	<44E248A7.4080107@v.loewis.de>	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>	<44E27939.9060505@canterbury.ac.nz>
	<44E2BAFD.50105@v.loewis.de>
	<ee2a432c0608152346m6a0adb54m135355bf4beeb462@mail.gmail.com>
Message-ID: <ebvdd0$j56$1@sea.gmane.org>

Neal Norwitz wrote:
> On 8/15/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>
>> That penalty is already paid today. Much code dealing with
>> ints has a type test whether it's an int or a long. If
>> int and long become subtypes of each other or of some abstract
>> type, performance will decrease even more because a subtype
>> test is quite expensive if the object is neither int nor
>> long (it has to traverse the entire base type hierarchy to
>> find out its not inherited from int).
> 
> I was playing around with a little patch to avoid that penalty.  It
> doesn't take any additional memory, just a handful of bits we aren't
> using. :-)
> 
> For the more common builtin types, it stores whether it's a subclass
> in tp_flags, so there's no function call necessary and it's a constant
> time operation.  It was faster when doing simple stuff.  Haven't
> thought much whether this is really worthwhile or not.

This might als be helpful when exceptions have to inherit from
BaseException in Py3k.

Georg


From martin at v.loewis.de  Wed Aug 16 17:29:37 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 17:29:37 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <17635.9587.937810.163966@montanaro.dyndns.org>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>
	<44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de>
	<17635.9587.937810.163966@montanaro.dyndns.org>
Message-ID: <44E339E1.8030202@v.loewis.de>

skip at pobox.com schrieb:
>     Guido> I worry that dropping the special allocator will be too slow.
> 
>     Greg> Surely there's some compromise that would allow recently-used ints
>     Greg> to be kept around, but reclaimed if memory becomes low?
> 
>     Martin> Hardly. The efficiency of the special-case allocator also comes
>     Martin> from the fact that it doesn't ever have to release memory. Just
>     Martin> try changing it to see what I mean.
> 
> Wouldn't use of obmalloc offset much of that?  Before obmalloc was
> available, the int free list was a huge win.  Is it likely to be such a huge
> win today?

That's my theory: it isn't a huge win. Guido has another theory: it's
still faster. Only benchmarking can tell.

Regards,
Martin

From martin at v.loewis.de  Wed Aug 16 17:34:48 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 17:34:48 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ca471dc20608160751v4011e77by7adcac1a464a66d5@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>	
	<44E1EFEF.20305@v.loewis.de>	
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>	
	<44E248A7.4080107@v.loewis.de> <44E27796.20101@canterbury.ac.nz>	
	<44E2B9F9.3020004@v.loewis.de>
	<ca471dc20608160751v4011e77by7adcac1a464a66d5@mail.gmail.com>
Message-ID: <44E33B18.9040403@v.loewis.de>

Guido van Rossum schrieb:
> I'm not sure that subclassing ints gives us much. We could make int
> and long "final" types, and then all we have to do is tweak type() and
> __class__ so that they always return the 'int' type.

I don't think this can work - there would be too many ways for the
"real" types to leak, anyway. People would come up with hacks
like reload(sys), and then complain that they have to use such
hacks.

> Alternatively, yes, there would be some minimal awareness of the two
> types in Python -- but nothing like we currently have; dispatching on
> exact type (which we discourage anyway) would be the only case. Would
> that be so bad?

I thought it was the ultimate goal of PEP 237 to unify int and long,
in all respects.

I'll do some benchmarking.

Regards,
Martin


From dinov at exchange.microsoft.com  Wed Aug 16 17:31:43 2006
From: dinov at exchange.microsoft.com (Dino Viehland)
Date: Wed, 16 Aug 2006 08:31:43 -0700
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
Message-ID: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>

We've been working on fixing some exception handling bugs in IronPython where we differ from CPython.  Along the way we ran into this issue which causes CPython to crash when the code below is run.  It crashes on both 2.4 and 2.5 beta 3.  The code's technically illegal, but it probably shouldn't crash either :)

def test():
    for abc in range(10):
        try: pass
        finally:
            try:
                continue
            except:
                pass


test()


From martin at v.loewis.de  Wed Aug 16 18:08:13 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 18:08:13 +0200
Subject: [Python-Dev] Benchmarking the int allocator (Was: Type of range
	object members)
In-Reply-To: <17635.9587.937810.163966@montanaro.dyndns.org>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>	<44E1EFEF.20305@v.loewis.de>	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>	<44E248A7.4080107@v.loewis.de>	<ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>	<44E27820.40205@canterbury.ac.nz>
	<44E2BA5C.7010405@v.loewis.de>
	<17635.9587.937810.163966@montanaro.dyndns.org>
Message-ID: <44E342ED.6060507@v.loewis.de>

skip at pobox.com schrieb:
> Wouldn't use of obmalloc offset much of that?  Before obmalloc was
> available, the int free list was a huge win.  Is it likely to be such a huge
> win today?

I have now some numbers. For the attached t.py, the unmodified svn
python gives

Test 1 3.25420880318
Test 2 1.86433696747

and the one with the attached patch gives

Test 1 3.45080399513
Test 2 2.09729003906

So there apparently is a performance drop on int allocations of about
5-10%.

On this machine (P4 3.2GHz) I could not find any difference in pystones
from this patch.

Notice that this test case is extremely focused on measuring int
allocation (I just noticed I should have omitted the for loop in
the second case, though).

Regards,
Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: t.py
Type: text/x-python
Size: 181 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060816/239c9e63/attachment.py 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: intalloc.diff
Url: http://mail.python.org/pipermail/python-dev/attachments/20060816/239c9e63/attachment.diff 

From jimjjewett at gmail.com  Wed Aug 16 18:39:47 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 16 Aug 2006 12:39:47 -0400
Subject: [Python-Dev] IDLE patches - bugfix or not?
In-Reply-To: <200608161210.46100.anthony@interlink.com.au>
References: <fb6fbf560608151319w21753f3ek575e7709b8ba3521@mail.gmail.com>
	<200608161210.46100.anthony@interlink.com.au>
Message-ID: <fb6fbf560608160939s78b45c8cr10cecf2bed87c2f6@mail.gmail.com>

> > python.org/sf/1540874 -- broken shortcut keys.  On windows, only one
> > entry per menu can be reached with the same shortcut letter, so
> > advertising others is just an attractive nuisance.  I'm not sure that
> > other systems wouldn't be able to use the hidden shortcuts.

On 8/15/06, Anthony Baxter <anthony at interlink.com.au> wrote:

> Tough call. I guess it depends on the other systems - I will try this on Linux
> at least, and see if it works there. If it's broken everywhere, then changing
> it would seem the least offensive choice.

Thank you.

Kurt Kaiser:
> I would have been inclined to make the other choice, i.e. 'p' as the
> hotkey for print, rather than the rarely used "save copy as".

The existing functionality (at least on windows) is that p brings up a
Path Browser window; print and save copy are *both* masked.  I agree
that p *should* be for print, but I didn't want to remove an existing
(working) shortcut this late.

So I just stopped advertising the (unusable) shortcut on print.  I did
add the (currently unused) 'y' for "save copy as", which I suppose
might be considered a feature.  Simply removing the claimed but broken
shortcut would indeed fix the attractive nuisance problem.

-jJ

From nas at arctrix.com  Wed Aug 16 18:59:31 2006
From: nas at arctrix.com (Neil Schemenauer)
Date: Wed, 16 Aug 2006 16:59:31 +0000 (UTC)
Subject: [Python-Dev] no remaining issues blocking 2.5 release
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
	<ebtlkr$joi$1@sea.gmane.org>
	<ee2a432c0608160006s3ebc47c8w6112036028802361@mail.gmail.com>
Message-ID: <ebvitj$94r$1@sea.gmane.org>

Neal Norwitz <nnorwitz at gmail.com> wrote:
> On 8/15/06, Neil Schemenauer <nas at arctrix.com> wrote:
>>
>> It would be nice if someone could bytecompile Lib using
>> Tools/compiler/compile.py and then run the test suite.
>
> Has this been done before?

Obviously not. :-)

> # This code causes python to segfault
> def foo(S):
>   all(x > 42 for x in S)

Hmm, it seems to work for me.  The bytecode generated by
Lib/compiler is the same as the normal compiler.  Do you have a full
test case?

  Neil


From guido at python.org  Wed Aug 16 19:08:00 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Aug 2006 10:08:00 -0700
Subject: [Python-Dev] Benchmarking the int allocator (Was: Type of range
	object members)
In-Reply-To: <44E342ED.6060507@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>
	<44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de>
	<17635.9587.937810.163966@montanaro.dyndns.org>
	<44E342ED.6060507@v.loewis.de>
Message-ID: <ca471dc20608161008x37c7f948o6530076a786e76a2@mail.gmail.com>

On 8/16/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I have now some numbers. For the attached t.py, the unmodified svn
> python gives
>
> Test 1 3.25420880318
> Test 2 1.86433696747
>
> and the one with the attached patch gives
>
> Test 1 3.45080399513
> Test 2 2.09729003906
>
> So there apparently is a performance drop on int allocations of about
> 5-10%.
>
> On this machine (P4 3.2GHz) I could not find any difference in pystones
> from this patch.
>
> Notice that this test case is extremely focused on measuring int
> allocation (I just noticed I should have omitted the for loop in
> the second case, though).

I think the test isn't hardly focused enough on int allocation. I
wonder if you could come up with a benchmark that repeatedly allocates
100s of 1000s of ints and then deletes them? What if it also allocates
other small objects so the ints become more fragmented?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jcarlson at uci.edu  Wed Aug 16 19:11:28 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 16 Aug 2006 10:11:28 -0700
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
Message-ID: <20060816100610.19DD.JCARLSON@uci.edu>


Dino Viehland <dinov at exchange.microsoft.com> wrote:
> 
> We've been working on fixing some exception handling bugs in
> IronPython where we differ from CPython.  Along the way we ran into
> this issue which causes CPython to crash when the code below is run.
> It crashes on both 2.4 and 2.5 beta 3.  The code's technically illegal,
> but it probably shouldn't crash either :)

> def test():
>     for abc in range(10):
>         try: pass
>         finally:
>             try:
>                 continue
>             except:
>                 pass
> 
> 
> test()

It also reliably crashes 2.3, though I don't see anything in there as
being illegal, but maybe I don't understand the langauge as well as I
think I do.

Note that there is another segfaulting bug in CPython with regards to
threads that was recently closed, but which is still a problem:
http://python.org/sf/780714

Does IronPython survive in that case?


 - Josiah


From g.brandl at gmx.net  Wed Aug 16 19:15:54 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 16 Aug 2006 19:15:54 +0200
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <20060816100610.19DD.JCARLSON@uci.edu>
References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<20060816100610.19DD.JCARLSON@uci.edu>
Message-ID: <ebvjsb$cfh$1@sea.gmane.org>

Josiah Carlson wrote:
> Dino Viehland <dinov at exchange.microsoft.com> wrote:
>> 
>> We've been working on fixing some exception handling bugs in
>> IronPython where we differ from CPython.  Along the way we ran into
>> this issue which causes CPython to crash when the code below is run.
>> It crashes on both 2.4 and 2.5 beta 3.  The code's technically illegal,
>> but it probably shouldn't crash either :)
> 
>> def test():
>>     for abc in range(10):
>>         try: pass
>>         finally:
>>             try:
>>                 continue
>>             except:
>>                 pass
>> 
>> 
>> test()
> 
> It also reliably crashes 2.3, though I don't see anything in there as
> being illegal, but maybe I don't understand the langauge as well as I
> think I do.

"continue" is not supported inside a finally clause. If you put the
continue directly there or into the except clause, the compiler will
tell you.

It looks like nobody thought about that corner case when writing the
compiler check for continue.

Georg


From guido at python.org  Wed Aug 16 19:23:43 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 16 Aug 2006 10:23:43 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ee2a432c0608152346m6a0adb54m135355bf4beeb462@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
	<44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de>
	<ee2a432c0608152346m6a0adb54m135355bf4beeb462@mail.gmail.com>
Message-ID: <ca471dc20608161023p732bab86xdded704d505e74b8@mail.gmail.com>

On 8/15/06, Neal Norwitz <nnorwitz at gmail.com> wrote:
> I was playing around with a little patch to avoid that penalty.  It
> doesn't take any additional memory, just a handful of bits we aren't
> using. :-)
>
> For the more common builtin types, it stores whether it's a subclass
> in tp_flags, so there's no function call necessary and it's a constant
> time operation.  It was faster when doing simple stuff.  Haven't
> thought much whether this is really worthwhile or not.

I like it! I wonder if you should use another bit for "inherits from
BaseException". That would make catching and raising exceptions a bit
faster. It applies cleanly to py3k -- perhaps you should just check it
in there? +1 from me!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Aug 16 20:29:30 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 16 Aug 2006 14:29:30 -0400
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ee2a432c0608152346m6a0adb54m135355bf4beeb462@mail.gmail.co
 m>
References: <44E2BAFD.50105@v.loewis.de>
	<d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
	<44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de>
Message-ID: <5.1.1.6.0.20060816142136.022968e8@sparrow.telecommunity.com>

At 11:46 PM 8/15/2006 -0700, Neal Norwitz wrote:
>On 8/15/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>
>>That penalty is already paid today. Much code dealing with
>>ints has a type test whether it's an int or a long. If
>>int and long become subtypes of each other or of some abstract
>>type, performance will decrease even more because a subtype
>>test is quite expensive if the object is neither int nor
>>long (it has to traverse the entire base type hierarchy to
>>find out its not inherited from int).
>
>I was playing around with a little patch to avoid that penalty.  It
>doesn't take any additional memory, just a handful of bits we aren't
>using. :-)
>
>For the more common builtin types, it stores whether it's a subclass
>in tp_flags, so there's no function call necessary and it's a constant
>time operation.  It was faster when doing simple stuff.  Haven't
>thought much whether this is really worthwhile or not.

It seems to me that you could drop the FAST_SUBCLASS bit, since none of the 
other bits will be set if it is not a subclass of a builtin.  That would 
free up one flag bit -- perhaps usable for that BaseException flag Guido 
wants.  :)

(Of course, if you can't inherit from both BaseException and one of the 
other builtin types, it can just be another enumeration value within the 
bit mask.)


From martin at v.loewis.de  Wed Aug 16 21:52:13 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 21:52:13 +0200
Subject: [Python-Dev] Benchmarking the int allocator (Was: Type of range
	object members)
In-Reply-To: <ca471dc20608161008x37c7f948o6530076a786e76a2@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>	
	<44E1EFEF.20305@v.loewis.de>	
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>	
	<44E248A7.4080107@v.loewis.de>	
	<ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>	
	<44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de>	
	<17635.9587.937810.163966@montanaro.dyndns.org>	
	<44E342ED.6060507@v.loewis.de>
	<ca471dc20608161008x37c7f948o6530076a786e76a2@mail.gmail.com>
Message-ID: <44E3776D.90804@v.loewis.de>

Guido van Rossum schrieb:
> I think the test isn't hardly focused enough on int allocation. I
> wonder if you could come up with a benchmark that repeatedly allocates
> 100s of 1000s of ints and then deletes them? 

The question is: where to store them? In a pre-allocated list, or in a
growing list?

def t3():
    m = [0]*100000
    s = time.time()
    i = 0
    for i in xrange(100):
        for k in xrange(100000):
            m[k] = k
    print "Test 3",time.time()-s

def t4():
    s = time.time()
    i = 0
    for i in xrange(100):
        m = []
        for k in xrange(100000):
            m.append(k)
    print "Test 4",time.time()-s

This is 100s of 1000s of ints in the inner loop; test 3 puts them
into a pre-allocated list, test 4 discards the list each time and
lets it grow.

> What if it also allocates
> other small objects so the ints become more fragmented?

This allocator doesn't bother much about fragmentation: it's
constant-time most of the time on allocation, and often on
deallocation (especially when the memory is fragmented).
Also, it's hard to find an object that is as small as an
int; I think a one-element tuple applies:

def t5():
    s = time.time()
    i = 0
    for i in xrange(100):
        m = []
        for k in xrange(100000):
            m.append((k,))
    print "Test 5",time.time()-s

The timings, for the best of three runs:
           Py2.5      +obmalloc-for-int    slowdown
Test 3     1.8s           2.1s              15%
Test 4     3.6s           3.8s               5%
test 5     7.5s           7.5s               0

Regards,
Martin

From allison at shasta.stanford.edu  Wed Aug 16 19:22:00 2006
From: allison at shasta.stanford.edu (Dennis Allison)
Date: Wed, 16 Aug 2006 10:22:00 -0700 (PDT)
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <20060816100610.19DD.JCARLSON@uci.edu>
Message-ID: <Pine.LNX.4.44.0608161009530.5803-100000@shasta.stanford.edu>


def test():
  for abc in range(10):
      try: pass
      finally:
          try:
              pass
          except: 
              pass
 
test()

does not raise a segmentation fault.

On Wed, 16 Aug 2006, Josiah Carlson wrote:

> 
> Dino Viehland <dinov at exchange.microsoft.com> wrote:
> > 
> > We've been working on fixing some exception handling bugs in
> > IronPython where we differ from CPython.  Along the way we ran into
> > this issue which causes CPython to crash when the code below is run.
> > It crashes on both 2.4 and 2.5 beta 3.  The code's technically illegal,
> > but it probably shouldn't crash either :)
> 
> > def test():
> >     for abc in range(10):
> >         try: pass
> >         finally:
> >             try:
> >                 continue
> >             except:
> >                 pass
> > 
> > 
> > test()
> 
> It also reliably crashes 2.3, though I don't see anything in there as
> being illegal, but maybe I don't understand the langauge as well as I
> think I do.
> 
> Note that there is another segfaulting bug in CPython with regards to
> threads that was recently closed, but which is still a problem:
> http://python.org/sf/780714
> 
> Does IronPython survive in that case?
> 
> 
>  - Josiah
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/allison%40shasta.stanford.edu
> 

-- 


From martin at v.loewis.de  Wed Aug 16 22:21:54 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Aug 2006 22:21:54 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ee2a432c0608152346m6a0adb54m135355bf4beeb462@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>	
	<44E1EFEF.20305@v.loewis.de>	
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>	
	<44E248A7.4080107@v.loewis.de>	
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>	
	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>	
	<44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de>
	<ee2a432c0608152346m6a0adb54m135355bf4beeb462@mail.gmail.com>
Message-ID: <44E37E62.4060501@v.loewis.de>

Neal Norwitz schrieb:
> I was playing around with a little patch to avoid that penalty.  It
> doesn't take any additional memory, just a handful of bits we aren't
> using. :-)

There are common schemes that allow constant-time issubclass tests,
although they do require more memory:

1. give each base class a small unique number, starting with 1
   (0 means no number has been assigned). Give each class a bitmap
   of all base classes, plus a field for the maximum-numbered
   base class. Then,

   def issubclass(C, B):
     return B.classnum and (B.classnum < C.maxbasenum) and\
            bit_set(C.basenums, B.classnum)

   Supports multiple inheritance, space requirement linear
   with the number of classes that are used as base classes.
   Numbering should recycle class numbers when a class is
   gced.

2. restrict optimization to single-inheritance. Give each class
   a "depth" (distance from object, 0 for object and
   multiply-inherited classes). Also give each class an array
   of bases, ordered by depth. Then,

   def issubclass(C, B):
     if not C.depth: return expensive_issubclass(C, B)
     return B.depth < C.depth and (C.bases[B.depth] is B)

   Space requirement is linear with the depth of the class
   (but I think tp_mro could be used, if the formula is
    changed to (C.bases[C.depth-B.depth] is B))

Regards,
Martin

From dinov at exchange.microsoft.com  Wed Aug 16 19:46:46 2006
From: dinov at exchange.microsoft.com (Dino Viehland)
Date: Wed, 16 Aug 2006 10:46:46 -0700
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <Pine.LNX.4.44.0608161009530.5803-100000@shasta.stanford.edu>
References: <20060816100610.19DD.JCARLSON@uci.edu>
	<Pine.LNX.4.44.0608161009530.5803-100000@shasta.stanford.edu>
Message-ID: <7AD436E4270DD54A94238001769C2227372749F734@DF-GRTDANE-MSG.exchange.corp.microsoft.com>

Yeah, continue inside the finally block is illegal.  If you don't have the extra try/except nesting then continue is detected as a syntax error.

-----Original Message-----
From: Dennis Allison [mailto:allison at shasta.stanford.edu]
Sent: Wednesday, August 16, 2006 10:22 AM
To: Josiah Carlson
Cc: Dino Viehland; python-dev at python.org
Subject: Re: [Python-Dev] 2.4 & 2.5 beta 3 crash


def test():
  for abc in range(10):
      try: pass
      finally:
          try:
              pass
          except:
              pass

test()

does not raise a segmentation fault.

On Wed, 16 Aug 2006, Josiah Carlson wrote:

>
> Dino Viehland <dinov at exchange.microsoft.com> wrote:
> >
> > We've been working on fixing some exception handling bugs in
> > IronPython where we differ from CPython.  Along the way we ran into
> > this issue which causes CPython to crash when the code below is run.
> > It crashes on both 2.4 and 2.5 beta 3.  The code's technically
> > illegal, but it probably shouldn't crash either :)
>
> > def test():
> >     for abc in range(10):
> >         try: pass
> >         finally:
> >             try:
> >                 continue
> >             except:
> >                 pass
> >
> >
> > test()
>
> It also reliably crashes 2.3, though I don't see anything in there as
> being illegal, but maybe I don't understand the langauge as well as I
> think I do.
>
> Note that there is another segfaulting bug in CPython with regards to
> threads that was recently closed, but which is still a problem:
> http://python.org/sf/780714
>
> Does IronPython survive in that case?
>
>
>  - Josiah
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/allison%40shasta.sta
> nford.edu
>

--


From dinov at exchange.microsoft.com  Wed Aug 16 19:49:22 2006
From: dinov at exchange.microsoft.com (Dino Viehland)
Date: Wed, 16 Aug 2006 10:49:22 -0700
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <20060816100610.19DD.JCARLSON@uci.edu>
References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<20060816100610.19DD.JCARLSON@uci.edu>
Message-ID: <7AD436E4270DD54A94238001769C2227372749F736@DF-GRTDANE-MSG.exchange.corp.microsoft.com>

IronPython actually enables / disables stack overflow checking through either a command line option or by calling sys.setrecursionlimit.  By default we have no recursion limit.

Without those command line options we'll end up having the CLR throw an unrecoverable stack overflow exception (and the process will be terminated).  It's a little bit better than a seg fault, but not much.  With stack overflow checking enabled we'll actually catch the recursion for this and throw a RuntimeError stating maximum recursion depth reached.

-----Original Message-----
From: Josiah Carlson [mailto:jcarlson at uci.edu]
Sent: Wednesday, August 16, 2006 10:11 AM
To: Dino Viehland; python-dev at python.org
Subject: Re: [Python-Dev] 2.4 & 2.5 beta 3 crash


Dino Viehland <dinov at exchange.microsoft.com> wrote:
>
> We've been working on fixing some exception handling bugs in
> IronPython where we differ from CPython.  Along the way we ran into
> this issue which causes CPython to crash when the code below is run.
> It crashes on both 2.4 and 2.5 beta 3.  The code's technically
> illegal, but it probably shouldn't crash either :)

> def test():
>     for abc in range(10):
>         try: pass
>         finally:
>             try:
>                 continue
>             except:
>                 pass
>
>
> test()

It also reliably crashes 2.3, though I don't see anything in there as being illegal, but maybe I don't understand the langauge as well as I think I do.

Note that there is another segfaulting bug in CPython with regards to threads that was recently closed, but which is still a problem:
http://python.org/sf/780714

Does IronPython survive in that case?


 - Josiah


From mal at egenix.com  Thu Aug 17 00:01:23 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 17 Aug 2006 00:01:23 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E37E62.4060501@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>		<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>		<44E1EFEF.20305@v.loewis.de>		<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>		<44E248A7.4080107@v.loewis.de>		<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>		<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>		<44E27939.9060505@canterbury.ac.nz>
	<44E2BAFD.50105@v.loewis.de>	<ee2a432c0608152346m6a0adb54m135355bf4beeb462@mail.gmail.com>
	<44E37E62.4060501@v.loewis.de>
Message-ID: <44E395B3.6000805@egenix.com>

Martin v. L?wis wrote:
> Neal Norwitz schrieb:
>> I was playing around with a little patch to avoid that penalty.  It
>> doesn't take any additional memory, just a handful of bits we aren't
>> using. :-)
> 
> There are common schemes that allow constant-time issubclass tests,
> although they do require more memory:
> 
> 1. give each base class a small unique number, starting with 1
>    (0 means no number has been assigned). Give each class a bitmap
>    of all base classes, plus a field for the maximum-numbered
>    base class. Then,
> 
>    def issubclass(C, B):
>      return B.classnum and (B.classnum < C.maxbasenum) and\
>             bit_set(C.basenums, B.classnum)
> 
>    Supports multiple inheritance, space requirement linear
>    with the number of classes that are used as base classes.
>    Numbering should recycle class numbers when a class is
>    gced.
> 
> 2. restrict optimization to single-inheritance. Give each class
>    a "depth" (distance from object, 0 for object and
>    multiply-inherited classes). Also give each class an array
>    of bases, ordered by depth. Then,
> 
>    def issubclass(C, B):
>      if not C.depth: return expensive_issubclass(C, B)
>      return B.depth < C.depth and (C.bases[B.depth] is B)
> 
>    Space requirement is linear with the depth of the class
>    (but I think tp_mro could be used, if the formula is
>     changed to (C.bases[C.depth-B.depth] is B))

Two more:

3. Use a global cache of class objects that caches
   the issubclass() lookups.

   This is only amortized constant time, but easy to implement
   and has a few other nice features such as e.g. enabling
   traversal of the complete class inheritance forest (or tree
   if you just have new-style classes).

   Use weak references to the classes if you want to be
   able to GC them.

4. "Freeze" classes after they are constructed, meaning
   that all attributes from base-classes get bound to the
   inheriting class.

   This also speeds up method lookups considerably. Works
   great with classic classes, I'm not
   sure about new-style classes using e.g. staticmethods,
   slots and the like.

   mxTools has an implementation for classic classes called
   freeze().

   If you add special attributes such as ._issubclass_XYZ
   in the process, issubclass() would then be a single
   attribute lookup which is constant time.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 16 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From kbk at shore.net  Thu Aug 17 01:27:47 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed, 16 Aug 2006 19:27:47 -0400
Subject: [Python-Dev] no remaining issues blocking 2.5 release
In-Reply-To: <20060816124309.GA4841@localhost.localdomain> (A. M. Kuchling's
	message of "Wed, 16 Aug 2006 08:43:09 -0400")
References: <ee2a432c0608142131g59e2739dg2b4545ca7eb19b24@mail.gmail.com>
	<ca471dc20608150914n7613fc60q5be564b6aaea9903@mail.gmail.com>
	<44E1F521.2010908@egenix.com>
	<200608161145.44938.anthony@interlink.com.au>
	<87mza5csbr.fsf@hydra.bayview.thirdcreek.com>
	<20060816124309.GA4841@localhost.localdomain>
Message-ID: <87bqqkb6rw.fsf@hydra.bayview.thirdcreek.com>

"A.M. Kuchling" <amk at amk.ca> writes:

> On Tue, Aug 15, 2006 at 10:44:40PM -0400, Kurt B. Kaiser wrote:
>> It would be nice if the key IDLE changes could make it to the "What's New
>> in Python X.X".  If Andrew is interested, I could draft something for him.
>
> Sure!  I can try to look through the IDLE NEWS file, but you'd
> certainly have a better idea of which changes are most significant to
> IDLE users. 
>
> (You can commit your changes to whatsnew25.tex, module Anthony's trunk
> freeze in 12 hours or so, or just write plain text and let me add the
> LaTeX markup.)

%=================================================================
\subsection{The IDLE Integrated Development Environment}

Calltips have been greatly improved and a class attribute listbox
provides completions.

When the Shell cursor is on a previous command, <Enter> retrieves
that command.  But instead of replacing the input line, the previous
command is appended, preserving indentation.  If there is an active
selection, the selection will be appended to the input line.

IDLE does a tabnanny and syntax check before every F5/Run.

A number of changes were made to improve Mac OSX compatibility.

The advanced keybinding dialog was enabled, allowing multiple
complex key combinations to be bound to a given IDLE event.

A number of improvements were made to the colorizer.

The 'with' statement is now a Code Context block opener.

IDLE honors the new quit() and exit() functions.

Keybindings were added for del-word-left and del-word-right.

%============================================================

-- 
KBK

From greg.ewing at canterbury.ac.nz  Thu Aug 17 03:25:46 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Aug 2006 13:25:46 +1200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E2BAFD.50105@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
	<44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de>
Message-ID: <44E3C59A.1020208@canterbury.ac.nz>

Martin v. L?wis wrote:
> Greg Ewing schrieb:
>>Also it means you'd pay a penalty every time you
>>access it
> 
> That penalty is already paid today.

You'd still have that penalty, *plus* the
overhead of bit masking to get at the value.

Maybe that wouldn't be noticeable among all
the other overheads, though.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Aug 17 03:30:17 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 17 Aug 2006 13:30:17 +1200
Subject: [Python-Dev] Making 'python -t' the default.
In-Reply-To: <ebv2hb$9cc$1@sea.gmane.org>
References: <9e804ac0608160437h619af70ck8952850fa6330ae8@mail.gmail.com>
	<ebv2hb$9cc$1@sea.gmane.org>
Message-ID: <44E3C6A9.3000200@canterbury.ac.nz>

Georg Brandl wrote:

> Or disallow tabs altogether.

-1. I'd be annoyed if Python started telling me I wasn't
allowed to write my source the way my preferred editor
(BBEdit) works best. Very annoyed.

--
Greg


From jjl at pobox.com  Thu Aug 17 03:58:22 2006
From: jjl at pobox.com (John J Lee)
Date: Thu, 17 Aug 2006 01:58:22 +0000 (UTC)
Subject: [Python-Dev] 2.5: recently introduced sgmllib regexp bug hangs
	Python
Message-ID: <Pine.LNX.4.64.0608170108310.8544@localhost>

Looks like revision 47154 introduced a regexp that hangs Python (Ctrl-C 
won't kill the process, CPU usage sits near 100%) under some 
circumstances.  There's a test case here:

http://python.org/sf/1541697


The problem isn't seen if you read the whole file at once (or almost the 
whole file at once).  (But that doesn't make it a non-bug, AFAICS.)

I'm not sure what the problem is, but presumably the relevant part of the 
patch is this:

+starttag = re.compile(r'<[a-zA-Z][-_.:a-zA-Z0-9]*\s*('
+        r'\s*([a-zA-Z_][-:.a-zA-Z_0-9]*)(\s*=\s*'
+        r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~@]'
+        r'[][\-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~\'"@]*(?=[\s>/<])))?'
+    r')*\s*/?\s*(?=[<>])')


The patch attached to bug 1515142 (also from Sam Ruby -- claims to fix a 
regression introduced by his recent sgmllib patches, and has not yet been 
applied) does NOT fix the problem.

If nobody has time to fix this, perhaps rev 47154 should be reverted?


commit message for -r47154:

"""
SF bug #1504333: sgmlib should allow angle brackets in quoted values
(modified patch by Sam Ruby; changed to use separate REs for start and end
  tags to reduce matching cost for end tags; extended tests; updated to 
avoid
  breaking previous changes to support IPv6 addresses in unquoted attribute
  values)
"""


John


From jjl at pobox.com  Thu Aug 17 04:13:06 2006
From: jjl at pobox.com (John J Lee)
Date: Thu, 17 Aug 2006 02:13:06 +0000 (UTC)
Subject: [Python-Dev] [wwwsearch-general] 2.5: recently introduced
 sgmllib regexp bug hangs Python
In-Reply-To: <Pine.LNX.4.64.0608170108310.8544@localhost>
References: <Pine.LNX.4.64.0608170108310.8544@localhost>
Message-ID: <Pine.LNX.4.64.0608170208500.8544@localhost>

On Thu, 17 Aug 2006, John J Lee wrote:
[...]
> If nobody has time to fix this, perhaps rev 47154 should be reverted?

I should have put it more strongly: I think it *should* in fact be 
reverted if nobody has time to fix it before the release candidate / final 
release of 2.5.  The revision in question is the most recent commit to 
sgmllib.py and certainly appears completely localised to that module. 
And a hung interpreter is worse than failing to parse some HTML, IMHO.


[...]
> commit message for -r47154:
>
> """
> SF bug #1504333: sgmlib should allow angle brackets in quoted values
> (modified patch by Sam Ruby; changed to use separate REs for start and end
>  tags to reduce matching cost for end tags; extended tests; updated to
> avoid
>  breaking previous changes to support IPv6 addresses in unquoted attribute
>  values)
> """
[...]


From nnorwitz at gmail.com  Thu Aug 17 07:26:09 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 16 Aug 2006 22:26:09 -0700
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
Message-ID: <ee2a432c0608162226m436760f3le4533db33193ed1d@mail.gmail.com>

Thanks Dino.

The attached patch should fix the problem.  Once RC1 is cut, I'll
check this in unless someone beats me to it.  Since the compiler
changed, I can't backport this.  If someone wants to make a similar
fix for 2.4 go for it.

n
--

On 8/16/06, Dino Viehland <dinov at exchange.microsoft.com> wrote:
> We've been working on fixing some exception handling bugs in IronPython where we differ from CPython.  Along the way we ran into this issue which causes CPython to crash when the code below is run.  It crashes on both 2.4 and 2.5 beta 3.  The code's technically illegal, but it probably shouldn't crash either :)
>
> def test():
>     for abc in range(10):
>         try: pass
>         finally:
>             try:
>                 continue
>             except:
>                 pass
>
>
> test()
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: finally-cont.diff
Type: text/x-patch
Size: 1658 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060816/a9670b69/attachment.bin 

From foom at fuhm.net  Thu Aug 17 07:49:04 2006
From: foom at fuhm.net (James Y Knight)
Date: Thu, 17 Aug 2006 01:49:04 -0400
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <ee2a432c0608162226m436760f3le4533db33193ed1d@mail.gmail.com>
References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0608162226m436760f3le4533db33193ed1d@mail.gmail.com>
Message-ID: <BF64882A-72F5-498B-AF87-C7E458C014C9@fuhm.net>

On Aug 17, 2006, at 1:26 AM, Neal Norwitz wrote:
> Thanks Dino.
>
> The attached patch should fix the problem.  Once RC1 is cut, I'll
> check this in unless someone beats me to it.  Since the compiler
> changed, I can't backport this.  If someone wants to make a similar
> fix for 2.4 go for it.

The attached patch is incorrect. It breaks the following perfectly  
valid code. Probably what would be right is to move (with appropriate  
changes) the while loop in the FINALLY_TRY case to outside the entire  
switch statement. Thus, if LOOP is reached, yay, add the jump, return  
success. If the FINALLY_END is reached first, boo, fail. If it's a  
FINALLY_TRY or EXCEPT, continue the loop.


def test():
     try: pass
     finally:
         for abc in range(10):
             continue

James


From nnorwitz at gmail.com  Thu Aug 17 07:57:54 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 16 Aug 2006 22:57:54 -0700
Subject: [Python-Dev] Type of range object members
In-Reply-To: <5.1.1.6.0.20060816142136.022968e8@sparrow.telecommunity.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>
	<44E1EFEF.20305@v.loewis.de>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>
	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>
	<44E27939.9060505@canterbury.ac.nz> <44E2BAFD.50105@v.loewis.de>
	<5.1.1.6.0.20060816142136.022968e8@sparrow.telecommunity.com>
Message-ID: <ee2a432c0608162257j382354a3jeb615ce20b28c99a@mail.gmail.com>

On 8/16/06, Phillip J. Eby <pje at telecommunity.com> wrote:
>
> It seems to me that you could drop the FAST_SUBCLASS bit, since none of the
> other bits will be set if it is not a subclass of a builtin.  That would
> free up one flag bit -- perhaps usable for that BaseException flag Guido
> wants.  :)

:-)  Right, I'm not using the bit currently.  I was thinking that it
would be interesting to change the CheckExact versions to also use
this.  It's a little more work, but you lose the second comparison for
Check.  I expect that it would be slower, but I was just curious.

So with the patch we currently have:

#define PyInt_CheckExact(op) ((op)->ob_type == &PyInt_Type)
#define PyInt_Check(op) (PyInt_CheckExact(op) || \
                 PyType_FastSubclass((op)->ob_type, Py_TPFLAGS_INT_SUBCLASS))

But we could have something like:

#define PyInt_CheckExact(op) (PyType_FastClass(op,Py_TPFLAGS_INT_CLASS))
#define PyInt_Check(op) (PyType_FastSubclass(op,Py_TPFLAGS_INT_SUBCLASS))

It would change the CheckExact()s from: op->ob_type ==
global-variable, to: op->ob_type & CONSTANT == CONSTANT.  Check would
be the same as the CheckExact, just with different constants.  The
Check version would then drop the || condition.

I might play with this at the sprint next week.  It does seem to make
sense to do BaseException too.  It will take 4 or 5 bits to handle the
current ones plus BaseException, which we can easily spare in
tp_flags.

n

From martin at v.loewis.de  Thu Aug 17 08:01:58 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 17 Aug 2006 08:01:58 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <44E3C59A.1020208@canterbury.ac.nz>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>	<44E1EFEF.20305@v.loewis.de>	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>	<44E248A7.4080107@v.loewis.de>	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>	<44E27939.9060505@canterbury.ac.nz>
	<44E2BAFD.50105@v.loewis.de> <44E3C59A.1020208@canterbury.ac.nz>
Message-ID: <44E40656.9050501@v.loewis.de>

Greg Ewing schrieb:
>>> Also it means you'd pay a penalty every time you
>>> access it
>> That penalty is already paid today.
> 
> You'd still have that penalty, *plus* the
> overhead of bit masking to get at the value.

No, the penalty gets smaller if there is only a single type.
For example, abstract.c now has

                if (res && (!PyInt_Check(res) && !PyLong_Check(res))) {
                        PyErr_Format(PyExc_TypeError,
                                     "__int__ returned non-int (type
%.200s)",
                                     res->ob_type->tp_name);
                        Py_DECREF(res);
                        return NULL;
                }

Currently, if a long int is returned, it performs two subtype tests.
If the long type is dropped, the second test can go away. In this
specific code, there is no penalty for a representation flag, since
the value is not accessed.

Code that wants to support both int and long and needs the value
often does PyLong_AsLong these days, which will support int as well.
This currently reads

        if (vv == NULL || !PyLong_Check(vv)) {
                if (vv != NULL && PyInt_Check(vv))
                        return PyInt_AsLong(vv);
                PyErr_BadInternalCall();
                return -1;
        }

Notice that this has two checks if this is an int, and both are subtype
checks. With a single type, this would become

        if (vv == NULL || !PyInt_Check(vv)) {
                PyErr_BadInternalCall();
                return -1;
        }
        if (!vv->ob_size)
                return PyInt_AsLong(vv);

Actually, the implementation of PyInt_AsLong might get inlined;
it currently starts with a third PyInt_Check.

So overall, I would expect that a single type would improve performance,
not decrease it.

As you say, any change is likely not noticeable in performance, though,
either way.

Regards,
Martin

From martin at v.loewis.de  Thu Aug 17 08:27:02 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 17 Aug 2006 08:27:02 +0200
Subject: [Python-Dev] Type of range object members
In-Reply-To: <ee2a432c0608162257j382354a3jeb615ce20b28c99a@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>	<ca471dc20608150827n63c0fe3cp7fdbc05aa877f80@mail.gmail.com>	<44E1EFEF.20305@v.loewis.de>	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>	<44E248A7.4080107@v.loewis.de>	<7A66B0E3-6BA6-4EE8-81AA-BA564CEC361C@fuhm.net>	<ca471dc20608151606u507a1168m5c81bc9338b7ccdb@mail.gmail.com>	<44E27939.9060505@canterbury.ac.nz>
	<44E2BAFD.50105@v.loewis.de>	<5.1.1.6.0.20060816142136.022968e8@sparrow.telecommunity.com>
	<ee2a432c0608162257j382354a3jeb615ce20b28c99a@mail.gmail.com>
Message-ID: <44E40C36.4000603@v.loewis.de>

Neal Norwitz schrieb:
> It would change the CheckExact()s from: op->ob_type ==
> global-variable, to: op->ob_type & CONSTANT == CONSTANT.  Check would
> be the same as the CheckExact, just with different constants.  The
> Check version would then drop the || condition.

Hmm. I don't see the for the FAST_SUBCLASS bit still.

I would set the relevant bit in the type object itself, and then have

#define PyInt_CheckExact(op) ((op)->ob_type == &PyInt_Type)
#define PyInt_Check(op) \
        PyType_FastSubclass((op)->ob_type, Py_TPFLAGS_INT_SUBCLASS)

Then, in inherit_special, I'd do

       type->tp_flags |= base->tp_flags & Py_TPFLAGS_FAST_SUBCLASS_MASK;

So you would have a pointer comparison for the exact check, and the
bit mask check for the subtype check.

It's likely that the pointer comparison is still more efficient: It
does *not*, normally, need to read a global variable to get the address
of PyInt_Type.

Currently, on x86, with non-PIC code on Linux, the pointer check compiles as

        cmpl    $PyInt_Type, 4(%eax)   ; %eax is the object

where the linker fills the address of PyInt_Type into the machine
instruction. OTOH, the access to the flags compiles as

        movl    4(%eax), %eax          ; %eax is the object
        movl    84(%eax), %eax
        andl    $2013265920, %eax
        cmpl    $2013265920, %eax

Even with PIC code, the address check is still more efficient:

        movl    PyInt_Type at GOT(%ecx), %eax
        cmpl    %eax, 4(%edx)          ; %edx is the object


Regards,
Martin

From thomas at python.org  Thu Aug 17 13:22:53 2006
From: thomas at python.org (Thomas Wouters)
Date: Thu, 17 Aug 2006 13:22:53 +0200
Subject: [Python-Dev] Benchmarking the int allocator (Was: Type of range
	object members)
In-Reply-To: <44E3776D.90804@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<ca471dc20608150910q17fe26b1o9fa33a9d70242c48@mail.gmail.com>
	<44E248A7.4080107@v.loewis.de>
	<ca471dc20608151529s5a7f3db2g63f35f63763563e1@mail.gmail.com>
	<44E27820.40205@canterbury.ac.nz> <44E2BA5C.7010405@v.loewis.de>
	<17635.9587.937810.163966@montanaro.dyndns.org>
	<44E342ED.6060507@v.loewis.de>
	<ca471dc20608161008x37c7f948o6530076a786e76a2@mail.gmail.com>
	<44E3776D.90804@v.loewis.de>
Message-ID: <9e804ac0608170422w5a7fbbfelce61e5ba74c85971@mail.gmail.com>

On 8/16/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> Guido van Rossum schrieb:
> > I think the test isn't hardly focused enough on int allocation. I
> > wonder if you could come up with a benchmark that repeatedly allocates
> > 100s of 1000s of ints and then deletes them?
>
> The question is: where to store them? In a pre-allocated list, or in a
> growing list?


Or you can expose Py_INCREF to Python code. I'm thinking about that for
big-mem tests anyway ;-)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060817/ada243d2/attachment.htm 

From fredrik at pythonware.com  Thu Aug 17 14:35:33 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 17 Aug 2006 14:35:33 +0200
Subject: [Python-Dev] recently introduced sgmllib regexp bughangs Python
References: <Pine.LNX.4.64.0608170108310.8544@localhost>
Message-ID: <ec1nql$bq6$1@sea.gmane.org>

John J Lee wrote:

> If nobody has time to fix this, perhaps rev 47154 should be reverted?

yes please.

</F> 




From anthony at interlink.com.au  Thu Aug 17 14:48:47 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu, 17 Aug 2006 22:48:47 +1000
Subject: [Python-Dev] [wwwsearch-general] 2.5: recently introduced
	sgmllib regexp bug hangs Python
In-Reply-To: <Pine.LNX.4.64.0608170208500.8544@localhost>
References: <Pine.LNX.4.64.0608170108310.8544@localhost>
	<Pine.LNX.4.64.0608170208500.8544@localhost>
Message-ID: <200608172248.49046.anthony@interlink.com.au>

On Thursday 17 August 2006 12:13, John J Lee wrote:
> On Thu, 17 Aug 2006, John J Lee wrote:
> [...]
>
> > If nobody has time to fix this, perhaps rev 47154 should be reverted?
>
> I should have put it more strongly: I think it *should* in fact be
> reverted if nobody has time to fix it before the release candidate / final
> release of 2.5.  The revision in question is the most recent commit to
> sgmllib.py and certainly appears completely localised to that module.
> And a hung interpreter is worse than failing to parse some HTML, IMHO.

Well, it's a bit late for the release candidate :-) but I agree - if it can't 
be fixed before 2.5 final, it should be reverted.

Anthony

-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From benji at zope.com  Thu Aug 17 15:50:54 2006
From: benji at zope.com (Benji York)
Date: Thu, 17 Aug 2006 09:50:54 -0400
Subject: [Python-Dev] [wwwsearch-general] 2.5: recently introduced
 sgmllib regexp bug hangs Python
In-Reply-To: <Pine.LNX.4.64.0608170108310.8544@localhost>
References: <Pine.LNX.4.64.0608170108310.8544@localhost>
Message-ID: <44E4743E.40807@zope.com>

John J Lee wrote:
> Looks like revision 47154 introduced a regexp that hangs Python

Revision 47155 was a similar change to the 2.4 branch.
-- 
Benji York
Senior Software Engineer
Zope Corporation

From anthony at python.org  Thu Aug 17 16:17:38 2006
From: anthony at python.org (Anthony Baxter)
Date: Fri, 18 Aug 2006 00:17:38 +1000
Subject: [Python-Dev] RELEASED Python 2.5 (release candidate 1)
Message-ID: <200608180017.52057.anthony@python.org>

On behalf of the Python development team and the Python
community, I'm happy to announce the first RELEASE CANDIDATE
of Python 2.5.

This is not yet the final release - it is not suitable for
production use. It is being released to solicit feedback
and hopefully expose bugs, as well as allowing you to
determine how changes in 2.5 might impact you. As a release
candidate, this is one of your last chances to test the new
code in 2.5 before the final release. *Please* try this
release out and let us know about any problems you find.

In particular, note that changes to improve Python's support
of 64 bit systems might require authors of C extensions
to change their code. More information (as well as source
distributions and Windows and Universal Mac OSX installers)
are available from the 2.5 website:

    http://www.python.org/2.5/

As of this release, Python 2.5 is now in *feature freeze*.
Unless absolutely necessary, no functionality changes will
be made between now and the final release of Python 2.5.

The plan now is to let the release candidate shake out any
last-minute bugs in Python 2.5, leading to a 2.5 final
release in early September. PEP 356 includes the schedule
and will be updated as the schedule evolves. At this
point, any testing you can do would be greatly, greatly
appreciated.

The new features in Python 2.5 are described in Andrew
Kuchling's What's New In Python 2.5. It's available from the
2.5 web page.

Amongst the language features added include conditional
expressions, the with statement, the merge of try/except
and try/finally into try/except/finally, enhancements to
generators to produce a coroutine kind of functionality, and
a brand new AST-based compiler implementation.

New modules added include hashlib, ElementTree, sqlite3,
wsgiref, uuid and ctypes. In addition, a new profiling
module cProfile was added.

Enjoy this new release,
Anthony

Anthony Baxter
anthony at python.org
Python Release Manager
(on behalf of the entire python-dev team)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060818/a331c29d/attachment.pgp 

From anthony at interlink.com.au  Thu Aug 17 16:23:13 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri, 18 Aug 2006 00:23:13 +1000
Subject: [Python-Dev] TRUNK IS UNFROZEN,
	available for 2.6 work if you are so inclined
Message-ID: <200608180023.14037.anthony@interlink.com.au>

The release candidate is done - we creep ever closer to 2.5 final. Hoooray!

All future 2.5 releases (including 2.5 final!) will now be done from the new 
release25-maint trunk 
(svn+ssh://pythondev at svn.python.org/python/branches/release25-maint) - so any 
changes you want to see after 2.5c1 and before 2.5 final will need to be 
applied to that branch as well as to the trunk. 

From now until the final release, I think we should say no checkins to the 
release25-maint branch without either myself or Neal signing off on it - for 
safety's sake, I'd recommend you email either the list here, or if you have 
to send private email, send it to both of us. If this policy offends you, 
please reply and let me know what you'd prefer. 

Right now, I don't really care about the trunk - although if you break the 
buildbot, I'm sure Neal will be very cranky at you :-)

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.


From martin at v.loewis.de  Thu Aug 17 16:47:45 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 17 Aug 2006 16:47:45 +0200
Subject: [Python-Dev] [Python-checkins] TRUNK IS UNFROZEN,
 available for 2.6 work if you are so inclined
In-Reply-To: <200608180023.14037.anthony@interlink.com.au>
References: <200608180023.14037.anthony@interlink.com.au>
Message-ID: <44E48191.6020008@v.loewis.de>

Anthony Baxter schrieb:
> Right now, I don't really care about the trunk - although if you break the 
> buildbot [...]

Thanks for reminding me. I just added a buildbot builder for the 2.5
branch (actually, the builder was already there, but I connected it
with the web server), so you can now see the status of the 2.5 branch
at

http://www.python.org/dev/buildbot/2.5/

Regards,
Martin

From anthony at interlink.com.au  Thu Aug 17 16:50:51 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri, 18 Aug 2006 00:50:51 +1000
Subject: [Python-Dev] [Python-checkins] TRUNK IS UNFROZEN,
	available for 2.6 work if you are so inclined
In-Reply-To: <44E48191.6020008@v.loewis.de>
References: <200608180023.14037.anthony@interlink.com.au>
	<44E48191.6020008@v.loewis.de>
Message-ID: <200608180050.55389.anthony@interlink.com.au>

On Friday 18 August 2006 00:47, Martin v. L?wis wrote:
> Anthony Baxter schrieb:
> > Right now, I don't really care about the trunk - although if you break
> > the buildbot [...]
>
> Thanks for reminding me. I just added a buildbot builder for the 2.5
> branch (actually, the builder was already there, but I connected it
> with the web server), so you can now see the status of the 2.5 branch
> at
>
> http://www.python.org/dev/buildbot/2.5/

You beat me to it by a couple of minutes - I'd added the entry to the 
master.cfg file but hadn't updated the apache config yet.

Anthoiny

-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From martin at v.loewis.de  Thu Aug 17 19:03:05 2006
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 17 Aug 2006 19:03:05 +0200
Subject: [Python-Dev] Fixing 2.5 windows buildbots
Message-ID: <44E4A149.7060409@v.loewis.de>

I'd like to fix the two build failures that the Windows buildbots
show for the 2.5 trunk. I'm not quite sure what the OpenSSL failure
is, yet, but the sqlite error should be fixable with the patch
below. Ok to work on this?

Martin

Index: Tools/buildbot/external.bat
===================================================================
--- Tools/buildbot/external.bat (Revision 51339)
+++ Tools/buildbot/external.bat (Arbeitskopie)
@@ -28,6 +28,7 @@
    cd tk8.4.12\win
    nmake -f makefile.vc TCLDIR=..\..\tcl8.4.12
    nmake -f makefile.vc TCLDIR=..\..\tcl8.4.12 INSTALLDIR=..\..\tcltk
install
+   cd ..\..
 )

 @rem sqlite

From nnorwitz at gmail.com  Thu Aug 17 19:12:46 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 17 Aug 2006 10:12:46 -0700
Subject: [Python-Dev] Fixing 2.5 windows buildbots
In-Reply-To: <44E4A149.7060409@v.loewis.de>
References: <44E4A149.7060409@v.loewis.de>
Message-ID: <ee2a432c0608171012m6a2b960dy79c43da7aea919ca@mail.gmail.com>

On 8/17/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I'd like to fix the two build failures that the Windows buildbots
> show for the 2.5 trunk. I'm not quite sure what the OpenSSL failure
> is, yet, but the sqlite error should be fixable with the patch
> below. Ok to work on this?

Please do.

From martin at v.loewis.de  Thu Aug 17 21:52:54 2006
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 17 Aug 2006 21:52:54 +0200
Subject: [Python-Dev] uuid tests failing on Windows
Message-ID: <44E4C916.9040008@v.loewis.de>

test_uuid1 and test_uuid4 fail consistently on Windows; apparently,
the generated universally-unique identifiers aren't even unique within
a set of 1000 identifiers.

Can somebody please fix that? If not, should we remove the uuid module
as being immature?

Regards,
Martin

From g.brandl at gmx.net  Thu Aug 17 21:56:49 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 17 Aug 2006 21:56:49 +0200
Subject: [Python-Dev] uuid tests failing on Windows
In-Reply-To: <44E4C916.9040008@v.loewis.de>
References: <44E4C916.9040008@v.loewis.de>
Message-ID: <ec2hm1$9uf$1@sea.gmane.org>

Martin v. L?wis wrote:
> test_uuid1 and test_uuid4 fail consistently on Windows; apparently,
> the generated universally-unique identifiers aren't even unique within
> a set of 1000 identifiers.
> 
> Can somebody please fix that? If not, should we remove the uuid module
> as being immature?

Patch #1541863 supposedly solves this.

Georg


From jackdied at jackdied.com  Thu Aug 17 22:00:46 2006
From: jackdied at jackdied.com (Jack Diederich)
Date: Thu, 17 Aug 2006 16:00:46 -0400
Subject: [Python-Dev] [Python-checkins] TRUNK IS UNFROZEN,
	available for 2.6 work if you are so inclined
In-Reply-To: <ec2eqa$ho$1@sea.gmane.org>
References: <200608180023.14037.anthony@interlink.com.au>
	<ec1v5d$6u2$1@sea.gmane.org>
	<20060817185049.GH5772@performancedrivers.com>
	<ec2eqa$ho$1@sea.gmane.org>
Message-ID: <20060817200046.GI5772@performancedrivers.com>

On Thu, Aug 17, 2006 at 09:07:53PM +0200, Georg Brandl wrote:
> Jack Diederich wrote:
> 
> > Looks good to me.  While you are on that page do you want to change
> > 
> > l = PyList_New(3);
> > x = PyInt_FromLong(1L);
> > PySequence_SetItem(l, 0, x); Py_DECREF(x);
> > x = PyInt_FromLong(2L);
> > PySequence_SetItem(l, 1, x); Py_DECREF(x);
> > x = PyString_FromString("three");
> > PySequence_SetItem(l, 2, x); Py_DECREF(x);
> > 
> > to
> > 
> > l = PyList_New(3);
> > x = PyInt_FromLong(1L);
> > PyList_SetItem(l, 0, x);
> > x = PyInt_FromLong(2L);
> > PyList_SetItem(l, 1, x);
> > x = PyString_FromString("three");
> > PyList_SetItem(l, 2, x);
> > 
> > The example code causes segfaults and probably always has (at last to 2.2)
> 
> Interesting! From a naive POV, the docs' example is quite right.
> 
> The segfault occurs because list_ass_item Py_DECREFs the old item (which
> segfaults because the old items are NULL in a newly created list).
> PyList_SetItem does a Py_XDECREF.
> 
> The docs to PyList_New, however, do not explicitly say that the new list
> must only be filled using PyList_SetItem.
> 
> So please, someone, decide what's broken here!

The docs, this is from a thread yesterday and today on c.l.py

http://groups.google.com/group/comp.lang.python/browse_frm/thread/158c8797ee2dccab/

-Jack

From martin at v.loewis.de  Thu Aug 17 22:06:24 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 17 Aug 2006 22:06:24 +0200
Subject: [Python-Dev] uuid tests failing on Windows
In-Reply-To: <ec2hm1$9uf$1@sea.gmane.org>
References: <44E4C916.9040008@v.loewis.de> <ec2hm1$9uf$1@sea.gmane.org>
Message-ID: <44E4CC40.7030105@v.loewis.de>

Georg Brandl schrieb:
>> Can somebody please fix that? If not, should we remove the uuid module
>> as being immature?
> 
> Patch #1541863 supposedly solves this.

Ah, good. I think it should go in.

Regards,
Martin

From glyph at divmod.com  Thu Aug 17 22:53:13 2006
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 17 Aug 2006 16:53:13 -0400
Subject: [Python-Dev] uuid tests failing on Windows
In-Reply-To: <44E4CC40.7030105@v.loewis.de>
Message-ID: <20060817205313.1717.1697957210.divmod.quotient.24160@ohm>

On Thu, 17 Aug 2006 22:06:24 +0200, "\"Martin v. L?wis\"" <martin at v.loewis.de> wrote:
>Georg Brandl schrieb:
>>> Can somebody please fix that? If not, should we remove the uuid module
>>> as being immature?
>>
>> Patch #1541863 supposedly solves this.
>
>Ah, good. I think it should go in.

Uh, I may be misunderstanding here, but that patch looks like it changes that part of the test for test_uuid4 to stop calling uuid4 and call uuid1 instead?

From fredrik at pythonware.com  Thu Aug 17 23:03:08 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 17 Aug 2006 23:03:08 +0200
Subject: [Python-Dev] TRUNK IS UNFROZEN,
	available for 2.6 work if you are so inclined
In-Reply-To: <ec2eqa$ho$1@sea.gmane.org>
References: <200608180023.14037.anthony@interlink.com.au>	<ec1v5d$6u2$1@sea.gmane.org>	<20060817185049.GH5772@performancedrivers.com>
	<ec2eqa$ho$1@sea.gmane.org>
Message-ID: <ec2lia$nid$1@sea.gmane.org>

Georg Brandl wrote:

>> The example code causes segfaults and probably always has (at last to 2.2)
> 
> Interesting! From a naive POV, the docs' example is quite right.

except that it doesn't work?

the problem is that it's using a *high-level* API to manipulate objects 
that are not properly initialized.

the documentation should be fixed (doctest for the C API, anyone?)

</F>


From fredrik at pythonware.com  Thu Aug 17 23:05:00 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 17 Aug 2006 23:05:00 +0200
Subject: [Python-Dev] TRUNK IS UNFROZEN,
	available for 2.6 work if you are so inclined
In-Reply-To: <ec2hag$8kr$1@sea.gmane.org>
References: <200608180023.14037.anthony@interlink.com.au>	<ec1v5d$6u2$1@sea.gmane.org>	<20060817185049.GH5772@performancedrivers.com>	<ec2eqa$ho$1@sea.gmane.org>
	<ec2hag$8kr$1@sea.gmane.org>
Message-ID: <ec2llq$nid$2@sea.gmane.org>

Georg Brandl wrote:

> Okay, now that I stumbled over and read the c.l.py thread, I think that we should
> 
> * remove the faulty example (the correct one is already in there)
> * add a note to PyList_New that the new list must be filled with items
>    before handing it to Python code or using it with abstract APIs such as
>    PySequence_SetItem.

a blurb at the beginning of the abstract API section that mentions that 
the abstract API should only be used on fully initialized Python objects 
might also be a good idea.

</F>


From nnorwitz at gmail.com  Thu Aug 17 23:15:47 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 17 Aug 2006 14:15:47 -0700
Subject: [Python-Dev] Sprints next week at Google
Message-ID: <ee2a432c0608171415h54cb6879v9418348d24e93d1d@mail.gmail.com>

This is a reminder (I don't think anyone else sent one, I sure hope not).

We are holding 4 days of sprints next week at Google offices in NY
city and Mt View, CA.  These are open if you'd like to attend.  It
would be very helpful to pre-register on the wiki as we can notify
security and generally make things easier.  Of the full-time Googlers,
Guido and Alex will be sprinting in CA and Jeremy and I will be
sprinting in NY.

    http://wiki.python.org/moin/GoogleSprint

Cheers,
n

From g.brandl at gmx.net  Thu Aug 17 23:30:59 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 17 Aug 2006 23:30:59 +0200
Subject: [Python-Dev] TRUNK IS UNFROZEN,
	available for 2.6 work if you are so inclined
In-Reply-To: <ec2lia$nid$1@sea.gmane.org>
References: <200608180023.14037.anthony@interlink.com.au>	<ec1v5d$6u2$1@sea.gmane.org>	<20060817185049.GH5772@performancedrivers.com>	<ec2eqa$ho$1@sea.gmane.org>
	<ec2lia$nid$1@sea.gmane.org>
Message-ID: <ec2n6k$sam$1@sea.gmane.org>

Fredrik Lundh wrote:
> Georg Brandl wrote:
> 
>>> The example code causes segfaults and probably always has (at last to 2.2)
>> 
>> Interesting! From a naive POV, the docs' example is quite right.
> 
> except that it doesn't work?

I wanted to say: From looking at the docs only, it should be alright.

> the problem is that it's using a *high-level* API to manipulate objects 
> that are not properly initialized.

The docs for PyList_New don't tell the user that the list object isn't
properly initialized, etc.

> the documentation should be fixed

Clearly. I'll do that (if the release managers give permission)

Georg


From martin at v.loewis.de  Thu Aug 17 23:58:27 2006
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 17 Aug 2006 23:58:27 +0200
Subject: [Python-Dev] uuid tests failing on Windows
In-Reply-To: <20060817205313.1717.1697957210.divmod.quotient.24160@ohm>
References: <20060817205313.1717.1697957210.divmod.quotient.24160@ohm>
Message-ID: <44E4E683.5090209@v.loewis.de>

glyph at divmod.com schrieb:
>>> Patch #1541863 supposedly solves this.
>> Ah, good. I think it should go in.
> 
> Uh, I may be misunderstanding here, but that patch looks like it
> changes that part of the test for test_uuid4 to stop calling uuid4
> and call uuid1 instead?

You misunderstand indeed: the chunk reads

-        for u in [uuid.uuid1() for i in range(1000)]:
+        for u in [uuid.uuid4() for i in range(1000)]:

so it currently calls uuid1, and will call uuid4 when patched.
test_uuid4 should have never failed, except that it uses uuid1
as the uniqueness test.

Regards,
Martin

From glyph at divmod.com  Fri Aug 18 00:22:40 2006
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 17 Aug 2006 18:22:40 -0400
Subject: [Python-Dev] uuid tests failing on Windows
In-Reply-To: <44E4E683.5090209@v.loewis.de>
Message-ID: <20060817222240.1717.1350793518.divmod.quotient.24211@ohm>

On Thu, 17 Aug 2006 23:58:27 +0200, "\"Martin v. L?wis\"" <martin at v.loewis.de> wrote:

>You misunderstand indeed: the chunk reads (...)

>it currently calls uuid1, and will call uuid4 when patched.
>test_uuid4 should have never failed, except that it uses uuid1
>as the uniqueness test.

Whooops.  I must have hit the "reverse diff" button in Emacs before reading it.

Thanks for the correction.

From arigo at tunes.org  Fri Aug 18 00:26:33 2006
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 18 Aug 2006 00:26:33 +0200
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <ee2a432c0608162226m436760f3le4533db33193ed1d@mail.gmail.com>
References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0608162226m436760f3le4533db33193ed1d@mail.gmail.com>
Message-ID: <20060817222633.GA26304@code0.codespeak.net>

Hi Neal,

On Wed, Aug 16, 2006 at 10:26:09PM -0700, Neal Norwitz wrote:
> @@ -2367,7 +2374,9 @@
>  	compiler_use_next_block(c, end);
>  	if (!compiler_push_fblock(c, FINALLY_END, end))
>  		return 0;
> +	c->c_in_finally = 1;
>  	VISIT_SEQ(c, stmt, s->v.TryFinally.finalbody);
> +	c->c_in_finally = 0;
>  	ADDOP(c, END_FINALLY);
>  	compiler_pop_fblock(c, FINALLY_END, end);

Without more inspection, I'd say that this looks like it won't do the
right thing about nested finally's, as in:

       ...
   finally:
       try:
           ...
       finally:
           ...
       continue


A bientot,

Armin

From guido at python.org  Fri Aug 18 00:44:58 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 17 Aug 2006 15:44:58 -0700
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <20060817222633.GA26304@code0.codespeak.net>
References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0608162226m436760f3le4533db33193ed1d@mail.gmail.com>
	<20060817222633.GA26304@code0.codespeak.net>
Message-ID: <ca471dc20608171544l33133a9bx58a73e60d8c7b0e7@mail.gmail.com>

High-level remark: this seems to need more unit tests. Perhaps TDD
(test-driven design) would be the best approach to obtaining a correct
patch?

--Guido

On 8/17/06, Armin Rigo <arigo at tunes.org> wrote:
> Hi Neal,
>
> On Wed, Aug 16, 2006 at 10:26:09PM -0700, Neal Norwitz wrote:
> > @@ -2367,7 +2374,9 @@
> >       compiler_use_next_block(c, end);
> >       if (!compiler_push_fblock(c, FINALLY_END, end))
> >               return 0;
> > +     c->c_in_finally = 1;
> >       VISIT_SEQ(c, stmt, s->v.TryFinally.finalbody);
> > +     c->c_in_finally = 0;
> >       ADDOP(c, END_FINALLY);
> >       compiler_pop_fblock(c, FINALLY_END, end);
>
> Without more inspection, I'd say that this looks like it won't do the
> right thing about nested finally's, as in:
>
>        ...
>    finally:
>        try:
>            ...
>        finally:
>            ...
>        continue
>
>
> A bientot,
>
> Armin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From amk at amk.ca  Fri Aug 18 02:26:17 2006
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 17 Aug 2006 20:26:17 -0400
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <20060817222633.GA26304@code0.codespeak.net>
References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0608162226m436760f3le4533db33193ed1d@mail.gmail.com>
	<20060817222633.GA26304@code0.codespeak.net>
Message-ID: <20060818002617.GA17684@Andrew-iBook2.local>

On Fri, Aug 18, 2006 at 12:26:33AM +0200, Armin Rigo wrote:
> Without more inspection, I'd say that this looks like it won't do the
> right thing about nested finally's, as in:

I guess it'll have to loop back up through the block stack:

for (j=i- 1; j>=0; j--) {
   switch (c->u->u_fblock[j].fb_type) {
        case FINALLY_TRY:
	   raise error;
	case LOOP:
	   j=0;  /* Exit the loop */
           break;
        default:
	   break;
  }
}

--amk

From anthony at interlink.com.au  Fri Aug 18 03:04:09 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri, 18 Aug 2006 11:04:09 +1000
Subject: [Python-Dev] TRUNK IS UNFROZEN,
	available for 2.6 work if you are so inclined
In-Reply-To: <ec2n6k$sam$1@sea.gmane.org>
References: <200608180023.14037.anthony@interlink.com.au>
	<ec2lia$nid$1@sea.gmane.org> <ec2n6k$sam$1@sea.gmane.org>
Message-ID: <200608181104.13441.anthony@interlink.com.au>

Fixing docs is fine - please checkin.

From anthony at interlink.com.au  Fri Aug 18 03:05:36 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri, 18 Aug 2006 11:05:36 +1000
Subject: [Python-Dev] uuid tests failing on Windows
In-Reply-To: <44E4CC40.7030105@v.loewis.de>
References: <44E4C916.9040008@v.loewis.de> <ec2hm1$9uf$1@sea.gmane.org>
	<44E4CC40.7030105@v.loewis.de>
Message-ID: <200608181105.38897.anthony@interlink.com.au>

On Friday 18 August 2006 06:06, Martin v. L?wis wrote:
> Georg Brandl schrieb:
> >> Can somebody please fix that? If not, should we remove the uuid module
> >> as being immature?
> >
> > Patch #1541863 supposedly solves this.
>
> Ah, good. I think it should go in.

Sounds fine to me. Making buildbot less red == goodness.


-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From nnorwitz at gmail.com  Fri Aug 18 09:29:00 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 18 Aug 2006 00:29:00 -0700
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <20060818002617.GA17684@Andrew-iBook2.local>
References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<ee2a432c0608162226m436760f3le4533db33193ed1d@mail.gmail.com>
	<20060817222633.GA26304@code0.codespeak.net>
	<20060818002617.GA17684@Andrew-iBook2.local>
Message-ID: <ee2a432c0608180029v700e3497w225a8fff8251a3a4@mail.gmail.com>

I did something similar to what Andrew suggested.

http://python.org/sf/1542451

Could everyone interested take a look and offer more test cases?

n
--
On 8/17/06, A.M. Kuchling <amk at amk.ca> wrote:
> On Fri, Aug 18, 2006 at 12:26:33AM +0200, Armin Rigo wrote:
> > Without more inspection, I'd say that this looks like it won't do the
> > right thing about nested finally's, as in:
>
> I guess it'll have to loop back up through the block stack:
>
> for (j=i- 1; j>=0; j--) {
>    switch (c->u->u_fblock[j].fb_type) {
>         case FINALLY_TRY:
>            raise error;
>         case LOOP:
>            j=0;  /* Exit the loop */
>            break;
>         default:
>            break;
>   }
> }
>
> --amk
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>

From duncan.booth at suttoncourtenay.org.uk  Fri Aug 18 10:41:19 2006
From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth)
Date: Fri, 18 Aug 2006 09:41:19 +0100
Subject: [Python-Dev] __del__ unexpectedly being called twice
Message-ID: <n2m-g.Xns9823615FCACE1duncanrcpcouk@127.0.0.1>

There's a thread on comp.lang.python at the moment under the subject "It is 
__del__ calling twice for some instances?" which seems to show that when 
releasing a long chain of old-style classes every 50th approximately has 
its finaliser called twice. I've verified that this happens on both Python 
1.4 and 1.5.


My guess is that there's a bug in the trashcan mechanism: calling the 
__del__ method means creating a descriptor, and if that descriptor gets 
queued in the trashcan then releasing it calls the __del__ method a second 
time. I'm not sure if there is going to be a particularly easy fix for 
that.

Would someone who knows this code (instance_dealloc in classobject.c) like 
to have a look at it, should I just submit a bug report, or isn't it worth 
bothering about?


The code which exhibits the problem:

#!/usr/local/bin/python -d
# -*- coding: koi8-u -*-
import sys

class foo:
    def __init__(self, other):
        self.other = other
        self._deleted = False

        global ini_cnt
        ini_cnt +=1

    def __del__(self):
        if self._deleted:
            print "aargh!"
        self._deleted = True
        global del_cnt
        del_cnt +=1
        print "del",del_cnt,"at",id(self)

def stat():
    print "-"*20
    print "ini_cnt = %d" % ini_cnt
    print "del_cnt = %d" % del_cnt
    print "difference = %d" % (ini_cnt-del_cnt)

ini_cnt = 0
del_cnt = 0
loop_cnt = 55

a = foo(None)

for i in xrange(loop_cnt):
    a = foo(a)

stat()
a = None
stat()


The original thread is at:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/293acf433a39583b/bfd4af9c6008a34e


From g.brandl at gmx.net  Fri Aug 18 10:48:48 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 18 Aug 2006 10:48:48 +0200
Subject: [Python-Dev] 2.4 & 2.5 beta 3 crash
In-Reply-To: <ee2a432c0608180029v700e3497w225a8fff8251a3a4@mail.gmail.com>
References: <7AD436E4270DD54A94238001769C2227372749F65E@DF-GRTDANE-MSG.exchange.corp.microsoft.com>	<ee2a432c0608162226m436760f3le4533db33193ed1d@mail.gmail.com>	<20060817222633.GA26304@code0.codespeak.net>	<20060818002617.GA17684@Andrew-iBook2.local>
	<ee2a432c0608180029v700e3497w225a8fff8251a3a4@mail.gmail.com>
Message-ID: <ec3utg$shq$2@sea.gmane.org>

Neal Norwitz wrote:
> I did something similar to what Andrew suggested.
> 
> http://python.org/sf/1542451
> 
> Could everyone interested take a look and offer more test cases?
> 

Note that this needs to be fixed in the compiler package too.

Georg


From tjreedy at udel.edu  Fri Aug 18 13:52:40 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 18 Aug 2006 07:52:40 -0400
Subject: [Python-Dev] __del__ unexpectedly being called twice
References: <n2m-g.Xns9823615FCACE1duncanrcpcouk@127.0.0.1>
Message-ID: <ec49m8$10q$1@sea.gmane.org>


"Duncan Booth" <duncan.booth at suttoncourtenay.org.uk> wrote in message 
news:n2m-g.Xns9823615FCACE1duncanrcpcouk at 127.0.0.1...
> There's a thread on comp.lang.python at the moment under the subject "It 
> is
> __del__ calling twice for some instances?" which seems to show that when
> releasing a long chain of old-style classes every 50th approximately has
> its finaliser called twice. I've verified that this happens on both 
> Python
> 1.4 and 1.5.

Should we assume you meant 2.4 and 2.5?




From duncan.booth at suttoncourtenay.org.uk  Fri Aug 18 14:03:25 2006
From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth)
Date: Fri, 18 Aug 2006 13:03:25 +0100
Subject: [Python-Dev] __del__ unexpectedly being called twice
References: <n2m-g.Xns9823615FCACE1duncanrcpcouk@127.0.0.1>
	<ec49m8$10q$1@sea.gmane.org>
Message-ID: <n2m-g.Xns982384111469Eduncanrcpcouk@127.0.0.1>

"Terry Reedy" <tjreedy at udel.edu> wrote in
news:ec49m8$10q$1 at sea.gmane.org: 

> 
> "Duncan Booth" <duncan.booth at suttoncourtenay.org.uk> wrote in message 
> news:n2m-g.Xns9823615FCACE1duncanrcpcouk at 127.0.0.1...
>> There's a thread on comp.lang.python at the moment under the subject
>> "It is
>> __del__ calling twice for some instances?" which seems to show that
>> when releasing a long chain of old-style classes every 50th
>> approximately has its finaliser called twice. I've verified that this
>> happens on both Python
>> 1.4 and 1.5.
> 
> Should we assume you meant 2.4 and 2.5?
> 
Probably. 2.5b3 to be a bit more precise.


From theller at python.net  Fri Aug 18 20:30:01 2006
From: theller at python.net (Thomas Heller)
Date: Fri, 18 Aug 2006 20:30:01 +0200
Subject: [Python-Dev] ctypes and win64
Message-ID: <ec50v9$oln$1@sea.gmane.org>

The _ctypes extension module does currently not even build on Win64.

I'm (slowly) working on this (for AMD64, not for itanium), but it may
take a good while before it is stable - It is not even fully implemented
currently.

The win64 msi installer installs the ctypes package anyway, but it cannot be
imported.

I suggest that it should be removed from the 2.5 win64 msi installers, so that
at least, when it is ready, can be installed as separate package.

Thanks,
Thomas


From martin at v.loewis.de  Fri Aug 18 20:47:06 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 18 Aug 2006 20:47:06 +0200
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <ec50v9$oln$1@sea.gmane.org>
References: <ec50v9$oln$1@sea.gmane.org>
Message-ID: <44E60B2A.6020101@v.loewis.de>

Thomas Heller schrieb:
> I suggest that it should be removed from the 2.5 win64 msi installers, so that
> at least, when it is ready, can be installed as separate package.

Unfortunately, I won't be able to work on this until the release is
made. Feel free to work on it, but I suggest to just not worry about
it.

There are many other problems with Win64 still, e.g. the test suite
doesn't pass.

Regards,
Martin

From theller at python.net  Fri Aug 18 20:54:56 2006
From: theller at python.net (Thomas Heller)
Date: Fri, 18 Aug 2006 20:54:56 +0200
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <44E60B2A.6020101@v.loewis.de>
References: <ec50v9$oln$1@sea.gmane.org> <44E60B2A.6020101@v.loewis.de>
Message-ID: <44E60D00.7090304@python.net>

Martin v. L?wis schrieb:
> Thomas Heller schrieb:
>> I suggest that it should be removed from the 2.5 win64 msi installers, so that
>> at least, when it is ready, can be installed as separate package.
> 
> Unfortunately, I won't be able to work on this until the release is
> made. Feel free to work on it, but I suggest to just not worry about
> it.
> 
> There are many other problems with Win64 still, e.g. the test suite
> doesn't pass.

And tons of compiler warnings, IIRC.  If the state is still experimental,
IMO it should be marked as such.

Thomas


From christopher at baus.net  Fri Aug 18 21:00:50 2006
From: christopher at baus.net (christopher baus)
Date: Fri, 18 Aug 2006 19:00:50 -0000 (GMT)
Subject: [Python-Dev] Failed building 2.5rc1 pythoncore on VC8
Message-ID: <2174.69.181.106.91.1155927650.squirrel@webmail.tuffmail.net>

Hi,

I downloaded RC1 to give it a go.

I failed to build pythoncore with VC8.

The error is:

Linking...
   Creating library .\./python25_d.lib and object .\./python25_d.exp
config.obj : error LNK2001: unresolved external symbol _init_types
./python25_d.dll : fatal error LNK1120: 1 unresolved externals

Thanks,

Chris


From jimjjewett at gmail.com  Fri Aug 18 22:24:08 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 18 Aug 2006 16:24:08 -0400
Subject: [Python-Dev] How does this help? Re: [Python-checkins] r51366 -
	python/trunk/Lib/idlelib/NEWS.txt
	python/trunk/Lib/idlelib/idlever.py
Message-ID: <fb6fbf560608181324m26d2a93dl4db344e23beac8f8@mail.gmail.com>

This makes things more consistent for today, but does it really ease
maintenance?

Why not just change it to:

from sys import version as IDLE_VERSION

so that it can be ignored from now on?

On 8/18/06, anthony.baxter <python-checkins at python.org> wrote:
> Author: anthony.baxter
> Date: Fri Aug 18 09:29:02 2006
> New Revision: 51366
>
> Modified:
>    python/trunk/Lib/idlelib/NEWS.txt
>    python/trunk/Lib/idlelib/idlever.py
> Log:
> Updating IDLE's version number to match Python's (as per python-dev
> discussion).

==============================================================================
> --- python/trunk/Lib/idlelib/idlever.py (original)
> +++ python/trunk/Lib/idlelib/idlever.py Fri Aug 18 09:29:02 2006
> @@ -1 +1 @@
> -IDLE_VERSION = "1.2c1"
> +IDLE_VERSION = "2.6a0"

From nnorwitz at gmail.com  Fri Aug 18 23:35:40 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 18 Aug 2006 14:35:40 -0700
Subject: [Python-Dev] Failed building 2.5rc1 pythoncore on VC8
In-Reply-To: <2174.69.181.106.91.1155927650.squirrel@webmail.tuffmail.net>
References: <2174.69.181.106.91.1155927650.squirrel@webmail.tuffmail.net>
Message-ID: <ee2a432c0608181435l8f219a2r604a8c0657329076@mail.gmail.com>

VC8 is not a supported compiler at this point.  However, patches are
greatly accepted.

The _types module was added late and probably VC6 and VC8 project
files did not get updated.  You can search for the necessary mods to
the VC7 proj file(s) on python-checkins.

n
--
On 8/18/06, christopher baus <christopher at baus.net> wrote:
> Hi,
>
> I downloaded RC1 to give it a go.
>
> I failed to build pythoncore with VC8.
>
> The error is:
>
> Linking...
>    Creating library .\./python25_d.lib and object .\./python25_d.exp
> config.obj : error LNK2001: unresolved external symbol _init_types
> ./python25_d.dll : fatal error LNK1120: 1 unresolved externals
>
> Thanks,
>
> Chris
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
>

From martin at v.loewis.de  Fri Aug 18 23:48:18 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 18 Aug 2006 23:48:18 +0200
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <44E60D00.7090304@python.net>
References: <ec50v9$oln$1@sea.gmane.org> <44E60B2A.6020101@v.loewis.de>
	<44E60D00.7090304@python.net>
Message-ID: <44E635A2.3020000@v.loewis.de>

Thomas Heller schrieb:
>> There are many other problems with Win64 still, e.g. the test suite
>> doesn't pass.
> 
> And tons of compiler warnings, IIRC.  If the state is still experimental,
> IMO it should be marked as such.

Most of the compiler warnings are irrelevant for most of the
applications, and some are bogus (in the sense that the code
is perfectly fine as it stands).

Not sure what "experimental" would mean; it normally means
that it may change or may be abandoned as a failure. I don't
think this is the case with this port (although I do want
to drop IA-64 binaries in 2.6, due to lack of interest).

If you think it means "has known bugs", then this applies to
all Python releases on all target systems, ever (just look
at the open bugs list).

Regards,
Martin

From christopher at baus.net  Sat Aug 19 01:46:18 2006
From: christopher at baus.net (christopher baus)
Date: Fri, 18 Aug 2006 23:46:18 -0000 (GMT)
Subject: [Python-Dev] Failed building 2.5rc1 pythoncore on VC8
In-Reply-To: <ee2a432c0608181435l8f219a2r604a8c0657329076@mail.gmail.com>
References: <2174.69.181.106.91.1155927650.squirrel@webmail.tuffmail.net>
	<ee2a432c0608181435l8f219a2r604a8c0657329076@mail.gmail.com>
Message-ID: <2136.69.181.106.91.1155944778.squirrel@webmail.tuffmail.net>

> VC8 is not a supported compiler at this point.  However, patches are
> greatly accepted.
>
> The _types module was added late and probably VC6 and VC8 project
> files did not get updated.  You can search for the necessary mods to
> the VC7 proj file(s) on python-checkins.
>

Index: pythoncore.vcproj
===================================================================
--- pythoncore.vcproj	(revision 1020)
+++ pythoncore.vcproj	(working copy)
@@ -1,7 +1,7 @@
 <?xml version="1.0" encoding="Windows-1252"?>
 <VisualStudioProject
 	ProjectType="Visual C++"
-	Version="8,00"
+	Version="8.00"
 	Name="pythoncore"
 	ProjectGUID="{CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26}"
 	RootNamespace="pythoncore"
@@ -558,6 +558,10 @@
 			>
 		</File>
 		<File
+			RelativePath="..\Modules\_typesmodule.c"
+			>
+		</File>
+		<File
 			RelativePath="..\Modules\_weakref.c"
 			>
 		</File>



From christopher at baus.net  Sat Aug 19 02:54:53 2006
From: christopher at baus.net (christopher baus)
Date: Sat, 19 Aug 2006 00:54:53 -0000 (GMT)
Subject: [Python-Dev] patch to add socket module project to vc8 solution
Message-ID: <2928.69.181.106.91.1155948893.squirrel@webmail.tuffmail.net>

This is a patch to add the socket module project to the vc8 solution.

Index: pcbuild.sln
===================================================================
--- pcbuild.sln	(revision 1020)
+++ pcbuild.sln	(working copy)
@@ -2,8 +2,8 @@
 # Visual Studio 2005
 Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pythoncore",
"pythoncore.vcproj", "{CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26}"
 	ProjectSection(ProjectDependencies) = postProject
+		{F0E0541E-F17D-430B-97C4-93ADF0DD284E} =
{F0E0541E-F17D-430B-97C4-93ADF0DD284E}
 		{C73F0EC1-358B-4177-940F-0846AC8B04CD} =
{C73F0EC1-358B-4177-940F-0846AC8B04CD}
-		{F0E0541E-F17D-430B-97C4-93ADF0DD284E} =
{F0E0541E-F17D-430B-97C4-93ADF0DD284E}
 	EndProjectSection
 EndProject
 Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pythonw",
"pythonw.vcproj", "{F4229CC3-873C-49AE-9729-DD308ED4CD4A}"
@@ -70,6 +70,8 @@
 EndProject
 Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "make_versioninfo",
"make_versioninfo.vcproj", "{F0E0541E-F17D-430B-97C4-93ADF0DD284E}"
 EndProject
+Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_socket",
"_socket.vcproj", "{324F66C2-44D0-4D50-B979-F9DAE7FD36DB}"
+EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Win32 = Debug|Win32
@@ -192,6 +194,14 @@
 		{F0E0541E-F17D-430B-97C4-93ADF0DD284E}.ReleaseAMD64|Win32.Build.0 =
Release|Win32
 		{F0E0541E-F17D-430B-97C4-93ADF0DD284E}.ReleaseItanium|Win32.ActiveCfg =
Release|Win32
 		{F0E0541E-F17D-430B-97C4-93ADF0DD284E}.ReleaseItanium|Win32.Build.0 =
Release|Win32
+		{324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.Debug|Win32.ActiveCfg = Debug|Win32
+		{324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.Debug|Win32.Build.0 = Debug|Win32
+		{324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.Release|Win32.ActiveCfg =
Release|Win32
+		{324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.Release|Win32.Build.0 =
Release|Win32
+		{324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.ReleaseAMD64|Win32.ActiveCfg =
ReleaseAMD64|Win32
+		{324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.ReleaseAMD64|Win32.Build.0 =
ReleaseAMD64|Win32
+		{324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.ReleaseItanium|Win32.ActiveCfg =
ReleaseItanium|Win32
+		{324F66C2-44D0-4D50-B979-F9DAE7FD36DB}.ReleaseItanium|Win32.Build.0 =
ReleaseItanium|Win32
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE



From christopher at baus.net  Sat Aug 19 02:56:42 2006
From: christopher at baus.net (christopher baus)
Date: Sat, 19 Aug 2006 00:56:42 -0000 (GMT)
Subject: [Python-Dev] patch to add socket module project to vc8 solution
In-Reply-To: <2928.69.181.106.91.1155948893.squirrel@webmail.tuffmail.net>
References: <2928.69.181.106.91.1155948893.squirrel@webmail.tuffmail.net>
Message-ID: <2931.69.181.106.91.1155949002.squirrel@webmail.tuffmail.net>

> This is a patch to add the socket module project to the vc8 solution.
>

Sorry the patch got munged by my mail client.  I attached it instead.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: addsocket.patch
Type: application/octet-stream
Size: 2294 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060819/5d64d98d/attachment.obj 

From jjl at pobox.com  Sat Aug 19 04:05:14 2006
From: jjl at pobox.com (John J Lee)
Date: Sat, 19 Aug 2006 02:05:14 +0000 (UTC)
Subject: [Python-Dev] One-line fix for urllib2 regression
Message-ID: <Pine.LNX.4.64.0608190157000.8383@localhost>

Revision 50842 made a change to an undocumented interface of urllib2 that 
I'm sure will break real code.

Patch 1542948 reverts the part of that commit that applied to urllib2, and 
adds a one-line fix in its place that addresses the problem that 50842 
fixed.

Details are on the patch tracker:

http://python.org/sf/1542948


Can somebody commit this for 2.5?


John


From guido at python.org  Sat Aug 19 04:21:10 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 18 Aug 2006 19:21:10 -0700
Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4?
Message-ID: <ca471dc20608181921r44d9e06bkc3ac24bb0a5fad69@mail.gmail.com>

I just got a report from a Windows user that os.spawnlp() is missing
from Python 2.4, despite being mentioned in the docs. Can someone
confirm this? My Windows box is resting. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From scott+python-dev at scottdial.com  Sat Aug 19 05:16:30 2006
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Fri, 18 Aug 2006 23:16:30 -0400
Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4?
In-Reply-To: <ca471dc20608181921r44d9e06bkc3ac24bb0a5fad69@mail.gmail.com>
References: <ca471dc20608181921r44d9e06bkc3ac24bb0a5fad69@mail.gmail.com>
Message-ID: <44E6828E.4000505@scottdial.com>

Guido van Rossum wrote:
> I just got a report from a Windows user that os.spawnlp() is missing
> from Python 2.4, despite being mentioned in the docs. Can someone
> confirm this? My Windows box is resting. :-)
> 

"Availability: Unix, Windows. spawnlp(), spawnlpe(), spawnvp() and 
spawnvpe() are not available on Windows. New in version 1.6."

One could argue that it presented poorly, but it reads completely 
correct. Alternatively one could says "The 'p' variants are unavailable 
on Windows." but that would be assuming someone understand there was a 
scheme to the names :-)

-- 
Scott Dial
scott at scottdial.com
scodial at indiana.edu

From steve at holdenweb.com  Sat Aug 19 05:27:53 2006
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 19 Aug 2006 04:27:53 +0100
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <44E635A2.3020000@v.loewis.de>
References: <ec50v9$oln$1@sea.gmane.org>
	<44E60B2A.6020101@v.loewis.de>	<44E60D00.7090304@python.net>
	<44E635A2.3020000@v.loewis.de>
Message-ID: <ec60f1$a53$1@sea.gmane.org>

Martin v. L?wis wrote:
> Thomas Heller schrieb:
> 
>>>There are many other problems with Win64 still, e.g. the test suite
>>>doesn't pass.
>>
>>And tons of compiler warnings, IIRC.  If the state is still experimental,
>>IMO it should be marked as such.
> 
> 
> Most of the compiler warnings are irrelevant for most of the
> applications, and some are bogus (in the sense that the code
> is perfectly fine as it stands).
> 
> Not sure what "experimental" would mean; it normally means
> that it may change or may be abandoned as a failure. I don't
> think this is the case with this port (although I do want
> to drop IA-64 binaries in 2.6, due to lack of interest).
> 
> If you think it means "has known bugs", then this applies to
> all Python releases on all target systems, ever (just look
> at the open bugs list).
> 
Reasonable enough, but I suspect that Thomas' suggestion might save us 
from raising false hopes. I'd suggest that the final release 
announcement point out that this is the first release containing 
specific support for 64-bit architectures (if indeed it is), and that 
the developers would therefore expect a higher proportion of bug reports 
on this release to be associated with those platforms. Committed to 
fixing, blah blah blah ...

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd          http://www.holdenweb.com
Skype: holdenweb       http://holdenweb.blogspot.com
Recent Ramblings     http://del.icio.us/steve.holden


From steve at holdenweb.com  Sat Aug 19 05:34:16 2006
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 19 Aug 2006 04:34:16 +0100
Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4?
In-Reply-To: <44E6828E.4000505@scottdial.com>
References: <ca471dc20608181921r44d9e06bkc3ac24bb0a5fad69@mail.gmail.com>
	<44E6828E.4000505@scottdial.com>
Message-ID: <44E686B8.6040609@holdenweb.com>

Scott Dial wrote:
> Guido van Rossum wrote:
> 
>>I just got a report from a Windows user that os.spawnlp() is missing
>>from Python 2.4, despite being mentioned in the docs. Can someone
>>confirm this? My Windows box is resting. :-)
>>
> 
> 
> "Availability: Unix, Windows. spawnlp(), spawnlpe(), spawnvp() and 
> spawnvpe() are not available on Windows. New in version 1.6."
> 
> One could argue that it presented poorly, but it reads completely 
> correct. Alternatively one could says "The 'p' variants are unavailable 
> on Windows." but that would be assuming someone understand there was a 
> scheme to the names :-)
> 
How about:

"Availability: Unix; Windows PARTIAL (spawnlp(), spawnlpe(), spawnvp() 
and spawnvpe() are not implemented). New in version 1.6"

regards
  Steve

PS: It's not resting, it has gone to meet its maker. It has joined the 
choir celestial, it has shuffled off this mortal coil, its clock has 
ceased to cycle, it is no more, it is an X-box. Oh, sorry, that's a 
Microsoft product isn't it.
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd          http://www.holdenweb.com
Skype: holdenweb       http://holdenweb.blogspot.com
Recent Ramblings     http://del.icio.us/steve.holden

From steve at holdenweb.com  Sat Aug 19 05:34:16 2006
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 19 Aug 2006 04:34:16 +0100
Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4?
In-Reply-To: <44E6828E.4000505@scottdial.com>
References: <ca471dc20608181921r44d9e06bkc3ac24bb0a5fad69@mail.gmail.com>
	<44E6828E.4000505@scottdial.com>
Message-ID: <44E686B8.6040609@holdenweb.com>

Scott Dial wrote:
> Guido van Rossum wrote:
> 
>>I just got a report from a Windows user that os.spawnlp() is missing
>>from Python 2.4, despite being mentioned in the docs. Can someone
>>confirm this? My Windows box is resting. :-)
>>
> 
> 
> "Availability: Unix, Windows. spawnlp(), spawnlpe(), spawnvp() and 
> spawnvpe() are not available on Windows. New in version 1.6."
> 
> One could argue that it presented poorly, but it reads completely 
> correct. Alternatively one could says "The 'p' variants are unavailable 
> on Windows." but that would be assuming someone understand there was a 
> scheme to the names :-)
> 
How about:

"Availability: Unix; Windows PARTIAL (spawnlp(), spawnlpe(), spawnvp() 
and spawnvpe() are not implemented). New in version 1.6"

regards
  Steve

PS: It's not resting, it has gone to meet its maker. It has joined the 
choir celestial, it has shuffled off this mortal coil, its clock has 
ceased to cycle, it is no more, it is an X-box. Oh, sorry, that's a 
Microsoft product isn't it.
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd          http://www.holdenweb.com
Skype: holdenweb       http://holdenweb.blogspot.com
Recent Ramblings     http://del.icio.us/steve.holden


From collinw at gmail.com  Sat Aug 19 05:40:30 2006
From: collinw at gmail.com (Collin Winter)
Date: Fri, 18 Aug 2006 22:40:30 -0500
Subject: [Python-Dev] Questions on unittest behaviour
Message-ID: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com>

[Sorry for accidentally cross-posting this to python-list]

While working on a test suite for unittest these past few weeks, I've
run across some behaviours that, while not obviously wrong, don't
strike me as quite right, either. Submitted for your consideration:

1) TestCase.tearDown() is only run if TestCase.setUp() succeeded. It
seems to me that tearDown() should always be run, regardless of any
failures in setUp() or the test method itself.

The case I'm considering is something like this, ie, a multi-part setUp():

> def setUp(self)
>     lock_file(testfile) # open_socket(), connect_to_database(), etc
>
>     something_that_raises_an_exception()
>
> def tearDown(self):
>     if file_is_locked(testfile):
>         unlock_file(testfile)

In this pseudo-code example, the file won't be unlocked if some later
operation in setUp() raises an exception. I propose that
TestCase.run() be changed to always run tearDown(), even if setUp()
raise an exception.

I'm undecided if this is a new feature (so it should go in for 2.6) or
a bug fix; I'm leaning toward the latter.

2) The TestLoader.testMethodPrefix attribute currently allows anything
to be assigned to it, including invalid objects and the empty string.
While the former will cause errors to be raised when one of
TestLoader's loadTestsFrom*() methods is called, the empty string will
raise no exception; rather, the loadTestsFrom*() methods will
interpret every possible attribute as being a test method, e.g.,
meaning you get things like assertEqual(), failUnlessEqual(), etc,
when TestLoader.loadTestsFromTestCase() is run.

I propose protecting testMethodPrefix with a property that validates
the assigned value, restricting input to non-empty instances of str. I
see this as a bug fix that should go in before 2.5-final.

3) TestLoader.loadTestsFromTestCase() accepts objects that are not
test cases and will happily look for appropriately-named methods on
any object you give it. This flexibility should be documented, or
proper input validation should be done (a bug fix for 2.5).

4) TestLoader.loadTestsFromName() (and by extension,
loadTestsFromNames(), too) raises an AttributeError if the name is the
empty string because -- as it correctly asserts -- the object does not
contain an attribute named ''. I recommend that this be tested for and
ValueError be raised (bug fix for 2.5).

This of course leads into the question of how much input validation
should be done on these names. Should loadTestsFrom{Name,Names}() make
sure the names are valid attribute names, or is this overkill?

5) When TestLoader.loadTestsFrom{Name,Names}() are given a name that
resolves to a classmethod on a TestCase subclass, the method is not
invoked. From the docs:

> The specifier name is a ``dotted name'' that may resolve either to a module, a test
> case class, a TestSuite instance, a test method within a test case class, or a
> callable object which returns a TestCase or TestSuite instance.

It is not documented which of these tests takes priority: is the
classmethod "a test method within a test case class" or is it a
callable? The same issue applies to staticmethods as well.


Once I get answers to these questions, I can finish off the last few
bits of the test suite and have it ready for 2.5-final.

Thanks,
Collin Winter

From g.brandl at gmx.net  Sat Aug 19 07:28:56 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 19 Aug 2006 07:28:56 +0200
Subject: [Python-Dev] One-line fix for urllib2 regression
In-Reply-To: <Pine.LNX.4.64.0608190157000.8383@localhost>
References: <Pine.LNX.4.64.0608190157000.8383@localhost>
Message-ID: <ec67ip$obv$1@sea.gmane.org>

John J Lee wrote:
> Revision 50842 made a change to an undocumented interface of urllib2 that 
> I'm sure will break real code.
> 
> Patch 1542948 reverts the part of that commit that applied to urllib2, and 
> adds a one-line fix in its place that addresses the problem that 50842 
> fixed.
> 
> Details are on the patch tracker:
> 
> http://python.org/sf/1542948
> 
> 
> Can somebody commit this for 2.5?

As the committer of the original patch, I have reviewed it and I'm for it.

Georg


From martin at v.loewis.de  Sat Aug 19 07:51:36 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 19 Aug 2006 07:51:36 +0200
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <ec60f1$a53$1@sea.gmane.org>
References: <ec50v9$oln$1@sea.gmane.org>	<44E60B2A.6020101@v.loewis.de>	<44E60D00.7090304@python.net>	<44E635A2.3020000@v.loewis.de>
	<ec60f1$a53$1@sea.gmane.org>
Message-ID: <44E6A6E8.3000101@v.loewis.de>

Steve Holden schrieb:
> Reasonable enough, but I suspect that Thomas' suggestion might save us 
> from raising false hopes. I'd suggest that the final release 
> announcement point out that this is the first release containing 
> specific support for 64-bit architectures (if indeed it is)

It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess),
and was released for Win64 throughout Python 2.4. ActiveState has
been releasing an AMD64 package for some time now.

Regards,
Martin

From ncoghlan at gmail.com  Sat Aug 19 08:23:21 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Aug 2006 16:23:21 +1000
Subject: [Python-Dev] Questions on unittest behaviour
In-Reply-To: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com>
References: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com>
Message-ID: <44E6AE59.50602@gmail.com>

Collin Winter wrote:
> [Sorry for accidentally cross-posting this to python-list]
> 
> While working on a test suite for unittest these past few weeks, I've
> run across some behaviours that, while not obviously wrong, don't
> strike me as quite right, either. Submitted for your consideration:
> 
> 1) TestCase.tearDown() is only run if TestCase.setUp() succeeded. It
> seems to me that tearDown() should always be run, regardless of any
> failures in setUp() or the test method itself.
> The case I'm considering is something like this, ie, a multi-part setUp():
> 
>> def setUp(self)
>>     lock_file(testfile) # open_socket(), connect_to_database(), etc
>>
>>     something_that_raises_an_exception()
>>
>> def tearDown(self):
>>     if file_is_locked(testfile):
>>         unlock_file(testfile)
> 
> In this pseudo-code example, the file won't be unlocked if some later
> operation in setUp() raises an exception. I propose that
> TestCase.run() be changed to always run tearDown(), even if setUp()
> raise an exception.
> 
> I'm undecided if this is a new feature (so it should go in for 2.6) or
> a bug fix; I'm leaning toward the latter.

On this point, I believe the current behaviour is correct and should be kept. 
If setUp() fails internally (such that the test step isn't going to be run) it 
needs to cleanup after itself, just like any other function. That way, the 
tearDown() method is allowed to assume that setUp() succeeded completely, 
instead of having to guard against the possibility that setUp() may have died 
partway through.

IOW, I consider the setUp() method in your example to be buggy. It should be 
written something like this:

  def setUp(self)
      lock_file(testfile) # open_socket(), connect_to_database(), etc
      try:
          something_that_may_raise_an_exception()
      except:
          unlock_file(testfile)
          raise

  def tearDown(self):
      unlock_file(testfile)

Alternatively, someone who prefers your style (with a tearDown() method that 
can handle a partially executed call to the setUp() method), can just write it as:

  def setUp(self)
      try:
          lock_file(testfile) # open_socket(), connect_to_database(), etc
          something_that_may_raise_an_exception()
      except:
          self.tearDown()
          raise

  def tearDown(self):
      if file_is_locked(testfile):
          unlock_file(testfile)

Consider the parallel to PEP 343's __enter__ and __exit__ methods - __exit__ 
is allowed to assume that it will only be called if __enter__ succeeded, 
because that is part of the semantics of the with statement.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From jml at mumak.net  Sat Aug 19 08:36:48 2006
From: jml at mumak.net (Jonathan Lange)
Date: Sat, 19 Aug 2006 16:36:48 +1000
Subject: [Python-Dev] Questions on unittest behaviour
In-Reply-To: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com>
References: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com>
Message-ID: <d06a5cd30608182336p6c8799ccw3a37f18dcbc49399@mail.gmail.com>

On 8/19/06, Collin Winter <collinw at gmail.com> wrote:
> 1) TestCase.tearDown() is only run if TestCase.setUp() succeeded. It
> seems to me that tearDown() should always be run, regardless of any
> failures in setUp() or the test method itself.
>
> The case I'm considering is something like this, ie, a multi-part setUp():
> > def setUp(self)
> >     lock_file(testfile) # open_socket(), connect_to_database(), etc
> >
> >     something_that_raises_an_exception()
> >
> > def tearDown(self):
> >     if file_is_locked(testfile):
> >         unlock_file(testfile)
>
> I'm undecided if this is a new feature (so it should go in for 2.6) or
> a bug fix; I'm leaning toward the latter.
>

Most existing tearDown() code assumes that setUp() has been
successfully called -- after all, that's what the docs say. It's also
the behaviour of other xUnit frameworks.

If this change is made, people will have to go through their
tearDown() methods and add checks like the one in your example in
order to make their tests correct. Further, it won't be obvious that
their tearDown() methods are incorrect until something happens to make
their setUp() methods incorrect.

I don't think this change is a good one. However, if it does go in, it
definitely shouldn't go in as a bug fix.

cheers,
jml

From anthony at interlink.com.au  Sat Aug 19 10:13:59 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat, 19 Aug 2006 18:13:59 +1000
Subject: [Python-Dev] How does this help? Re: [Python-checkins] r51366 -
	python/trunk/Lib/idlelib/NEWS.txt
	python/trunk/Lib/idlelib/idlever.py
In-Reply-To: <fb6fbf560608181324m26d2a93dl4db344e23beac8f8@mail.gmail.com>
References: <fb6fbf560608181324m26d2a93dl4db344e23beac8f8@mail.gmail.com>
Message-ID: <200608191814.05404.anthony@interlink.com.au>

On Saturday 19 August 2006 06:24, Jim Jewett wrote:
> This makes things more consistent for today, but does it really ease
> maintenance?
>
> Why not just change it to:
>
> from sys import version as IDLE_VERSION
>
> so that it can be ignored from now on?

After the fuss about changing distutils' version number? :-)

I'd very much like to do this, but I figured this was a good first step.


-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From anthony at interlink.com.au  Sat Aug 19 10:16:11 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat, 19 Aug 2006 18:16:11 +1000
Subject: [Python-Dev] One-line fix for urllib2 regression
In-Reply-To: <ec67ip$obv$1@sea.gmane.org>
References: <Pine.LNX.4.64.0608190157000.8383@localhost>
	<ec67ip$obv$1@sea.gmane.org>
Message-ID: <200608191816.12174.anthony@interlink.com.au>

On Saturday 19 August 2006 15:28, Georg Brandl wrote:
> John J Lee wrote:
> > Revision 50842 made a change to an undocumented interface of urllib2 that
> > I'm sure will break real code.
> >
> > Patch 1542948 reverts the part of that commit that applied to urllib2,
> > and adds a one-line fix in its place that addresses the problem that
> > 50842 fixed.
> >
> > Details are on the patch tracker:
> >
> > http://python.org/sf/1542948
> >
> >
> > Can somebody commit this for 2.5?
>
> As the committer of the original patch, I have reviewed it and I'm for it.

I agree with Neal's comment - this needs tests to make sure it doesn't break 
again. It's OK to apply it with the tests. 

Anthony


-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From tim.peters at gmail.com  Sat Aug 19 11:19:40 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 19 Aug 2006 05:19:40 -0400
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <44E6A6E8.3000101@v.loewis.de>
References: <ec50v9$oln$1@sea.gmane.org> <44E60B2A.6020101@v.loewis.de>
	<44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de>
	<ec60f1$a53$1@sea.gmane.org> <44E6A6E8.3000101@v.loewis.de>
Message-ID: <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com>

[Steve Holden]
>> Reasonable enough, but I suspect that Thomas' suggestion might save us
>> from raising false hopes. I'd suggest that the final release
>> announcement point out that this is the first release containing
>> specific support for 64-bit architectures (if indeed it is)

[Martin v. L?wis]
> It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess),
> and was released for Win64 throughout Python 2.4. ActiveState has
> been releasing an AMD64 package for some time now.

Python has also been used on 64-bit Crays, and I actually did the
first 64-bit port in 1993 (to a KSR Unix machine -- took less than a
day to get it running fine!  Guido's an excellent C coder.).  Win64 is
the first (and probably only forever more) where sizeof(long) <
sizeof(void*), and that caused some Win64-unique pain, and may cause
more.

BTW, at least two of the people at the NFS sprint earlier this year
were compiling and running Python on Win64 laptops.  It's "solid
enough", and surely nobody expects that Win64 users expect 100%
perfection of anything they run <0.5 wink>.

From steve at holdenweb.com  Sat Aug 19 12:28:13 2006
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 19 Aug 2006 11:28:13 +0100
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com>
References: <ec50v9$oln$1@sea.gmane.org> <44E60B2A.6020101@v.loewis.de>	
	<44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de>	
	<ec60f1$a53$1@sea.gmane.org> <44E6A6E8.3000101@v.loewis.de>
	<1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com>
Message-ID: <44E6E7BD.8000903@holdenweb.com>

Tim Peters wrote:
> [Steve Holden]
> 
>>> Reasonable enough, but I suspect that Thomas' suggestion might save us
>>> from raising false hopes. I'd suggest that the final release
>>> announcement point out that this is the first release containing
>>> specific support for 64-bit architectures (if indeed it is)
> 
> 
> [Martin v. L?wis]
> 
>> It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess),
>> and was released for Win64 throughout Python 2.4. ActiveState has
>> been releasing an AMD64 package for some time now.
> 
> 
> Python has also been used on 64-bit Crays, and I actually did the
> first 64-bit port in 1993 (to a KSR Unix machine -- took less than a
> day to get it running fine!  Guido's an excellent C coder.).  Win64 is
> the first (and probably only forever more) where sizeof(long) <
> sizeof(void*), and that caused some Win64-unique pain, and may cause
> more.
> 
> BTW, at least two of the people at the NFS sprint earlier this year
> were compiling and running Python on Win64 laptops.  It's "solid
> enough", and surely nobody expects that Win64 users expect 100%
> perfection of anything they run <0.5 wink>.
> 
I suppose we should be grateful thirteen years experience has gone into 
64-bit support. I agree that "some breakage may occur" is true of any 
release and architecture though in the eyes of this lurker, at least, 
python-dev does a more-than-averagely conscientious job.

It's going to be very interesting to see what comes out of the Google 
sprints. I am sure the 64-bitters will be out in force, so there'll be 
useful data about any such problems. <irony wink="0.97">64 bits is, I am 
sure, as much as anyone is ever going to need, so the pain will finally 
be over.</irony>

It's good to see the idea of industry support for open source sprints 
taking off. Tomorrow, the world ... :-)

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd          http://www.holdenweb.com
Skype: holdenweb       http://holdenweb.blogspot.com
Recent Ramblings     http://del.icio.us/steve.holden

From kbk at shore.net  Sat Aug 19 17:40:03 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Sat, 19 Aug 2006 11:40:03 -0400
Subject: [Python-Dev] [Python-checkins] How does this help? Re: r51366 -
	python/trunk/Lib/idlelib/NEWS.txt
	python/trunk/Lib/idlelib/idlever.py
In-Reply-To: <200608191814.05404.anthony@interlink.com.au> (Anthony Baxter's
	message of "Sat, 19 Aug 2006 18:13:59 +1000")
References: <fb6fbf560608181324m26d2a93dl4db344e23beac8f8@mail.gmail.com>
	<200608191814.05404.anthony@interlink.com.au>
Message-ID: <87fyfs4tv0.fsf@hydra.bayview.thirdcreek.com>

Anthony Baxter <anthony at interlink.com.au> writes:

> On Saturday 19 August 2006 06:24, Jim Jewett wrote:
>> This makes things more consistent for today, but does it really ease
>> maintenance?
>>
>> Why not just change it to:
>>
>> from sys import version as IDLE_VERSION
>>
>> so that it can be ignored from now on?
>
> After the fuss about changing distutils' version number? :-)
>
> I'd very much like to do this, but I figured this was a good first step.

Jim Jewett's suggestion could actually be misleading, IMO.

There are users, especially on Windows, who run IDLE out of svn or run
their own version, but don't build Python.

Here's my suggestion: we set IDLE_VERSION="" in idlever.py.  If the IDLE
hacker wants to set a specific version, he sets that string.  I'll change
the code so if the string is empty, IDLE will use the Python version, as
Jim suggested; otherwise, the user supplied string.

-- 
KBK

From jack at psynchronous.com  Sat Aug 19 16:27:45 2006
From: jack at psynchronous.com (Jack Diederich)
Date: Sat, 19 Aug 2006 10:27:45 -0400
Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4?
In-Reply-To: <44E686B8.6040609@holdenweb.com>
References: <ca471dc20608181921r44d9e06bkc3ac24bb0a5fad69@mail.gmail.com>
	<44E6828E.4000505@scottdial.com> <44E686B8.6040609@holdenweb.com>
Message-ID: <20060819142745.GJ5772@performancedrivers.com>

On Sat, Aug 19, 2006 at 04:34:16AM +0100, Steve Holden wrote:
> Scott Dial wrote:
> > Guido van Rossum wrote:
> > 
> >>I just got a report from a Windows user that os.spawnlp() is missing
> >>from Python 2.4, despite being mentioned in the docs. Can someone
> >>confirm this? My Windows box is resting. :-)
> >>
> > 
> > 
> > "Availability: Unix, Windows. spawnlp(), spawnlpe(), spawnvp() and 
> > spawnvpe() are not available on Windows. New in version 1.6."
> > 
> > One could argue that it presented poorly, but it reads completely 
> > correct. Alternatively one could says "The 'p' variants are unavailable 
> > on Windows." but that would be assuming someone understand there was a 
> > scheme to the names :-)
> > 
> How about:
> 
> "Availability: Unix; Windows PARTIAL (spawnlp(), spawnlpe(), spawnvp() 
> and spawnvpe() are not implemented). New in version 1.6"
> 
Or

*Availability: Unix: All, Windows: spawnl(), spawnle(), spawnv(), 
spawnve() only.  New in version 1.6

Might as well positively list the half that is there instead of the half
that isn't.

-Jack

From jack at psynchronous.com  Sat Aug 19 16:40:42 2006
From: jack at psynchronous.com (Jack Diederich)
Date: Sat, 19 Aug 2006 10:40:42 -0400
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com>
References: <ec50v9$oln$1@sea.gmane.org> <44E60B2A.6020101@v.loewis.de>
	<44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de>
	<ec60f1$a53$1@sea.gmane.org> <44E6A6E8.3000101@v.loewis.de>
	<1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com>
Message-ID: <20060819144042.GK5772@performancedrivers.com>

On Sat, Aug 19, 2006 at 05:19:40AM -0400, Tim Peters wrote:
> [Steve Holden]
> >> Reasonable enough, but I suspect that Thomas' suggestion might save us
> >> from raising false hopes. I'd suggest that the final release
> >> announcement point out that this is the first release containing
> >> specific support for 64-bit architectures (if indeed it is)
> 
> [Martin v. L?wis]
> > It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess),
> > and was released for Win64 throughout Python 2.4. ActiveState has
> > been releasing an AMD64 package for some time now.
> 
> Python has also been used on 64-bit Crays, and I actually did the
> first 64-bit port in 1993 (to a KSR Unix machine -- took less than a
> day to get it running fine!  Guido's an excellent C coder.).  Win64 is
> the first (and probably only forever more) where sizeof(long) <
> sizeof(void*), and that caused some Win64-unique pain, and may cause
> more.
> 
> BTW, at least two of the people at the NFS sprint earlier this year
> were compiling and running Python on Win64 laptops.  It's "solid
> enough", and surely nobody expects that Win64 users expect 100%
> perfection of anything they run <0.5 wink>.

It has always "just worked" for me on Opterons + Debian.
Python 2.4 (#1, May 31 2005, 10:19:45) 
[GCC 3.3.5 (Debian 1:3.3.5-12)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.maxint
9223372036854775807

Thanks to the Alphas for making AMD64 on *nix a no-brainer,

-Jack

From collinw at gmail.com  Sat Aug 19 19:06:17 2006
From: collinw at gmail.com (Collin Winter)
Date: Sat, 19 Aug 2006 12:06:17 -0500
Subject: [Python-Dev] Questions on unittest behaviour
In-Reply-To: <44E6AE59.50602@gmail.com>
References: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com>
	<44E6AE59.50602@gmail.com>
Message-ID: <43aa6ff70608191006g5f9bf89fqfe56409792deb866@mail.gmail.com>

On 8/19/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Alternatively, someone who prefers your style (with a tearDown() method that
> can handle a partially executed call to the setUp() method), can just write it as:
>
>   def setUp(self)
>       try:
>           lock_file(testfile) # open_socket(), connect_to_database(), etc
>           something_that_may_raise_an_exception()
>       except:
>           self.tearDown()
>           raise
>
>   def tearDown(self):
>       if file_is_locked(testfile):
>           unlock_file(testfile)
>
> Consider the parallel to PEP 343's __enter__ and __exit__ methods - __exit__
> is allowed to assume that it will only be called if __enter__ succeeded,
> because that is part of the semantics of the with statement.

I can accept that.

Any thoughts on the other four items?

Collin Winter

From tjreedy at udel.edu  Sat Aug 19 19:17:07 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 19 Aug 2006 13:17:07 -0400
Subject: [Python-Dev] os.spawnlp() missing on Windows in 2.4?
References: <ca471dc20608181921r44d9e06bkc3ac24bb0a5fad69@mail.gmail.com><44E6828E.4000505@scottdial.com>
	<44E686B8.6040609@holdenweb.com>
	<20060819142745.GJ5772@performancedrivers.com>
Message-ID: <ec7h2j$b57$1@sea.gmane.org>


"Jack Diederich" <jack at psynchronous.com> wrote in message 
news:20060819142745.GJ5772 at performancedrivers.com...
> *Availability: Unix: All, Windows: spawnl(), spawnle(), spawnv(),
> spawnve() only.  New in version 1.6
>
> Might as well positively list the half that is there instead of the half
> that isn't.

Definitately.  Succinct and clear. 




From ncoghlan at gmail.com  Sat Aug 19 20:07:53 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 20 Aug 2006 04:07:53 +1000
Subject: [Python-Dev] Questions on unittest behaviour
In-Reply-To: <43aa6ff70608191006g5f9bf89fqfe56409792deb866@mail.gmail.com>
References: <43aa6ff70608182040i43c28461i81011a18d5923ccc@mail.gmail.com>	
	<44E6AE59.50602@gmail.com>
	<43aa6ff70608191006g5f9bf89fqfe56409792deb866@mail.gmail.com>
Message-ID: <44E75379.2020506@gmail.com>

Collin Winter wrote:
> Any thoughts on the other four items?

Generally speaking, I'm not sure it's worth the effort to do the input 
validation. All of the cases you suggest ruling out do indeed seem to be 
insane, but someone may have defined a subclass that does something based on 
the current behaviour.

Given the timing, I suggest just documenting and testing the current behaviour 
for Python 2.5, and then creating a tracker item targeting 2.6 to cover the 
lack of sanity checks (and whether or not more should be added).

Some of the problems have multiple possible solutions (e.g., using 
inspect.getargspec() to check for zero-argument callables when searching for 
test methods, instead of trying to call everything unittest can lay its hands 
on) so we need more time to think about them.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From jjl at pobox.com  Sun Aug 20 02:27:10 2006
From: jjl at pobox.com (John J Lee)
Date: Sun, 20 Aug 2006 00:27:10 +0000 (UTC)
Subject: [Python-Dev] String formatting / unicode 2.5 bug?
Message-ID: <Pine.LNX.4.64.0608192354450.8398@localhost>

Is this a bug?

# run with 2.4 and then with 2.5 (I'm running release25-maint:51410)
class a(object):

     def __getattribute__(self, name):
         print "accessing %r.%s" % (self, name)
         return object.__getattribute__(self, name)

     def __str__(self):
         print "__str__"
         return u'hi'

print repr(str(a()))
print
print repr("%s" % a())


John


From jackilyn at gmail.com  Sun Aug 20 02:24:59 2006
From: jackilyn at gmail.com (Jackilyn Hoxworth)
Date: Sat, 19 Aug 2006 19:24:59 -0500
Subject: [Python-Dev] SSH Key Added
Message-ID: <58940d080608191724g6596e97av754ee61e30ed5e17@mail.gmail.com>

Can I have this key added?

-- 
Jackilyn Hoxworth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060819/9da4c3a8/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jah_key.pub
Type: application/octet-stream
Size: 606 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060819/9da4c3a8/attachment.obj 

From ncoghlan at gmail.com  Sun Aug 20 07:03:21 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 20 Aug 2006 15:03:21 +1000
Subject: [Python-Dev] String formatting / unicode 2.5 bug?
In-Reply-To: <Pine.LNX.4.64.0608192354450.8398@localhost>
References: <Pine.LNX.4.64.0608192354450.8398@localhost>
Message-ID: <44E7ED19.6090907@gmail.com>

John J Lee wrote:
> Is this a bug?

I don't believe so - the string formatting documentation states that the 
result will be unicode if either the format string is unicode or any of the 
objects passed to a %s format code is unicode.

That latter part has just been extended to include any object that returns 
Unicode from __str__, instead of being restricted to actual Unicode instances.

Note that the following behaves the same way regardless of whether you use 2.4 
or 2.5:
"%s" % 'hi'
"%s" % u'hi'

And once the result has been promoted to unicode, __unicode__ is used directly:

 >>> print repr("%s%s" % (a(), a()))
__str__
accessing <__main__.a object at 0x00AF66F0>.__unicode__
__str__
accessing <__main__.a object at 0x00AF6390>.__unicode__
__str__
u'hihi'

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From python-dev at zesty.ca  Sun Aug 20 07:38:57 2006
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Sun, 20 Aug 2006 00:38:57 -0500 (CDT)
Subject: [Python-Dev] uuid module - byte order issue
In-Reply-To: <7168d65a0608041126y6f02f127of48530b752afef74@mail.gmail.com>
References: <7168d65a0608031311g3479824ds18a9ede4a33cae7e@mail.gmail.com>
	<Pine.LNX.4.58.0608041217290.32166@server1.LFW.org>
	<7168d65a0608041126y6f02f127of48530b752afef74@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0608200037380.32166@server1.LFW.org>


On Fri, 4 Aug 2006, Oren Tirosh wrote:
> Compatibility with Windows "GUIDs" may be one of the most important
> use cases for the UUID module. It's important to resolve this or users
> will have unpleasant surprises. I did.
[...]
> alternatives:
>
> 1. Default is big endian byte order. Little endian is explicit.
> 2. Default is native byte order. Little endian and big endian are explicit.
> 3. No default. Little endian and big endian are both explicit.

I implemented and committed the first alternative above (as a new
attribute, 'bytes_le'), shortly before the freeze.


-- ?!ng

From thomas at python.org  Sun Aug 20 08:05:39 2006
From: thomas at python.org (Thomas Wouters)
Date: Sun, 20 Aug 2006 08:05:39 +0200
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <20060819144042.GK5772@performancedrivers.com>
References: <ec50v9$oln$1@sea.gmane.org> <44E60B2A.6020101@v.loewis.de>
	<44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de>
	<ec60f1$a53$1@sea.gmane.org> <44E6A6E8.3000101@v.loewis.de>
	<1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com>
	<20060819144042.GK5772@performancedrivers.com>
Message-ID: <9e804ac0608192305t56fdcf8fl3a3abe37ac296df5@mail.gmail.com>

On 8/19/06, Jack Diederich <jack at psynchronous.com> wrote:

> It has always "just worked" for me on Opterons + Debian.
> Python 2.4 (#1, May 31 2005, 10:19:45)
> [GCC 3.3.5 (Debian 1:3.3.5-12)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.maxint
> 9223372036854775807


While that's certainly useful behaviour, it isn't exactly the same thing as
the 64-bit support in Python 2.5. 64-bit longs have always worked (as Tim
said, CPython is good C), but large parts of CPython were using ints instead
of longs -- and I'm fairly certain there are still quite a few bugs with
container types and more than 31-bits worth of elements. (I say that because
I found more than I hoped for, writing the bigmem tests a few months back.
And those tests only consider bytestrings, lists and tuples.) So, while
sys.maxint doesn't change, and any container with sane amounts of elements
will almost certainly work (one would hope that's tested enough by now),
real-world code that actually uses, say, more than 100Gb worth of memory in
a single container may still break. Unless we write more bigmem tests :>

Luxury-problem-anyone?'ly y'rs,
-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060820/401fd8ee/attachment.htm 

From fredrik at pythonware.com  Sun Aug 20 10:46:55 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 20 Aug 2006 10:46:55 +0200
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <44E6A6E8.3000101@v.loewis.de>
References: <ec50v9$oln$1@sea.gmane.org>	<44E60B2A.6020101@v.loewis.de>	<44E60D00.7090304@python.net>	<44E635A2.3020000@v.loewis.de>	<ec60f1$a53$1@sea.gmane.org>
	<44E6A6E8.3000101@v.loewis.de>
Message-ID: <ec97hv$12d$1@sea.gmane.org>

Martin v. L?wis wrote:

> It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess)

make that "over a decade".  the first Python system I built was on 
tru64, back in 1995 (portions of the the initial prototype was written 
on a 286 box under MS-DOS, but the bulk was developed on tru64 and 
deployed on tru64...)

</F>


From jjl at pobox.com  Sun Aug 20 14:45:08 2006
From: jjl at pobox.com (John J Lee)
Date: Sun, 20 Aug 2006 12:45:08 +0000 (UTC)
Subject: [Python-Dev] String formatting / unicode 2.5 bug?
In-Reply-To: <44E7ED19.6090907@gmail.com>
References: <Pine.LNX.4.64.0608192354450.8398@localhost>
	<44E7ED19.6090907@gmail.com>
Message-ID: <Pine.LNX.4.64.0608201151320.8426@localhost>

On Sun, 20 Aug 2006, Nick Coghlan wrote:

> John J Lee wrote:
>>  Is this a bug?
>
> I don't believe so - the string formatting documentation states that the 
> result will be unicode if either the format string is unicode or any of the 
> objects passed to a %s format code is unicode.
>
> That latter part has just been extended to include any object that returns 
> Unicode from __str__, instead of being restricted to actual Unicode 
> instances.
>
> Note that the following behaves the same way regardless of whether you use 
> 2.4 or 2.5:
> "%s" % 'hi'
> "%s" % u'hi'

Given that, the following wording should be changed:

http://docs.python.org/lib/typesseq-strings.html

Conversion  Meaning                                           Notes
...
s           String (converts any python object using str()).  (4)
...
(4) If the object or format provided is a unicode string, the resulting 
string will also be unicode.


The note (4) says that the result will be unicode, but it doesn't say how, 
in this case, that comes about.  This case is confusing because the docs 
claim string formatting with %s "converts ... using str()", and yet 
str(a()) returns a bytestring.  Does it *really* use str, or just __str__? 
Surely the latter? (given the observed behaviour, and not reading the C 
source)


FWIW, this change broke epydoc (fails with an AssertionError -- so perhaps 
without the assert it would still "work", dunno).


> And once the result has been promoted to unicode, __unicode__ is used 
> directly:
>
>> > >  print repr("%s%s" % (a(), a()))
> __str__
> accessing <__main__.a object at 0x00AF66F0>.__unicode__
> __str__
> accessing <__main__.a object at 0x00AF6390>.__unicode__
> __str__
> u'hihi'

I don't understand this part.  Why is __unicode__ called?  Your example 
doesn't appear to show this happening "once [i.e., because?] the result 
has been promoted to unicode" -- if that were true, it would "stand to 
reason" <wink> that the interpreter would then conclude it should call
__unicode__ for all remaining %s, and not bother with __str__.  If OTOH 
__unicode__ is called because __str__ returned a unicode object, it makes
(very slightly) more sense that it goes through the same 
__str__-then-__unicode__ rigmarole for each object on the RHS of the %.

But none of that seems to make a huge amount of sense.  I've now found the 
September 2004 discussion of this, and I'm none the wiser.


John


From guido at python.org  Sun Aug 20 17:24:16 2006
From: guido at python.org (Guido van Rossum)
Date: Sun, 20 Aug 2006 08:24:16 -0700
Subject: [Python-Dev] What should the focus for 2.6 be?
Message-ID: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>

I've been thinking a bit about a focus for the 2.6 release.

We are now officially starting parallel development of 2.6 and 3.0. I
really don't expect that we'll be able to merge the easily into the
3.0 branch much longer, so effectively 3.0 will be a fork of 2.5.

I wonder if it would make sense to focus in 2.6 on making porting of
2.6 code to 3.0 easier, rather than trying to introduce new features
in 2.6. We've done releases without new language features before;
notable 2.3 didn't add anything new (except making a few __future__
imports redundant) and concentrated on bugfixes, performance, and
library additions.

Some projects that could be undertaken in 2.6:

- add warnings when apply() is used
- add warnings when string exceptions or non-BaseException-derived
exceptions are used (this is already planned in PEP 252, which has a
very specific roll-out plan)
- add warnings when has_key() is used
- add warnings when the result of dict.keys(), .values(), .items() is
used for anything else than iterating over it
- a warning if a class defines has_key() but not __contains__().
- add __contains__ to dbm and gdbm
- add warnings to modules and built-ins that will die in 3.0

Some of these warnings should be suppressed by default, but enabled by
a command line option. We should also do the work on the standard
library to avoid the warnings: get rid of apply(), use 'x in d'
instead of 'd.has_key(x)', etc. I've recently done some of this work
in the 3.0 branch (e.g. dbm/gdbm are fresh in my memory).

Another area that could use a lot of work (and from which 3.0 could
also benefit directly) is converting all unit tests to using either
unittest.py or doctest.py. There are still at least 40 tests written
in the old "compare the output with one we prepared earlier" style.

Of course, if people are chomping at the bit to implement certain new
features (and those features are generally approved of) then I don't
want to stop them; but I would recommend that our effort may better be
focused on smoothing the 2.6/3.0 transition rather than looking for
new features to add to 2.6.

I am often approached by people who object against this or that
feature proposal not because they dislike the proposed feature in
particular, but because they believe Python is already large enough,
and they worry that if we keep adding features at the current pace, it
will soon become too unwieldy, and hence harder to learn or to keep in
one's brain. I am very sympathetic to this concern (and you should be
too!). This is one of the reasons that so much of the Python 3000 work
is about ripping out old stuff and simplifying / unifiying things.
Dropping two common data types (long and unicode -- of course they
will really be merged into their simpler counterparts int and str, but
it means much less to learn/remember) is one example. Ripping out
classic classes and string exceptions another. Removing dead library
modules a third.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Sun Aug 20 17:32:35 2006
From: barry at python.org (Barry Warsaw)
Date: Sun, 20 Aug 2006 11:32:35 -0400
Subject: [Python-Dev] What should the focus for 2.6 be?
In-Reply-To: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
References: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
Message-ID: <692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Aug 20, 2006, at 11:24 AM, Guido van Rossum wrote:

> I wonder if it would make sense to focus in 2.6 on making porting of
> 2.6 code to 3.0 easier, rather than trying to introduce new features
> in 2.6. We've done releases without new language features before;
> notable 2.3 didn't add anything new (except making a few __future__
> imports redundant) and concentrated on bugfixes, performance, and
> library additions.

+1, and there are other benefits to this approach too.

First, the pace of change appears to slow, which addresses another  
source of complaints.  Because instead of a slew of new features  
every 18 months, we really see that slew only every three years, with  
a stabilizing and bug fixing release in between.  Another benefit is  
that with a de-emphasis on new features, we can spend more time  
improving the library and documentation.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBROiAk3EjvBPtnXfVAQKeMQP/QBfHJexDveKoj/nfjRjic3+HBvTupkoA
bga7GmLV7Rn14AoHf+L6n3IhKkE1sIVXwzUmEoOeVN74h3trJSAeEYPjCF7Vt3//
3fZ4SgAlEy3nsOwRYufUtyYU9r36H7Fn7dKTtj+hJCVAzZdAOERy8ZMAEoSOw+Q4
vNfudLPznDQ=
=7D1V
-----END PGP SIGNATURE-----

From rganesan at myrealbox.com  Sun Aug 20 17:35:11 2006
From: rganesan at myrealbox.com (Ganesan Rajagopal)
Date: Sun, 20 Aug 2006 21:05:11 +0530
Subject: [Python-Dev] ctypes and win64
References: <ec50v9$oln$1@sea.gmane.org> <44E60B2A.6020101@v.loewis.de>
	<44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de>
	<ec60f1$a53$1@sea.gmane.org> <44E6A6E8.3000101@v.loewis.de>
	<ec97hv$12d$1@sea.gmane.org>
Message-ID: <xkvhoduf1kuo.fsf@grajagop-lnx.cisco.com>

>>>>> Fredrik Lundh <fredrik at pythonware.com> writes:

> Martin v. L?wis wrote:
>> It isn't. Python ran on 64-bit Alpha for nearly a decade now (I guess)

> make that "over a decade".  the first Python system I built was on 
> tru64, back in 1995 (portions of the the initial prototype was written 
> on a 286 box under MS-DOS, but the bulk was developed on tru64 and 
> deployed on tru64...)

Well, there's 64-bit support and then there's 64-bit support. While all Unix
or Unix like OSs follow LP64 model, I believe Win64 follows a P64 model. So
it's a whole new ball game.

Ganesan

-- 
Ganesan Rajagopal


From nas at arctrix.com  Sun Aug 20 18:39:29 2006
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 20 Aug 2006 16:39:29 +0000 (UTC)
Subject: [Python-Dev] String formatting / unicode 2.5 bug?
References: <Pine.LNX.4.64.0608192354450.8398@localhost>
	<44E7ED19.6090907@gmail.com>
	<Pine.LNX.4.64.0608201151320.8426@localhost>
Message-ID: <eca381$4oq$1@sea.gmane.org>

John J Lee <jjl at pobox.com> wrote:
> The note (4) says that the result will be unicode, but it doesn't say how, 
> in this case, that comes about.  This case is confusing because the docs 
> claim string formatting with %s "converts ... using str()", and yet 
> str(a()) returns a bytestring.  Does it *really* use str, or just __str__? 
> Surely the latter? (given the observed behaviour, and not reading the C 
> source)

It uses __str__ and confirms that the returned object is a 'str' or
'unicode'.  The docs are not precise but they were not for 2.4
either.  Note the following case:

    '%s' % u'Hello!'

The operand is not forced to be a str.

  Neil


From jjl at pobox.com  Sun Aug 20 18:55:09 2006
From: jjl at pobox.com (John J Lee)
Date: Sun, 20 Aug 2006 16:55:09 +0000 (UTC)
Subject: [Python-Dev] String formatting / unicode 2.5 bug?
In-Reply-To: <eca381$4oq$1@sea.gmane.org>
References: <Pine.LNX.4.64.0608192354450.8398@localhost>
	<44E7ED19.6090907@gmail.com>
	<Pine.LNX.4.64.0608201151320.8426@localhost>
	<eca381$4oq$1@sea.gmane.org>
Message-ID: <Pine.LNX.4.64.0608201652120.8499@localhost>

On Sun, 20 Aug 2006, Neil Schemenauer wrote:

> John J Lee <jjl at pobox.com> wrote:
>> The note (4) says that the result will be unicode, but it doesn't say how,
>> in this case, that comes about.  This case is confusing because the docs
>> claim string formatting with %s "converts ... using str()", and yet
>> str(a()) returns a bytestring.  Does it *really* use str, or just __str__?
>> Surely the latter? (given the observed behaviour, and not reading the C
>> source)
>
> It uses __str__ and confirms that the returned object is a 'str' or
> 'unicode'.  The docs are not precise but they were not for 2.4
> either.  Note the following case:
[...]

OK, but I assume you're not saying that the fact that the docs were broken 
in 2.4 implies they shouldn't be fixed now?

I would suggest revised wording, but I'm clearly confused about what 
actually goes on under the hood...


John


From aleaxit at gmail.com  Mon Aug 21 00:08:11 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Sun, 20 Aug 2006 15:08:11 -0700
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <44E6E7BD.8000903@holdenweb.com>
References: <ec50v9$oln$1@sea.gmane.org> <44E60B2A.6020101@v.loewis.de>	
	<44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de>	
	<ec60f1$a53$1@sea.gmane.org> <44E6A6E8.3000101@v.loewis.de>
	<1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com>
	<44E6E7BD.8000903@holdenweb.com>
Message-ID: <A69DA3E5-5202-4F67-9B3D-F3FC1DF86BEC@gmail.com>


On Aug 19, 2006, at 3:28 AM, Steve Holden wrote:
    ...
> It's going to be very interesting to see what comes out of the Google
> sprints. I am sure the 64-bitters will be out in force, so there'll be

Hmmm, we'll be working on our laptops, as is typical of sprints, so  
I'm not sure how many 64-bit machines will in fact be around -- we'll  
see.

> useful data about any such problems. <irony wink="0.97">64 bits is,  
> I am
> sure, as much as anyone is ever going to need, so the pain will  
> finally
> be over.</irony>
>
> It's good to see the idea of industry support for open source sprints
> taking off. Tomorrow, the world ... :-)

Sprints are indeed a fascinating idea and have proven they work, in  
an open-source context -- I do wonder if they could be made to work  
in other contexts, and I'm sure many others are wondering similarly.


Alex




From thomas at python.org  Mon Aug 21 03:31:45 2006
From: thomas at python.org (Thomas Wouters)
Date: Mon, 21 Aug 2006 03:31:45 +0200
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <A69DA3E5-5202-4F67-9B3D-F3FC1DF86BEC@gmail.com>
References: <ec50v9$oln$1@sea.gmane.org> <44E60B2A.6020101@v.loewis.de>
	<44E60D00.7090304@python.net> <44E635A2.3020000@v.loewis.de>
	<ec60f1$a53$1@sea.gmane.org> <44E6A6E8.3000101@v.loewis.de>
	<1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com>
	<44E6E7BD.8000903@holdenweb.com>
	<A69DA3E5-5202-4F67-9B3D-F3FC1DF86BEC@gmail.com>
Message-ID: <9e804ac0608201831q2b1f48e0qcb4164fd8ab927dd@mail.gmail.com>

On 8/21/06, Alex Martelli <aleaxit at gmail.com> wrote:
>
>
> On Aug 19, 2006, at 3:28 AM, Steve Holden wrote:
>     ...
> > It's going to be very interesting to see what comes out of the Google
> > sprints. I am sure the 64-bitters will be out in force, so there'll be
>
> Hmmm, we'll be working on our laptops, as is typical of sprints, so
> I'm not sure how many 64-bit machines will in fact be around -- we'll
> see.
>

FWIW, I have access to a pair of AMD64 machines with 16Gb RAM and several
hundred Gb swap. I can't give anyone else access, but I can run tests and
debug.


> Sprints are indeed a fascinating idea and have proven they work, in
> an open-source context -- I do wonder if they could be made to work
> in other contexts, and I'm sure many others are wondering similarly.
>

We've had commercial, private "sprints" at XS4ALL for years, every 6 months
or so, until last year. We didn't call them sprints and they obviously
weren't as freeform as the Python sprints, but the concept was the same (as
well as the atmosphere.) We stopped having them because the number of
concurrent projects was consistently larger than the number of programmers,
which makes the sprinting setup somewhat useless.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060821/8571fe76/attachment-0001.htm 

From alexander.belopolsky at gmail.com  Mon Aug 21 06:55:07 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 21 Aug 2006 00:55:07 -0400
Subject: [Python-Dev] A cast from Py_ssize_t to long
In-Reply-To: <44E174EB.1080305@v.loewis.de>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<44E174EB.1080305@v.loewis.de>
Message-ID: <6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local>


On Aug 15, 2006, at 3:16 AM, Martin v. L?wis wrote:

>
> Where does it assume that it is safe to case ssize_t -> long?
> That would be a bug.

Is this a bug?

file_readinto(PyFileObject *f, PyObject *args)
{
...
         Py_ssize_t ndone, nnow;
...
         return PyInt_FromLong((long)ndone);
}

See Objects/fileobject.c (revision 51420).


From fredrik at pythonware.com  Mon Aug 21 08:17:32 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 21 Aug 2006 08:17:32 +0200
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <A69DA3E5-5202-4F67-9B3D-F3FC1DF86BEC@gmail.com>
References: <ec50v9$oln$1@sea.gmane.org>
	<44E60B2A.6020101@v.loewis.de>		<44E60D00.7090304@python.net>
	<44E635A2.3020000@v.loewis.de>		<ec60f1$a53$1@sea.gmane.org>
	<44E6A6E8.3000101@v.loewis.de>	<1f7befae0608190219m5429d505i8a155e943e76d3fd@mail.gmail.com>	<44E6E7BD.8000903@holdenweb.com>
	<A69DA3E5-5202-4F67-9B3D-F3FC1DF86BEC@gmail.com>
Message-ID: <ecbj5s$i4d$1@sea.gmane.org>

Alex Martelli wrote:

> Sprints are indeed a fascinating idea and have proven they work, in  
> an open-source context -- I do wonder if they could be made to work  
> in other contexts, and I'm sure many others are wondering similarly.

"war room" development and other colocation approaches are nothing new, 
and works very well in many contexts.

</F>


From talin at acm.org  Mon Aug 21 08:20:34 2006
From: talin at acm.org (Talin)
Date: Sun, 20 Aug 2006 23:20:34 -0700
Subject: [Python-Dev] What should the focus for 2.6 be?
In-Reply-To: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
References: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
Message-ID: <44E950B2.4060305@acm.org>

Guido van Rossum wrote:
> I've been thinking a bit about a focus for the 2.6 release.
> 
> We are now officially starting parallel development of 2.6 and 3.0. I
> really don't expect that we'll be able to merge the easily into the
> 3.0 branch much longer, so effectively 3.0 will be a fork of 2.5.
> 
> I wonder if it would make sense to focus in 2.6 on making porting of
> 2.6 code to 3.0 easier, rather than trying to introduce new features
> in 2.6. We've done releases without new language features before;
> notable 2.3 didn't add anything new (except making a few __future__
> imports redundant) and concentrated on bugfixes, performance, and
> library additions.

I've been thinking about the transition to unicode strings, and I want 
to put forward a notion that might allow the transition to be done 
gradually instead of all at once.

The idea would be to temporarily introduce a new name for 8-bit strings 
- let's call it "ascii". An "ascii" object would be exactly the same as 
today's 8-bit strings.

The 'str' builtin symbol would be assigned to 'ascii' by default, but 
you could assign it to 'unicode' if you wanted to default to wide strings:

    str = ascii   # Selects 8-bit strings by default
    str = unicode # Selects unicode strings by default

In order to make the transition, what you would do is to temporarily 
undefine the 'str' symbol from the code base - in other words, remove 
'str' from the builtin namespace, and then migrate all of the code -- 
replacing any library reference to 'str' with a reference to 'ascii' 
*or* updating that function to deal with unicode strings. Once you get 
all of the unit tests running again, you can re-introduce 'str', but now 
you know that since none of the libraries refer to 'str' directly, you 
can safely change its definition.

All of this could be done while retaining compatibility with existing 
3rd party code - as long as 'str = ascii' is defined. So you turn it on 
to run your Python programs, and turn it off when you want to work on 
3.0 migration.

The next step (which would not be backwards compatible) would be to 
gradually remove 'ascii' from the code base -- wherever that name 
occurs, it would be a signal that the function needs to be updated to 
use 'unicode' instead.

Finally, once the last occurance of 'ascii' is removed, the final step 
is to do a search and replace of all occurances of 'unicode' with 'str'.

I know this seems round-about, and is more work than doing it all in one 
shot. However, I know from past experience that the trickiest part of 
doing a pervasive change to a code base like this is just keeping track 
of what parts have been migrated and what parts have not. Many times in 
the past I've changed the definition of a ubiquitous type by temporarily 
renaming it, thus vacating the old name so that it can be defined anew, 
without conflict.

-- Talin

From g.brandl at gmx.net  Mon Aug 21 10:56:33 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 21 Aug 2006 10:56:33 +0200
Subject: [Python-Dev] What should the focus for 2.6 be?
In-Reply-To: <44E950B2.4060305@acm.org>
References: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
	<44E950B2.4060305@acm.org>
Message-ID: <ecbsg2$c4p$1@sea.gmane.org>

Talin wrote:
> Guido van Rossum wrote:
>> I've been thinking a bit about a focus for the 2.6 release.
>> 
>> We are now officially starting parallel development of 2.6 and 3.0. I
>> really don't expect that we'll be able to merge the easily into the
>> 3.0 branch much longer, so effectively 3.0 will be a fork of 2.5.
>> 
>> I wonder if it would make sense to focus in 2.6 on making porting of
>> 2.6 code to 3.0 easier, rather than trying to introduce new features
>> in 2.6. We've done releases without new language features before;
>> notable 2.3 didn't add anything new (except making a few __future__
>> imports redundant) and concentrated on bugfixes, performance, and
>> library additions.
> 
> I've been thinking about the transition to unicode strings, and I want 
> to put forward a notion that might allow the transition to be done 
> gradually instead of all at once.
> 
> The idea would be to temporarily introduce a new name for 8-bit strings 
> - let's call it "ascii". An "ascii" object would be exactly the same as 
> today's 8-bit strings.
> 
> The 'str' builtin symbol would be assigned to 'ascii' by default, but 
> you could assign it to 'unicode' if you wanted to default to wide strings:
> 
>     str = ascii   # Selects 8-bit strings by default
>     str = unicode # Selects unicode strings by default

This doesn't change the type of string literals.

Georg


From ncoghlan at gmail.com  Mon Aug 21 11:37:13 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 21 Aug 2006 19:37:13 +1000
Subject: [Python-Dev] String formatting / unicode 2.5 bug?
In-Reply-To: <Pine.LNX.4.64.0608201151320.8426@localhost>
References: <Pine.LNX.4.64.0608192354450.8398@localhost>	<44E7ED19.6090907@gmail.com>
	<Pine.LNX.4.64.0608201151320.8426@localhost>
Message-ID: <44E97EC9.4040502@gmail.com>

John J Lee wrote:
>> And once the result has been promoted to unicode, __unicode__ is used 
>> directly:
>>
>>>>>  print repr("%s%s" % (a(), a()))
>> __str__
>> accessing <__main__.a object at 0x00AF66F0>.__unicode__
>> __str__
>> accessing <__main__.a object at 0x00AF6390>.__unicode__
>> __str__
>> u'hihi'
> 
> I don't understand this part.  Why is __unicode__ called?  Your example 
> doesn't appear to show this happening "once [i.e., because?] the result 
> has been promoted to unicode" -- if that were true, it would "stand to 
> reason" <wink> that the interpreter would then conclude it should call
> __unicode__ for all remaining %s, and not bother with __str__.

It does try to call unicode directly, but because the example object doesn't 
supply __unicode__ it ends up falling back to __str__ instead. The behaviour 
is clearer when the example object provides both methods:

 >>> # Example (2.5b3)
... class a(object):
...      def __str__(self):
...          print "running __str__"
...          return u'hi'
...      def __unicode__(self):
...          print "running __unicode__"
...          return u'hi'
...
 >>> print repr("%s%s" % (a(), a()))
running __str__
running __unicode__
running __unicode__
u'hihi'

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From thomas at python.org  Mon Aug 21 13:23:46 2006
From: thomas at python.org (Thomas Wouters)
Date: Mon, 21 Aug 2006 13:23:46 +0200
Subject: [Python-Dev] A cast from Py_ssize_t to long
In-Reply-To: <6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<44E174EB.1080305@v.loewis.de>
	<6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local>
Message-ID: <9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com>

On 8/21/06, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:
>
>
> On Aug 15, 2006, at 3:16 AM, Martin v. L?wis wrote:
>
> >
> > Where does it assume that it is safe to case ssize_t -> long?
> > That would be a bug.
>
> Is this a bug?
>
> file_readinto(PyFileObject *f, PyObject *args)
> {
> ...
>          Py_ssize_t ndone, nnow;
> ...
>          return PyInt_FromLong((long)ndone);
> }
>
> See Objects/fileobject.c (revision 51420).


Yes. Py_ssize_t can be bigger than a long (on LLP64 systems, such as Win64).
It doesn't matter on other systems, and you have to read more than 31 bits
worth of data to detect it even on Win64, but it's still a bug.
file_readinto should be using PyInt_FromSsize_t() instead. (There is the
SAFE_DOWNCAST macro for cases where we know the *value* of the (s)size_t
will always fit in a long, on any supported system, but that isn't the case
here.)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060821/4e09574e/attachment.htm 

From jimjjewett at gmail.com  Mon Aug 21 16:03:46 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 21 Aug 2006 10:03:46 -0400
Subject: [Python-Dev] Questions on unittest behaviour
Message-ID: <fb6fbf560608210703i1f0859a9jb60745d536b3be85@mail.gmail.com>

Collin Winter:

> [ improvements to unittest]

These would all have been (mostly) reasonable when the module was
first added to the standard library.  But it has been there as an ugly
fragile unchanging beast for several years, and is ironically not well
endowed with unit tests.  I therefore think it might be too late for
these changes.

> 2) The TestLoader.testMethodPrefix attribute currently allows anything
> to be assigned to it, including invalid objects and the empty string.
>  While the former will cause errors ... you get things like assertEqual(),
> failUnlessEqual(), etc, when TestLoader.loadTestsFromTestCase() is run.

It makes the interface ugly, and learning harder, but for code this
old, either people catch it with the first actual run of their tests
suites, or a subclass is handling it (perhaps to trigger a default, or
to call functions through a foreign interface).

> 3) TestLoader.loadTestsFromTestCase() accepts objects that are not
> test cases and will happily look for appropriately-named methods on
> any object you give it. This flexibility should be documented, or
> proper input validation should be done (a bug fix for 2.5).

People are probably using this, so I think a deprecation in 2.6 is the
fastest it should change.

> 4) TestLoader.loadTestsFromName() (and by extension,
> loadTestsFromNames(), too) raises an AttributeError if the name is the
> empty string because -- as it correctly asserts -- the object does not
> contain an attribute named ''. I recommend that this be tested for and
> ValueError be raised (bug fix for 2.5).

This seems reasonable for new code, but not this late in 2.5 -- if
people are going through an auto-generated list of attributes, they
may well be catching AttributeError and not ValueError.

> 5) When TestLoader.loadTestsFrom{Name,Names}() are given a name that
> resolves to a classmethod on a TestCase subclass, the method is not
> invoked.  ...
> It is not documented which of these tests takes priority: is the
> classmethod "a test method within a test case class" or is it a
> callable? The same issue applies to staticmethods as well.

Documentation probably should get added.

-jJ

From alexander.belopolsky at gmail.com  Mon Aug 21 16:08:02 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 21 Aug 2006 10:08:02 -0400
Subject: [Python-Dev] A cast from Py_ssize_t to long
In-Reply-To: <9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<44E174EB.1080305@v.loewis.de>
	<6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local>
	<9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com>
Message-ID: <d38f5330608210708t18d5eedapcc8816874d1b966@mail.gmail.com>

On 8/21/06, Thomas Wouters <thomas at python.org> wrote:
[snip]
> > Is this a bug?
[snap]
> Yes. Py_ssize_t can be bigger than a long (on LLP64 systems, such as Win64).

Here is a similar problem:
typedef struct {
   ...
   size_t  pos;
   ...
} mmap_object;
...
mmap_tell_method(mmap_object *self, PyObject *unused)
{
        CHECK_VALID(NULL);
        return PyInt_FromLong((long) self->pos);
}

See Modules/mmapmodule.c .

Here a cast to ssize_t would, technically speaking, not be safe
either, but it may be worth using ssize_t anyways.

Is there a simple automated way to detect situations like this? Maybe
there is a win64 compiler that would generate a warning.

From thomas at python.org  Mon Aug 21 16:34:25 2006
From: thomas at python.org (Thomas Wouters)
Date: Mon, 21 Aug 2006 16:34:25 +0200
Subject: [Python-Dev] A cast from Py_ssize_t to long
In-Reply-To: <d38f5330608210708t18d5eedapcc8816874d1b966@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<44E174EB.1080305@v.loewis.de>
	<6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local>
	<9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com>
	<d38f5330608210708t18d5eedapcc8816874d1b966@mail.gmail.com>
Message-ID: <9e804ac0608210734m7786148ak34e3610d1523f8d0@mail.gmail.com>

On 8/21/06, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:

> Here is a similar problem:
> typedef struct {
>    ...
>    size_t  pos;
>    ...
> } mmap_object;
> ...
> mmap_tell_method(mmap_object *self, PyObject *unused)
> {
>         CHECK_VALID(NULL);
>         return PyInt_FromLong((long) self->pos);
> }
>
> See Modules/mmapmodule.c .
>
> Here a cast to ssize_t would, technically speaking, not be safe
> either, but it may be worth using ssize_t anyways.


It should call PyInt_FromSize_t, without any casting. That will make it a
PyLong if it's bigger than a Py_ssize_t, too.

Is there a simple automated way to detect situations like this? Maybe
> there is a win64 compiler that would generate a warning.
>

I doubt it. Explicit casts are meant to silence warnings (among other
things.) Warning for all casts is bound to generate quite a lot of warnings.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060821/3f0f7968/attachment.htm 

From alexander.belopolsky at gmail.com  Mon Aug 21 16:47:35 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Mon, 21 Aug 2006 10:47:35 -0400
Subject: [Python-Dev] A cast from Py_ssize_t to long
In-Reply-To: <9e804ac0608210734m7786148ak34e3610d1523f8d0@mail.gmail.com>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<44E174EB.1080305@v.loewis.de>
	<6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local>
	<9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com>
	<d38f5330608210708t18d5eedapcc8816874d1b966@mail.gmail.com>
	<9e804ac0608210734m7786148ak34e3610d1523f8d0@mail.gmail.com>
Message-ID: <E467224E-CE36-4381-B447-C11B398CF89A@local>


On Aug 21, 2006, at 10:34 AM, Thomas Wouters wrote:

>
> Is there a simple automated way to detect situations like this? Maybe
> there is a win64 compiler that would generate a warning.
>
> I doubt it. Explicit casts are meant to silence warnings (among  
> other things.) Warning for all casts is bound to generate quite a  
> lot of warnings.
>

There are also cases of implicit casts like this  that were not  
caught so far:

static Py_ssize_t
mmap_buffer_getreadbuf(mmap_object *self, Py_ssize_t index, const  
void **ptr)
{
  ...
         return self->size;
}

static Py_ssize_t
mmap_buffer_getwritebuf(mmap_object *self, Py_ssize_t index, const  
void **ptr)
{
...
         return self->size;
}

I don't have any system with sizeof(size_t) != sizeof(long), but it  
maybe worth the effort to review the warnings on such system.



From thomas at python.org  Mon Aug 21 16:56:23 2006
From: thomas at python.org (Thomas Wouters)
Date: Mon, 21 Aug 2006 16:56:23 +0200
Subject: [Python-Dev] A cast from Py_ssize_t to long
In-Reply-To: <E467224E-CE36-4381-B447-C11B398CF89A@local>
References: <d38f5330608141608o7c748a20ka1daa2504896b213@mail.gmail.com>
	<44E174EB.1080305@v.loewis.de>
	<6A93E89B-F2F6-4D5B-AACB-2EDC09A1D2CB@local>
	<9e804ac0608210423r22541508q35f54a444b2ffab9@mail.gmail.com>
	<d38f5330608210708t18d5eedapcc8816874d1b966@mail.gmail.com>
	<9e804ac0608210734m7786148ak34e3610d1523f8d0@mail.gmail.com>
	<E467224E-CE36-4381-B447-C11B398CF89A@local>
Message-ID: <9e804ac0608210756r1d9806efo84ed12a9848c1644@mail.gmail.com>

On 8/21/06, Alexander Belopolsky <alexander.belopolsky at gmail.com> wrote:

> There are also cases of implicit casts like this  that were not
> caught so far:
>
> static Py_ssize_t
> mmap_buffer_getreadbuf(mmap_object *self, Py_ssize_t index, const
> void **ptr)
> {
>   ...
>          return self->size;
> }
>
> static Py_ssize_t
> mmap_buffer_getwritebuf(mmap_object *self, Py_ssize_t index, const
> void **ptr)
> {
> ...
>          return self->size;
> }
>
> I don't have any system with sizeof(size_t) != sizeof(long), but it
> maybe worth the effort to review the warnings on such system.


GCC on a LP64 machine does not generate warnings for the above code. It
doesn't have anything to do with 64-bit or 32-bit anyway, since Py_ssize_t
and size_t are supposed to be the same size. They just different in
signedness, and that's the case on any system. Even if those functions were
defined to return longs instead of Py_ssize_t's, GCC wouldn't generate a
warning; it's a valid (implicit) downcast that might lose bits. I believe
there's a Windows compiler that goes warn for such cases, but if so, I'm
sure it generates a whole lot of spurious ones at the moment.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060821/f6bef468/attachment.htm 

From guido at python.org  Mon Aug 21 20:11:50 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 21 Aug 2006 11:11:50 -0700
Subject: [Python-Dev] What should the focus for 2.6 be?
In-Reply-To: <44E950B2.4060305@acm.org>
References: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
	<44E950B2.4060305@acm.org>
Message-ID: <ca471dc20608211111r467d6a18sf3df2579b54ab988@mail.gmail.com>

I'll keep this in mind -- with the caveat that Georg mentioned.

For the next 96 hours I'm going to be severely limited in bandwidth
due to the physical requirements of the sprint at Google. I'd
appreciate not receiving too much email during this period...

--Guido

On 8/20/06, Talin <talin at acm.org> wrote:
> Guido van Rossum wrote:
> > I've been thinking a bit about a focus for the 2.6 release.
> >
> > We are now officially starting parallel development of 2.6 and 3.0. I
> > really don't expect that we'll be able to merge the easily into the
> > 3.0 branch much longer, so effectively 3.0 will be a fork of 2.5.
> >
> > I wonder if it would make sense to focus in 2.6 on making porting of
> > 2.6 code to 3.0 easier, rather than trying to introduce new features
> > in 2.6. We've done releases without new language features before;
> > notable 2.3 didn't add anything new (except making a few __future__
> > imports redundant) and concentrated on bugfixes, performance, and
> > library additions.
>
> I've been thinking about the transition to unicode strings, and I want
> to put forward a notion that might allow the transition to be done
> gradually instead of all at once.
>
> The idea would be to temporarily introduce a new name for 8-bit strings
> - let's call it "ascii". An "ascii" object would be exactly the same as
> today's 8-bit strings.
>
> The 'str' builtin symbol would be assigned to 'ascii' by default, but
> you could assign it to 'unicode' if you wanted to default to wide strings:
>
>     str = ascii   # Selects 8-bit strings by default
>     str = unicode # Selects unicode strings by default
>
> In order to make the transition, what you would do is to temporarily
> undefine the 'str' symbol from the code base - in other words, remove
> 'str' from the builtin namespace, and then migrate all of the code --
> replacing any library reference to 'str' with a reference to 'ascii'
> *or* updating that function to deal with unicode strings. Once you get
> all of the unit tests running again, you can re-introduce 'str', but now
> you know that since none of the libraries refer to 'str' directly, you
> can safely change its definition.
>
> All of this could be done while retaining compatibility with existing
> 3rd party code - as long as 'str = ascii' is defined. So you turn it on
> to run your Python programs, and turn it off when you want to work on
> 3.0 migration.
>
> The next step (which would not be backwards compatible) would be to
> gradually remove 'ascii' from the code base -- wherever that name
> occurs, it would be a signal that the function needs to be updated to
> use 'unicode' instead.
>
> Finally, once the last occurance of 'ascii' is removed, the final step
> is to do a search and replace of all occurances of 'unicode' with 'str'.
>
> I know this seems round-about, and is more work than doing it all in one
> shot. However, I know from past experience that the trickiest part of
> doing a pervasive change to a code base like this is just keeping track
> of what parts have been migrated and what parts have not. Many times in
> the past I've changed the definition of a ubiquitous type by temporarily
> renaming it, thus vacating the old name so that it can be defined anew,
> without conflict.
>
> -- Talin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Mon Aug 21 21:24:54 2006
From: brett at python.org (Brett Cannon)
Date: Mon, 21 Aug 2006 12:24:54 -0700
Subject: [Python-Dev] What should the focus for 2.6 be?
In-Reply-To: <692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org>
References: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
	<692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org>
Message-ID: <bbaeab100608211224y6d18539en739f9edce853c66d@mail.gmail.com>

On 8/20/06, Barry Warsaw <barry at python.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Aug 20, 2006, at 11:24 AM, Guido van Rossum wrote:
>
> > I wonder if it would make sense to focus in 2.6 on making porting of
> > 2.6 code to 3.0 easier, rather than trying to introduce new features
> > in 2.6. We've done releases without new language features before;
> > notable 2.3 didn't add anything new (except making a few __future__
> > imports redundant) and concentrated on bugfixes, performance, and
> > library additions.
>
> +1, and there are other benefits to this approach too.
>
> First, the pace of change appears to slow, which addresses another
> source of complaints.  Because instead of a slew of new features
> every 18 months, we really see that slew only every three years, with
> a stabilizing and bug fixing release in between.  Another benefit is
> that with a de-emphasis on new features, we can spend more time
> improving the library and documentation.


I think fixing tests and documentation would be a great thing to focus
2.6on.  Not glamourous, I know, but it is needed.

For tests, I hope to get some decorators and such written that will help
classify tests.  Also adding a function to denote what module is being
tested would be good (to avoid the issue of a dependent import for testing
failing and then everyone just thinking the test was skipped).  Lastly,
testing the C API using ctypes would be really good since it is not
thorougly tested.

As for the docs, they just need a thorough updating.  As to whether we
should come up with some other format for Py3K with better semantic
information and that is easier to read is another question entirely.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060821/7ee49990/attachment.htm 

From jcarlson at uci.edu  Mon Aug 21 23:21:30 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 21 Aug 2006 14:21:30 -0700
Subject: [Python-Dev] What should the focus for 2.6 be?
In-Reply-To: <44E950B2.4060305@acm.org>
References: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
	<44E950B2.4060305@acm.org>
Message-ID: <20060821081944.1A0F.JCARLSON@uci.edu>


Talin <talin at acm.org> wrote:
[snip]
> I've been thinking about the transition to unicode strings, and I want 
> to put forward a notion that might allow the transition to be done 
> gradually instead of all at once.
> 
> The idea would be to temporarily introduce a new name for 8-bit strings 
> - let's call it "ascii". An "ascii" object would be exactly the same as 
> today's 8-bit strings.

There are two parts to the unicode conversion; all literals are unicode,
and we don't have strings anymore, we have bytes.  Without offering the
bytes object, then people can't really convert their code.  String
literals can be handled with the -U command line option (and perhaps
having the interpreter do the str=unicode assignment during startup).


In any case, as I look at Py3k and the future of Python, in each release,
I ask "what are the compelling features that make me want to upgrade?"
In each of the 1.5-2.5 series that I've looked at, each has had some
compelling feature or another that has basically required that I upgrade,
or seriously consider upgrading (bugfixes for stuff that has bitten me,
new syntax that I use, significant increases in speed, etc.) .

As we approach Py3k, I again ask, "what are the compelling features?"
Wholesale breakage of anything that uses ascii strings as text or binary
data? A completely changed IO stack (requiring re-learning of everything
known about Python IO)?  Dictionary .keys(), .values(), and .items()
being their .iter*() equivalents (making it just about impossible to
optimize for Py3k dictionary behavior now)?

I understand getting rid of the cruft, really I do (you should see some
cruft I've been replacing lately). But some of that cruft is useful, or
really, some of that cruft has no alternative currently, which will
require significant rewrites of user code when Py3k is released.  When
everyone has to rewrite their code, they are going to ask, "Why don't I
just stick with the maintenance 2.x? It's going to be maintained for a
few more years yet, and I don't need to rewrite all of my disk IO,
strings in dictionary code, etc.  I will be right along with them (no
offense intended to those currently working towards py3k).

I can code defensively against buffer-sturating DOS attacks with my
socket code, but I can't code defensively to handle some (never mind all)
of the changes and incompatabilities that Py3k will bring.

Here's my suggestion: every feature, syntax, etc., that is slated for
Py3k, let us release bit by bit in the 2.x series.  That lets the 2.x
series evolve into the 3.x series in a somewhat more natural way than
the currently proposed *everything breaks*.  If it takes 1, 2, 3, or 10
more releases in the 2.x series to get to all of the 3.x features, great.
At least people will have a chance to convert, or at least write correct
code for the future.

Say 2.6 gets bytes and special factories (or a special encoding argument)
for file/socket to return bytes instead of strings, and only accept
bytes objects to .write() methods (unless an encoding on the file, etc.,
was previously given). Given these bytes objects, it may even make sense
to offer the .readinto() method that Alex B has been asking for (which
would make 3 built-in objects that could reasonably support readinto:
bytes, array, mmap).

If the IO library is available for 2.6, toss that in there, or offer it
in PyPI as an evolving library.

I would suggest pushing off the dict changes until 2.7 or later, as
there are 340+ examples of dict.keys() in the Python 2.5b2 standard
library, at least half of which are going to need to be changed to
list(dict.keys()) or otherwise.  The breakage in user code will likely
be at least as substantial.


Those are just examples that come to mind now, but I'm sure there are
others changes with similar issues.

 - Josiah


From exarkun at divmod.com  Mon Aug 21 23:38:17 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 21 Aug 2006 17:38:17 -0400
Subject: [Python-Dev] [Python-3000]  What should the focus for 2.6 be?
In-Reply-To: <20060821081944.1A0F.JCARLSON@uci.edu>
Message-ID: <20060821213817.1717.1725966885.divmod.quotient.28023@ohm>

On Mon, 21 Aug 2006 14:21:30 -0700, Josiah Carlson <jcarlson at uci.edu> wrote:
>
>Talin <talin at acm.org> wrote:
>[snip]
>> I've been thinking about the transition to unicode strings, and I want
>> to put forward a notion that might allow the transition to be done
>> gradually instead of all at once.
>>
>> The idea would be to temporarily introduce a new name for 8-bit strings
>> - let's call it "ascii". An "ascii" object would be exactly the same as
>> today's 8-bit strings.
>
>There are two parts to the unicode conversion; all literals are unicode,
>and we don't have strings anymore, we have bytes.  Without offering the
>bytes object, then people can't really convert their code.  String
>literals can be handled with the -U command line option (and perhaps
>having the interpreter do the str=unicode assignment during startup).
>

A third step would ease this transition significantly: a unicode_literals __future__ import.

>
>Here's my suggestion: every feature, syntax, etc., that is slated for
>Py3k, let us release bit by bit in the 2.x series.  That lets the 2.x
>series evolve into the 3.x series in a somewhat more natural way than
>the currently proposed *everything breaks*.  If it takes 1, 2, 3, or 10
>more releases in the 2.x series to get to all of the 3.x features, great.
>At least people will have a chance to convert, or at least write correct
>code for the future.

This really seems like the right idea.  "Shoot the moon" upgrades are
almost always worse than incremental upgrades.

The incremental path is better for everyone involved.  For developers of
Python, it gets more people using and providing feedback on the new
features being developed.  For developers with Python, it keeps the scope
of a particular upgrade more manageable, letting them developer focus on a
much smaller set of changes to be made to their application.

Jean-Paul

From ncoghlan at gmail.com  Tue Aug 22 12:11:45 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 22 Aug 2006 20:11:45 +1000
Subject: [Python-Dev] [Python-checkins] r51458 -
	peps/trunk/pep-0000.txt	peps/trunk/pep-0362.txt
In-Reply-To: <20060821235408.8E80D1E4009@bag.python.org>
References: <20060821235408.8E80D1E4009@bag.python.org>
Message-ID: <44EAD861.7010201@gmail.com>

brett.cannon wrote:
> Added: peps/trunk/pep-0362.txt
> +Relation With Other PEPs
> +========================
> +
> +"Keyword-Only Arguments [#pep-3102]_
> +------------------------------------
> +
> +If keyword-only parameters come into existence, the Parameter object
> +will require modification.  A ``keyword_only`` attribute will be added
> +that holds a boolean representing whether the parameter is
> +keyword-only or not.

Why an extra attribute instead of using "x.position is None"? Unlike 
default_value, None is not otherwise a legitimate value for the position 
attribute.


> +Where should keyword-only parameters be stored in the Signature object?
> +-----------------------------------------------------------------------
> +
> +If PEP 3102 [#pep-3102]_ ends up being accepted, there is the
> +possibility that storing keyword-only parameters in a set instead of
> +in the ``parameters`` attribute of the Signature object makes more
> +sense.  Since keyword-only parameters do not have any semantic meaning
> +in terms of their position within the signature, there is no direct
> +semantic gain in storing it in the parameter list.
> +
> +Storing keyword-only paramaters in a set makes it much more explicit
> +that keyword-only parameters have no inherent order.  It does have the
> +drawback, though, that if one wants to process all parameters at once
> +they would need to perform extra work to make sure to go through both
> +the parameter list and set.

Throwing the keyword arguments in a set still loses information that was 
present in the source code. Preserving the ordering in the signature object 
means that the author of the function can control the order in which the 
keyword arguments are displayed by introspection-based help utilities (e.g. 
putting "more important" arguments first.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From amk at amk.ca  Tue Aug 22 14:41:20 2006
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 22 Aug 2006 08:41:20 -0400
Subject: [Python-Dev] What should the focus for 2.6 be?
In-Reply-To: <bbaeab100608211224y6d18539en739f9edce853c66d@mail.gmail.com>
References: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
	<692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org>
	<bbaeab100608211224y6d18539en739f9edce853c66d@mail.gmail.com>
Message-ID: <20060822124120.GA15449@rogue.amk.ca>

On Mon, Aug 21, 2006 at 12:24:54PM -0700, Brett Cannon wrote:
> As for the docs, they just need a thorough updating.  

Michael Hudson was working on a new guide to extending/embedding
Python.  Incorporating that should be a goal for 2.6 (the document may
still need to be finished -- I'm not sure what state it's in).

The HOWTOs should be incorporated into the documentation build
process; this will mean you need both LaTeX and Docutils to build all
the documentation.  I already have a Makefile patch for this; perhaps
I should just check it into the trunk now.

--amk

From mwh at python.net  Tue Aug 22 17:26:12 2006
From: mwh at python.net (Michael Hudson)
Date: Tue, 22 Aug 2006 16:26:12 +0100
Subject: [Python-Dev] What should the focus for 2.6 be?
In-Reply-To: <20060822124120.GA15449@rogue.amk.ca> (A. M. Kuchling's message
	of "Tue, 22 Aug 2006 08:41:20 -0400")
References: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
	<692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org>
	<bbaeab100608211224y6d18539en739f9edce853c66d@mail.gmail.com>
	<20060822124120.GA15449@rogue.amk.ca>
Message-ID: <2mpsesrduz.fsf@starship.python.net>

"A.M. Kuchling" <amk at amk.ca> writes:

> On Mon, Aug 21, 2006 at 12:24:54PM -0700, Brett Cannon wrote:
>> As for the docs, they just need a thorough updating.  
>
> Michael Hudson was working on a new guide to extending/embedding
> Python.  Incorporating that should be a goal for 2.6 (the document may
> still need to be finished -- I'm not sure what state it's in).

Oh my word, even I had nearly forgotten about that.

I think the latest version is here:

   http://starship.python.net/crew/mwh/toext/

and source is here:

   http://codespeak.net/svn/user/mwh/toext/

It's certainly not even half-finished, and I'm not sure how much time
or enthusiasm I'm likely to be able to find for it in the medium term.

Cheers,
mwh

-- 
  ZAPHOD:  OK, so ten out of ten for style, but minus several million
           for good thinking, eh?
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 2

From barry at python.org  Tue Aug 22 17:33:12 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 22 Aug 2006 11:33:12 -0400
Subject: [Python-Dev] What should the focus for 2.6 be?
In-Reply-To: <2mpsesrduz.fsf@starship.python.net>
References: <ca471dc20608200824l2129308ne26e5332ce7585ce@mail.gmail.com>
	<692A12F5-78C0-43BB-8041-C4CDAF4405B6@python.org>
	<bbaeab100608211224y6d18539en739f9edce853c66d@mail.gmail.com>
	<20060822124120.GA15449@rogue.amk.ca>
	<2mpsesrduz.fsf@starship.python.net>
Message-ID: <5BFC95A4-1202-40EA-9EA9-0397DB2A5F55@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Aug 22, 2006, at 11:26 AM, Michael Hudson wrote:

> Oh my word, even I had nearly forgotten about that.
>
> I think the latest version is here:
>
>    http://starship.python.net/crew/mwh/toext/
>
> and source is here:
>
>    http://codespeak.net/svn/user/mwh/toext/
>
> It's certainly not even half-finished, and I'm not sure how much time
> or enthusiasm I'm likely to be able to find for it in the medium term.

I'd be willing to help out with this, as well as flesh out some of  
the more, er, under-documented parts of the C API.  I do a /ton/ of  
embedding/extending work for my Real Job so I think I could justify  
some work time devoted to improving things here.

I'll try to take a look at MWH's stuff, but should we go ahead and  
check it into Python's svn at some point?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBROsjuHEjvBPtnXfVAQLEcQQAiyH3LaJysEJrpbHPQhXUMBlQ/+ULDatc
D1pBUmN02jY33fg14Z8Tw7AjJ5cOWsDFeF+gh6TBXvawpvF2q/XafdmtjkjwvQpa
8KfAl5JeNHSyTTd5tcKuZ1y2mQ1XpVVIqICT8PP24l+6RyCbnRhTe6MaLUHnUegm
bSl728r3s64=
=7WIt
-----END PGP SIGNATURE-----

From brett at python.org  Tue Aug 22 19:16:01 2006
From: brett at python.org (Brett Cannon)
Date: Tue, 22 Aug 2006 10:16:01 -0700
Subject: [Python-Dev] [Python-checkins] r51458 - peps/trunk/pep-0000.txt
	peps/trunk/pep-0362.txt
In-Reply-To: <44EAD861.7010201@gmail.com>
References: <20060821235408.8E80D1E4009@bag.python.org>
	<44EAD861.7010201@gmail.com>
Message-ID: <bbaeab100608221016j23a54dc8qc04eebd0a32bc4cc@mail.gmail.com>

On 8/22/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> brett.cannon wrote:
> > Added: peps/trunk/pep-0362.txt
> > +Relation With Other PEPs
> > +========================
> > +
> > +"Keyword-Only Arguments [#pep-3102]_
> > +------------------------------------
> > +
> > +If keyword-only parameters come into existence, the Parameter object
> > +will require modification.  A ``keyword_only`` attribute will be added
> > +that holds a boolean representing whether the parameter is
> > +keyword-only or not.
>
> Why an extra attribute instead of using "x.position is None"?


Didn't think of that.  =)

Unlike
> default_value, None is not otherwise a legitimate value for the position
> attribute.
>

Makes sense.  I will talk it over with Jiwon.  Might want to rename
'position' to 'positional' then in order to make it even more obvious what
the information stores.

> +Where should keyword-only parameters be stored in the Signature object?
> > +-----------------------------------------------------------------------
> > +
> > +If PEP 3102 [#pep-3102]_ ends up being accepted, there is the
> > +possibility that storing keyword-only parameters in a set instead of
> > +in the ``parameters`` attribute of the Signature object makes more
> > +sense.  Since keyword-only parameters do not have any semantic meaning
> > +in terms of their position within the signature, there is no direct
> > +semantic gain in storing it in the parameter list.
> > +
> > +Storing keyword-only paramaters in a set makes it much more explicit
> > +that keyword-only parameters have no inherent order.  It does have the
> > +drawback, though, that if one wants to process all parameters at once
> > +they would need to perform extra work to make sure to go through both
> > +the parameter list and set.
>
> Throwing the keyword arguments in a set still loses information that was
> present in the source code. Preserving the ordering in the signature
> object
> means that the author of the function can control the order in which the
> keyword arguments are displayed by introspection-based help utilities (e.g
> .
> putting "more important" arguments first.


That's a good enough argument for me.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060822/dd37116f/attachment.htm 

From brett at python.org  Tue Aug 22 19:32:22 2006
From: brett at python.org (Brett Cannon)
Date: Tue, 22 Aug 2006 10:32:22 -0700
Subject: [Python-Dev] [Python-checkins] r51458 - peps/trunk/pep-0000.txt
	peps/trunk/pep-0362.txt
In-Reply-To: <44EAD861.7010201@gmail.com>
References: <20060821235408.8E80D1E4009@bag.python.org>
	<44EAD861.7010201@gmail.com>
Message-ID: <bbaeab100608221032y440c3654wcf2cb2631ffb10f1@mail.gmail.com>

On 8/22/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> brett.cannon wrote:
> > Added: peps/trunk/pep-0362.txt
> > +Relation With Other PEPs
> > +========================
> > +
> > +"Keyword-Only Arguments [#pep-3102]_
> > +------------------------------------
> > +
> > +If keyword-only parameters come into existence, the Parameter object
> > +will require modification.  A ``keyword_only`` attribute will be added
> > +that holds a boolean representing whether the parameter is
> > +keyword-only or not.
>
> Why an extra attribute instead of using "x.position is None"? Unlike
> default_value, None is not otherwise a legitimate value for the position
> attribute.


OK, talked over with Jiwon and the room here at the sprint and the
preference was to keep it as-is.  Reasons were it allowed the position in
the signature to be kept with the object (and thus not be reliant on the
Signature object to know its position in the actual output), and the
possibility of disparate types being in the attribute.  Plus Guido preferred
the original way.  =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060822/3233101e/attachment.html 

From jjl at pobox.com  Tue Aug 22 19:42:02 2006
From: jjl at pobox.com (John J Lee)
Date: Tue, 22 Aug 2006 18:42:02 +0100 (GMT Standard Time)
Subject: [Python-Dev] String formatting / unicode 2.5 bug?
In-Reply-To: <44E97EC9.4040502@gmail.com>
References: <Pine.LNX.4.64.0608192354450.8398@localhost>
	<44E7ED19.6090907@gmail.com>
	<Pine.LNX.4.64.0608201151320.8426@localhost>
	<44E97EC9.4040502@gmail.com>
Message-ID: <Pine.WNT.4.64.0608211442280.2404@shaolin>

On Mon, 21 Aug 2006, Nick Coghlan wrote:

> John J Lee wrote:
>>> And once the result has been promoted to unicode, __unicode__ is used 
>>> directly:
>>>
>>>>>>  print repr("%s%s" % (a(), a()))
>>> __str__
>>> accessing <__main__.a object at 0x00AF66F0>.__unicode__
>>> __str__
>>> accessing <__main__.a object at 0x00AF6390>.__unicode__
>>> __str__
>>> u'hihi'
>> 
>> I don't understand this part.  Why is __unicode__ called?  Your example 
>> doesn't appear to show this happening "once [i.e., because?] the result has 
>> been promoted to unicode" -- if that were true, it would "stand to reason" 
>> <wink> that the interpreter would then conclude it should call
>> __unicode__ for all remaining %s, and not bother with __str__.
>
> It does try to call unicode directly, but because the example object doesn't 
> supply __unicode__ it ends up falling back to __str__ instead. The behaviour 
> is clearer when the example object provides both methods:
[...]

If the interpreter is falling back from __unicode__ to __str__ (rather 
than the other way around, kind-of), that makes much more sense.  I 
understood that __unicode__ was not provided, of course -- what wasn't 
clear to me was why the interpreter was calling/accessing those 
methods/attributes in the sequence it does.  Still not sure I understand 
what the third __str__ above comes from, but until I've thought it through 
again, that's my problem.


John

From igor.bukanov at gmail.com  Tue Aug 22 15:17:20 2006
From: igor.bukanov at gmail.com (Igor Bukanov)
Date: Tue, 22 Aug 2006 15:17:20 +0200
Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary
Message-ID: <df0b33100608220617y5760c9f8q8fe713b2e1fc6c7b@mail.gmail.com>

Consider the following example:

for i in range(3):
  try:
    print i
    break
  except:
    print "Unexpected exception!"
  finally:
    print "Finally"

When executed, it naturally prints
  0
  Finally
since break does not use exceptions to transfer the control and as
such can not be stopped using catch-all exception handler.

Now consider a similar example using generators:

def gen():
  for i in range(3):
    try:
      yield i
    except:
      print "Unexpected exception!"
    finally:
      print "Finally"

for i in gen():
  print i
  break

This example prints:
  0
  Unexpected exception!
  Finally
  Exception exceptions.RuntimeError: 'generator ignored GeneratorExit'
in <generator object at 0xb7daaa8c> ignored

Suddenly with generators a program can mess with control transfer
since it uses explicit GeneratorExit which can be caught and ignored.
This is unintuitive IMO.

This example also suggests how to fix generators. One just need to
change the close method so it would cause return executed right after
the yield instead of throw.

I.e. replace the current text from
http://www.python.org/dev/peps/pep-0342/

4. Add a close() method for generator-iterators, which raises
GeneratorExit at the point where the generator was paused.  If the
generator then raises StopIteration (by exiting normally, or due to
already being closed) or GeneratorExit (by not catching the
exception), close() returns to its caller.  If the generator yields a
value, a RuntimeError is raised.  If the generator raises any other
exception, it is propagated to the caller. close() does nothing if the
generator has already exited due to an exception or normal exit.

by simpler one:

4. Add a close() method for generator-iterators, which executes normal
return at the point where the generator was paused.  If the generator
then raises StopIteration (by exiting normally, or due to already
being closed), close() returns to its caller.  If the generator yields
a value, a RuntimeError is raised.  If the generator raises any other
exception, it is propagated to the caller. close() does nothing if the
generator has already exited due to an exception or normal exit.

This not only fixes the above discrepancy between normal flow control
and generators, removes GeneratorExit and simplifies the generator
protocol, but also bring a new feature allowing to have easy to grasp
feature table of the iterator methods:

next: continue after yield
throw: raise after yield
close: return after yield

Regards, Igor

From ncoghlan at gmail.com  Wed Aug 23 13:33:09 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 23 Aug 2006 21:33:09 +1000
Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary
In-Reply-To: <df0b33100608220617y5760c9f8q8fe713b2e1fc6c7b@mail.gmail.com>
References: <df0b33100608220617y5760c9f8q8fe713b2e1fc6c7b@mail.gmail.com>
Message-ID: <44EC3CF5.8030502@gmail.com>

Igor Bukanov wrote:
> This example also suggests how to fix generators. One just need to
> change the close method so it would cause return executed right after
> the yield instead of throw.
> 
> I.e. replace the current text from
> http://www.python.org/dev/peps/pep-0342/
<snip close() = throw(GeneratorExit)>

> by simpler one:
<snip close() = externally forced return>

Simpler is in the eye of the beholder - the current close() merely uses
throw() to raise a particular kind of exception 'asynchronously' (which is
already a familiar concept due to KeyboardInterrupt). What you're suggesting
is a completely new flow control concept that doesn't exist anywhere else in
the language.

I suggested during the PEP 352 discussions that GeneratorExit should inherit
directly from BaseException, so that the fix to your example would be to
replace the bare "except:" with "except Exception:" (because it can swallow
KeyboardInterrupt the version you posted is buggy regardless of how g.close()
works).

Guido didn't like that idea [1], so the correct way to write your exception
handler is to either explicitly catch & reraise GeneratorExit before doing a
general "except Exception:" catch-all clause, or to do the yield outside the
try-except clause (but inside the try-finally) (obviously, this makes more
sense when what you're yielding is an expression rather than a single variable).

I think Guido is correct that GeneratorExit should be a normal exception from
a technical point of view (as GE should never reach the outer layers of a
program), but I also believe it is going to be a PITA from a practical point
of view because I think the natural way of writing a generator with exception
handling is to include the yield inside the try/except block instead of
storing the result in a variable and yielding afterwards.

However, since the status quo is the result of BDFL preference, it's going to
be up to Guido as to whether or not he considers this enough of a potential
pain to change his mind. As there isn't actually any new evidence here (it's
all still hypothetical), I don't really expect that to happen :)

That said, with the PEP 342 implementation now being a bit more mature, I am
curious as to exactly *how* GeneratorExit is supposed to leak out of code that
was supposed to catch it (which was Guido's main concern at the time he
decided that GE should remain a normal Exception). GeneratorExit really
shouldn't be getting raised by anything other than g.close(), which takes
great pains to ensure that the exception gets handled correctly (and if it
*doesn't* get handled correctly, the resulting exception that reports that
fact is a RuntimeError, not GeneratorExit).

> This not only fixes the above discrepancy between normal flow control
> and generators, removes GeneratorExit and simplifies the generator
> protocol, but also bring a new feature allowing to have easy to grasp
> feature table of the iterator methods:
> 
> next: continue after yield

This isn't right, though. next()/send() just resume execution after a yield. 
They are only equivalent to continue if the yield is at the end of a loop body.

> throw: raise after yield
> close: return after yield

In the current implementation, close is just a special case of throw
(g.throw(GeneratorExit)), the same as next() is just a special case of send
(g.send(None)).

2 concepts to understand (send(), throw()) is simpler to my mind than 3
(send(), throw(), close()).

Cheers,
Nick.

[1] http://mail.python.org/pipermail/python-dev/2006-March/062825.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Wed Aug 23 13:44:44 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 23 Aug 2006 21:44:44 +1000
Subject: [Python-Dev] String formatting / unicode 2.5 bug?
In-Reply-To: <Pine.WNT.4.64.0608211442280.2404@shaolin>
References: <Pine.LNX.4.64.0608192354450.8398@localhost>	<44E7ED19.6090907@gmail.com>	<Pine.LNX.4.64.0608201151320.8426@localhost>	<44E97EC9.4040502@gmail.com>
	<Pine.WNT.4.64.0608211442280.2404@shaolin>
Message-ID: <44EC3FAC.3030202@gmail.com>

John J Lee wrote:
> On Mon, 21 Aug 2006, Nick Coghlan wrote:
> 
>> John J Lee wrote:
>>>> And once the result has been promoted to unicode, __unicode__ is used 
>>>> directly:
>>>>
>>>>>>>  print repr("%s%s" % (a(), a()))
>>>> __str__
>>>> accessing <__main__.a object at 0x00AF66F0>.__unicode__
>>>> __str__
>>>> accessing <__main__.a object at 0x00AF6390>.__unicode__
>>>> __str__
>>>> u'hihi'
>>> I don't understand this part.  Why is __unicode__ called?  Your example 
>>> doesn't appear to show this happening "once [i.e., because?] the result has 
>>> been promoted to unicode" -- if that were true, it would "stand to reason" 
>>> <wink> that the interpreter would then conclude it should call
>>> __unicode__ for all remaining %s, and not bother with __str__.
>> It does try to call unicode directly, but because the example object doesn't 
>> supply __unicode__ it ends up falling back to __str__ instead. The behaviour 
>> is clearer when the example object provides both methods:
> [...]
> 
> If the interpreter is falling back from __unicode__ to __str__ (rather 
> than the other way around, kind-of), that makes much more sense.  I 
> understood that __unicode__ was not provided, of course -- what wasn't 
> clear to me was why the interpreter was calling/accessing those 
> methods/attributes in the sequence it does.  Still not sure I understand 
> what the third __str__ above comes from, but until I've thought it through 
> again, that's my problem.

The sequence is effectively:
x, y = a(), a()
str(x)     # calls x.__str__
unicode(x) # tries x.__unicode__, fails, falls back to x.__str__
unicode(y) # tries y.__unicode__, fails, falls back to y.__str__

The trick in 2.5 is that the '%s' format code, instead of actually calling
str(x), calls x.__str__() directly, and promotes the result to Unicode if
x.__str__() returns a Unicode result.

I'll try to do something to clear up that section of the documentation before
2.5 final.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Wed Aug 23 16:28:24 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 23 Aug 2006 07:28:24 -0700
Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary
In-Reply-To: <44EC3CF5.8030502@gmail.com>
References: <df0b33100608220617y5760c9f8q8fe713b2e1fc6c7b@mail.gmail.com>
	<44EC3CF5.8030502@gmail.com>
Message-ID: <ca471dc20608230728o2da97395o5abe2a20b3b865de@mail.gmail.com>

This discussion probably happens as a result of attempts to copy
Python's design in JavaScript. I propose that JavaScript do whatever
it wants and its developers leave Python alone -- maybe some time in
the future we'll be able to compare the two approaches. I think it's
impossible to do so at the moment.

--Guido

On 8/23/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Igor Bukanov wrote:
> > This example also suggests how to fix generators. One just need to
> > change the close method so it would cause return executed right after
> > the yield instead of throw.
> >
> > I.e. replace the current text from
> > http://www.python.org/dev/peps/pep-0342/
> <snip close() = throw(GeneratorExit)>
>
> > by simpler one:
> <snip close() = externally forced return>
>
> Simpler is in the eye of the beholder - the current close() merely uses
> throw() to raise a particular kind of exception 'asynchronously' (which is
> already a familiar concept due to KeyboardInterrupt). What you're suggesting
> is a completely new flow control concept that doesn't exist anywhere else in
> the language.
>
> I suggested during the PEP 352 discussions that GeneratorExit should inherit
> directly from BaseException, so that the fix to your example would be to
> replace the bare "except:" with "except Exception:" (because it can swallow
> KeyboardInterrupt the version you posted is buggy regardless of how g.close()
> works).
>
> Guido didn't like that idea [1], so the correct way to write your exception
> handler is to either explicitly catch & reraise GeneratorExit before doing a
> general "except Exception:" catch-all clause, or to do the yield outside the
> try-except clause (but inside the try-finally) (obviously, this makes more
> sense when what you're yielding is an expression rather than a single variable).
>
> I think Guido is correct that GeneratorExit should be a normal exception from
> a technical point of view (as GE should never reach the outer layers of a
> program), but I also believe it is going to be a PITA from a practical point
> of view because I think the natural way of writing a generator with exception
> handling is to include the yield inside the try/except block instead of
> storing the result in a variable and yielding afterwards.
>
> However, since the status quo is the result of BDFL preference, it's going to
> be up to Guido as to whether or not he considers this enough of a potential
> pain to change his mind. As there isn't actually any new evidence here (it's
> all still hypothetical), I don't really expect that to happen :)
>
> That said, with the PEP 342 implementation now being a bit more mature, I am
> curious as to exactly *how* GeneratorExit is supposed to leak out of code that
> was supposed to catch it (which was Guido's main concern at the time he
> decided that GE should remain a normal Exception). GeneratorExit really
> shouldn't be getting raised by anything other than g.close(), which takes
> great pains to ensure that the exception gets handled correctly (and if it
> *doesn't* get handled correctly, the resulting exception that reports that
> fact is a RuntimeError, not GeneratorExit).
>
> > This not only fixes the above discrepancy between normal flow control
> > and generators, removes GeneratorExit and simplifies the generator
> > protocol, but also bring a new feature allowing to have easy to grasp
> > feature table of the iterator methods:
> >
> > next: continue after yield
>
> This isn't right, though. next()/send() just resume execution after a yield.
> They are only equivalent to continue if the yield is at the end of a loop body.
>
> > throw: raise after yield
> > close: return after yield
>
> In the current implementation, close is just a special case of throw
> (g.throw(GeneratorExit)), the same as next() is just a special case of send
> (g.send(None)).
>
> 2 concepts to understand (send(), throw()) is simpler to my mind than 3
> (send(), throw(), close()).
>
> Cheers,
> Nick.
>
> [1] http://mail.python.org/pipermail/python-dev/2006-March/062825.html
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>              http://www.boredomandlaziness.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Aug 23 16:35:42 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 23 Aug 2006 10:35:42 -0400
Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary
In-Reply-To: <df0b33100608220617y5760c9f8q8fe713b2e1fc6c7b@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com>

I rather like it, actually.  I don't recall there being any real reasons to 
catch a GeneratorExit exception, but even if there were, you could do the 
equivalent like this:

     try:
         closed = True
         yield 1
         closed = False
     finally:
         if closed:
             # special closing code

Unfortunately, this is a pretty major change to be making to the 
specification (PEP 342) when we've just released 2.5c1; it seems like 
something that requires at least a beta release.

But aside from that, it seems like a fairly neat solution to the 
problem.  I'd like to hear from Jython and IronPython representatives 
regarding implementability for those platforms, though.  Our original 
assumption was that if they could implement throw() then they could clearly 
implement close(), since close() was defined in terms of throw().  An 
asynchronous return might be another matter.  But if that hurdle is 
crossable, then I agree that modelling close() as an early return is more 
intuitive, eliminates a built-in exception, etc.

At 03:17 PM 8/22/2006 +0200, Igor Bukanov wrote:
>Consider the following example:
>
>for i in range(3):
>   try:
>     print i
>     break
>   except:
>     print "Unexpected exception!"
>   finally:
>     print "Finally"
>
>When executed, it naturally prints
>   0
>   Finally
>since break does not use exceptions to transfer the control and as
>such can not be stopped using catch-all exception handler.
>
>Now consider a similar example using generators:
>
>def gen():
>   for i in range(3):
>     try:
>       yield i
>     except:
>       print "Unexpected exception!"
>     finally:
>       print "Finally"
>
>for i in gen():
>   print i
>   break
>
>This example prints:
>   0
>   Unexpected exception!
>   Finally
>   Exception exceptions.RuntimeError: 'generator ignored GeneratorExit'
>in <generator object at 0xb7daaa8c> ignored
>
>Suddenly with generators a program can mess with control transfer
>since it uses explicit GeneratorExit which can be caught and ignored.
>This is unintuitive IMO.
>
>This example also suggests how to fix generators. One just need to
>change the close method so it would cause return executed right after
>the yield instead of throw.
>
>I.e. replace the current text from
>http://www.python.org/dev/peps/pep-0342/
>
>4. Add a close() method for generator-iterators, which raises
>GeneratorExit at the point where the generator was paused.  If the
>generator then raises StopIteration (by exiting normally, or due to
>already being closed) or GeneratorExit (by not catching the
>exception), close() returns to its caller.  If the generator yields a
>value, a RuntimeError is raised.  If the generator raises any other
>exception, it is propagated to the caller. close() does nothing if the
>generator has already exited due to an exception or normal exit.
>
>by simpler one:
>
>4. Add a close() method for generator-iterators, which executes normal
>return at the point where the generator was paused.  If the generator
>then raises StopIteration (by exiting normally, or due to already
>being closed), close() returns to its caller.  If the generator yields
>a value, a RuntimeError is raised.  If the generator raises any other
>exception, it is propagated to the caller. close() does nothing if the
>generator has already exited due to an exception or normal exit.
>
>This not only fixes the above discrepancy between normal flow control
>and generators, removes GeneratorExit and simplifies the generator
>protocol, but also bring a new feature allowing to have easy to grasp
>feature table of the iterator methods:
>
>next: continue after yield
>throw: raise after yield
>close: return after yield
>
>Regards, Igor
>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: 
>http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com


From cemerick at snowtide.com  Wed Aug 23 17:11:32 2006
From: cemerick at snowtide.com (Chas Emerick)
Date: Wed, 23 Aug 2006 11:11:32 -0400
Subject: [Python-Dev] Python + Java Integration
Message-ID: <B53BCFBF-4FF1-42A8-B668-CB3E5513486E@snowtide.com>

This may seem like it's coming out of left field for a minute, but  
bear with me.

There is no doubt that Ruby's success is a concern for anyone who  
sees it as diminishing Python's status.  One of the reasons for  
Ruby's success is certainly the notion (originally advocated by Bruce  
Tate, if I'm not mistaken) that it is the "next Java" -- the language  
and environment that mainstream Java developers are, or will, look to  
as a natural next step.

One thing that would help Python in this "debate" (or, perhaps simply  
put it in the running, at least as a "next Java" candidate) would be  
if Python had an easier migration path for Java developers that  
currently rely upon various third-party libraries.  The wealth of  
third-party libraries available for Java has always been one of its  
great strengths.  Ergo, if Python had an easy-to-use, recommended way  
to use those libraries within the Python environment, that would be a  
significant advantage to present to Java developers and those who  
would choose Ruby over Java.  Platform compatibility is always a huge  
motivator for those looking to migrate or upgrade.

In that vein, I would point to JPype (http://jpype.sourceforge.net).   
JPype is a module that gives "python programs full access to java  
class libraries".  My suggestion would be to either:

(a) include JPype in the standard library, or barring that,
(b) make a very strong push to support JPype

(a) might be difficult or cumbersome technically, as JPype does need  
to build against Java headers, which may or may not be possible given  
the way that Python is distributed, etc.

However, (b) is very feasible.  I can't really say what "supporting  
JPype" means exactly -- maybe GvR and/or other heavyweights in the  
Python community make public statements regarding its existence and  
functionality, maybe JPype gets a strong mention or placement on  
python.org....all those details are obviously not up to me, and I  
don't know the workings of the "official" Python organizations enough  
to make serious suggestions.

Regardless of the form of support, I think raising people's awareness  
of JPype and what it adds to the Python environment would be a Good  
Thing (tm).

For our part, we've used JPype to make PDFTextStream (our previously  
Java-only PDF text extraction library) available and supported for  
Python.  You can read some about it here:

http://snowtide.com/PDFTextStream.Python

And I've blogged about how PDFTextStream.Python came about, and how  
we worked with Steve M?nard, the maintainer of JPype, to make it all  
happen (watch out for this URL wrapping):

http://blog.snowtide.com/2006/08/21/working-together-pythonjava-open- 
sourcecommercial

Cheers,

Chas Emerick
Founder, Snowtide Informatics Systems
Enterprise-class PDF content extraction

cemerick at snowtide.com
http://snowtide.com | +1 413.519.6365



From jeremy.kloth at 4suite.org  Wed Aug 23 17:21:27 2006
From: jeremy.kloth at 4suite.org (Jeremy Kloth)
Date: Wed, 23 Aug 2006 09:21:27 -0600
Subject: [Python-Dev] [4suite] cDomlette deallocation bug?
In-Reply-To: <44EB71F2.2080509@cs.washington.edu>
References: <44EB71F2.2080509@cs.washington.edu>
Message-ID: <200608230921.28027.jeremy.kloth@4suite.org>

I'm CC python-dev as this may be a bug in the gc module. Although this may be 
more of a c.l.p. post, I think that this usage shouldn't cause segfaults 
regardless.

On Tuesday 22 August 2006 3:06 pm, Robert Fleming wrote:
> [step 1 unnecessary]
> 2. test program (test.c):
> /******** snip ********/
> #include <Python.h>
>
> #include <stdio.h>
>
> int main(int argc, char* argv[])
> {
>   int i;
>   for (i = 0; i<10; i++)
>     {
>       printf("%d\n", i);
>
>       Py_Initialize();
>       (void)PyRun_SimpleString("import sys\n"
>                    "sys.path.append('.')\n"
>                    "import cDomlettec");

Change the string to just "import gc" and the segfault still happens.

>       Py_Finalize();
>     }
>
>   return EXIT_SUCCESS;
> }
> /****** snip ********/
>
> 3. compile with:
> gcc -I/usr/include/python2.4 test.c -lpython2.4
>
> 4. run program; output is:
> 0
> 1
> Segmentation fault
>
> Am I doing something wrong?  I'm pretty new to embedding Python, but I'm
> hoping to be able to cleanly load and unload 4Suite this way

It just so happens that cDomlettec imports gc internally which, by the change 
above shows, causes the segfault.  Hopefully someone with more knowledge of 
GC internals can comment on this.

-- 
Jeremy Kloth
http://4suite.org/

From theller at python.net  Wed Aug 23 22:03:39 2006
From: theller at python.net (Thomas Heller)
Date: Wed, 23 Aug 2006 22:03:39 +0200
Subject: [Python-Dev] ctypes and win64
In-Reply-To: <44E60B2A.6020101@v.loewis.de>
References: <ec50v9$oln$1@sea.gmane.org> <44E60B2A.6020101@v.loewis.de>
Message-ID: <ecicar$apg$1@sea.gmane.org>

Martin v. L?wis schrieb:
> Thomas Heller schrieb:
>> I suggest that it [the ctypes package] should be removed from the 2.5 win64 msi installers, so that
>> at least, when it is ready, can be installed as separate package.
> 
> Unfortunately, I won't be able to work on this until the release is
> made. Feel free to work on it, but I suggest to just not worry about
> it.

Patch for Tools\msi\msi.py uploaded, and assigned to you.

http://python.org/sf/1545507

Thomas


From sreeram at tachyontech.net  Wed Aug 23 23:22:55 2006
From: sreeram at tachyontech.net (K.S.Sreeram)
Date: Thu, 24 Aug 2006 02:52:55 +0530
Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array lookup?
Message-ID: <44ECC72F.10303@tachyontech.net>

Hi all,

I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup,
and was wondering if that can be optimized to a simple array lookup.

If i'm right there are 3 kinds of name lookups: locals, outer
scopes(closures), and globals. (not counting attribute lookup). Locals
are identified by, either the presence of assignments, or their presence
in the arg list. So all name lookups can be classified into the 3 types
at compile/load time.

Since we know, at load time, which names are global.. Can't we simply
build a global name table and replace LOAD_GLOBALs with a lookup at the
corresponding index into the global name table?

The module's dict object will need to be special so that whenever a name
gets 'set', the global name table should get updated.

Is this optimization possible or have i missed something?

cheers
[sreeram;]

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20060824/d936dba5/attachment.pgp 

From tjreedy at udel.edu  Wed Aug 23 23:58:39 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 23 Aug 2006 17:58:39 -0400
Subject: [Python-Dev] Python + Java Integration
References: <B53BCFBF-4FF1-42A8-B668-CB3E5513486E@snowtide.com>
Message-ID: <ecij2f$22t$1@sea.gmane.org>


"Chas Emerick" <cemerick at snowtide.com> wrote in message 
news:B53BCFBF-4FF1-42A8-B668-CB3E5513486E at snowtide.com...
>One thing that would help Python in this "debate"

In general, discussion of promoting/marketing python is more a topic for 
comp.lang.python than for this list.  Also see below.

>My suggestion would be to either:
>(a) include JPype in the standard library, or barring that,

Externally developed modules become part of the standard library by 
becoming sufficiently 'widely' tested and used in the Python community. 
One means for this is announcement and discussion in c.l.p.

That and technical considerations aside, this strikes me as one of several 
specialized 'bridge' tools, none of which should be in that standard lib 
but all of which should be listed and accessible in the Python Package 
Index.

>(b) make a very strong push to support JPype

We are mostly all volunteers here.  Go ahead and scratch this itch if you 
want.  If it does not yet already, I would agree that python.org should 
have a page on bridging between Python and other languages (not just Java, 
though).  C.l.py would be a place to discuss the idea and collect 
suggestions for inclusion on such a page

>For our part, we've used JPype to make PDFTextStream (our previously
>Java-only PDF text extraction library) available and supported for Python.

Success strories that provide new information are welcome on c.l.py. 
Perhaps a couple of paragraphs summarizing the content that you link to.

Terry Jan Reedy




From brett at python.org  Thu Aug 24 00:28:55 2006
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Aug 2006 15:28:55 -0700
Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array
	lookup?
In-Reply-To: <44ECC72F.10303@tachyontech.net>
References: <44ECC72F.10303@tachyontech.net>
Message-ID: <bbaeab100608231528h21ce82f9ifeb8af0122b307ad@mail.gmail.com>

On 8/23/06, K.S.Sreeram <sreeram at tachyontech.net> wrote:
>
> Hi all,
>
> I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup,
> and was wondering if that can be optimized to a simple array lookup.


No, not as the language  stands now.

If i'm right there are 3 kinds of name lookups: locals, outer
> scopes(closures), and globals. (not counting attribute lookup). Locals
> are identified by, either the presence of assignments, or their presence
> in the arg list. So all name lookups can be classified into the 3 types
> at compile/load time.
>
> Since we know, at load time, which names are global.. Can't we simply
> build a global name table and replace LOAD_GLOBALs with a lookup at the
> corresponding index into the global name table?


But we don't know statically what the globals will be.  You can import a
module and put something in its global namespace externally.  That is done
after load time or compile time.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060823/0533d23a/attachment.htm 

From thomas at python.org  Thu Aug 24 00:29:21 2006
From: thomas at python.org (Thomas Wouters)
Date: Thu, 24 Aug 2006 00:29:21 +0200
Subject: [Python-Dev] [Python-checkins] r51525 - in python/trunk:
	Lib/test/test_float.py Objects/floatobject.c
In-Reply-To: <9e804ac0608231516k3a08f0c5m3c4375fe0a09a24a@mail.gmail.com>
References: <20060823204203.3C4911E4009@bag.python.org>
	<9e804ac0608231516k3a08f0c5m3c4375fe0a09a24a@mail.gmail.com>
Message-ID: <9e804ac0608231529y7c8b941fyf1cb21a7a4ef5a6b@mail.gmail.com>

Since Alex isn't on python-dev, forwarding this for his convenience (he said
he wanted to reply.) Tim also replied to his checkin, but I didn't forward
that message.

---------- Forwarded message ----------
From: Thomas Wouters <thomas at python.org>
Date: Aug 24, 2006 12:16 AM
Subject: Re: [Python-checkins] r51525 - in python/trunk:
Lib/test/test_float.py Objects/floatobject.c
To: "alex. martelli" <python-checkins at python.org>



On 8/23/06, alex.martelli <python-checkins at python.org> wrote:
>
>
> --- python/trunk/Objects/floatobject.c  (original)
> +++ python/trunk/Objects/floatobject.c  Wed Aug 23 22:42:02 2006
> @@ -821,12 +821,12 @@
>         ix = pow(iv, iw);
>         PyFPE_END_PROTECT(ix)
>         Py_ADJUST_ERANGE1(ix);
> -       if (errno != 0) {
> +        /* we need to ignore ERANGE here and just return inf */
> +       if (errno != 0 && errno != ERANGE) {
>                 /* We don't expect any errno value other than ERANGE, but
>                  * the range of libm bugs appears unbounded.
>                  */
> -               PyErr_SetFromErrno(errno == ERANGE ? PyExc_OverflowError :
> -                                                    PyExc_ValueError);
> +               PyErr_SetFromErrno(PyExc_ValueError);
>                 return NULL;


I don't think this can be right. The returnvalue of pow() in the case of
ERANGE isn't defined anywhere that I can find, at least. How can we assume
it is +Infinity? As I tried to say over the visiphone, you can't even use
compile-time or startup-time checks for the behaviour because 'undefined'
also means it need not be consistent, either. The best we could do, if we
really wanted to return +inf instead of raising OverflowError (which I
don't, but don't really care about either), is generate +Infinity in some
guaranteed way. I'm sure Tim can come up with a convenient, portable way <
0.9 wink>.

But I'd prefer to keep it the way it was: x*x and x**2 don't always do the
same thing. Floats have a lot more confusing features like that, such as
10*x - 9*x need not be x. I don't see the added value of trying to make it
consistent in just this case, even if it's portable.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!


-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/5949f7c3/attachment.html 

From aleaxit at gmail.com  Thu Aug 24 00:39:43 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 23 Aug 2006 15:39:43 -0700
Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array
	lookup?
In-Reply-To: <44ECC72F.10303@tachyontech.net>
References: <44ECC72F.10303@tachyontech.net>
Message-ID: <A1482118-BBCB-429A-AB3C-81B9E206082C@gmail.com>


On Aug 23, 2006, at 2:22 PM, K.S.Sreeram wrote:

> Hi all,
>
> I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup,
> and was wondering if that can be optimized to a simple array lookup.
>
> If i'm right there are 3 kinds of name lookups: locals, outer
> scopes(closures), and globals. (not counting attribute lookup). Locals
> are identified by, either the presence of assignments, or their  
> presence
> in the arg list. So all name lookups can be classified into the 3  
> types
> at compile/load time.
>
> Since we know, at load time, which names are global.. Can't we simply
> build a global name table and replace LOAD_GLOBALs with a lookup at  
> the
> corresponding index into the global name table?

At the time the function's body gets compiled, the global (or  
builtin) it's trying to access might or might not be there -- as long  
as it gets added afterwards, before the function's body gets  
_executed_, no problem (in today's scheme).  It's not obvious to me  
how you could compile a ``corresponding index'' into the LOAD_GLOBAL  
opcode, since that index is in general unknown at compile time.

> The module's dict object will need to be special so that whenever a  
> name
> gets 'set', the global name table should get updated.

It seems that you'd need to chase down and modify all of the  
LOAD_GLOBAL opcodes too, at every such modification.   (the concept  
of modifying builtins becomes extremely scary...).  Considering the  
amortized speed of a dict lookup for an interned string (hash value  
cached and thus immediately available, equality comparison with other  
interned string a single machine-level operation), it's not clear to  
me that the huge complexity (and potential performance impact) of all  
this could ever possibly be justified.

A change in Python semantics allowing some level of "nailing down" of  
builtins (and perhaps globals too) *COULD* easily yield large  
performance benefits, but that's a subject for the Python-3000  
mailing list; as long as builtins and globals stay as fluid as today,  
I'm skeptical on the optimization opportunities here.


Alex



From noamraph at gmail.com  Thu Aug 24 00:50:34 2006
From: noamraph at gmail.com (Noam Raphael)
Date: Thu, 24 Aug 2006 01:50:34 +0300
Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array
	lookup?
In-Reply-To: <bbaeab100608231528h21ce82f9ifeb8af0122b307ad@mail.gmail.com>
References: <44ECC72F.10303@tachyontech.net>
	<bbaeab100608231528h21ce82f9ifeb8af0122b307ad@mail.gmail.com>
Message-ID: <b348a0850608231550y6b20aa54g9330fbbaaa054b0f@mail.gmail.com>

2006/8/24, Brett Cannon <brett at python.org>:
>
>
>
> On 8/23/06, K.S.Sreeram <sreeram at tachyontech.net> wrote:
> > Hi all,
> >
> > I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup,
> > and was wondering if that can be optimized to a simple array lookup.
>
>
> No, not as the language  stands now.
>
> > If i'm right there are 3 kinds of name lookups: locals, outer
> > scopes(closures), and globals. (not counting attribute lookup). Locals
> > are identified by, either the presence of assignments, or their presence
> > in the arg list. So all name lookups can be classified into the 3 types
> > at compile/load time.
> >
> > Since we know, at load time, which names are global.. Can't we simply
> > build a global name table and replace LOAD_GLOBALs with a lookup at the
> > corresponding index into the global name table?
>
>
> But we don't know statically what the globals will be.  You can import a
> module and put something in its global namespace externally.  That is done
> after load time or compile time.
>
I think that it can be implemented for the language as it stands now.
I don't know whether it will be a good thing or not.

In principle, you can add a feature to dict implementation that will
allow it to notify when the value of a specific key changes. If you
have that, you can change LOAD_GLOBAL implementation to:
1. look for the global.
2. ask for notification when the global dict changes in a way that
will change the meaning of the global.
3. change the LOAD_GLOBAL opcode to something like LOAD_CONST, and set
the notification from the dict to update the LOAD_CONST opcode to the
new object.

In that way, LOAD_GLOBAL will cause a dict lookup only once. Changing
the value of globals will require more work, though.

Again, I'm not saying that it's desired, just that it's possible.

Have a good day,
Noam

From brett at python.org  Thu Aug 24 00:59:39 2006
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Aug 2006 15:59:39 -0700
Subject: [Python-Dev] PEP 362 open issues
Message-ID: <bbaeab100608231559j14d2f7c3s37eef593a9f1809e@mail.gmail.com>

I have been spending my Google sprint time on writing and implementing PEP
362 (http://www.python.org/dev/peps/pep-0362/), which finally defines
Signature objects for functions.  With the implementation at a place I am
happy with, I wanted to ask about the open issues with the PEP.

The first question is whether  to have Signature objects on all functions by
default or to leave it in the 'inspect' module and have it be something
totally optional.  My vote is the latter for now and we can add them to all
functions if they turn out to be useful and popular.

The second open issue is how to handle binding arguments to tuple parameters
for the Signature.bind() method.  If you have the function ``def foo((a, b):
pass``, how do you have the Signature.bind() method handle 'a', and 'b'?
You can either take the proposed arguments and in the dict mapping arguments
to parameters have an entry for 'a' and 'b' individually, or have an entry
for '(a, (b,))'.  My vote is the latter since you cannot address 'a' or 'b'
individually as keyword arguments and this makes more sense with function
annotations since they won't apply to 'a' and 'b' individually but instead
to the entire tuple.  Either way there is the issue of exhausting an
iterator (like a generator) trying to find out if it is the proper length to
unpack, for which I raise a TypeError instead of exhaust a possibly mutable
iterator currently.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060823/926f341c/attachment.htm 

From guido at python.org  Thu Aug 24 01:10:03 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 23 Aug 2006 16:10:03 -0700
Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary
In-Reply-To: <df0b33100608231552ka6e6e99na18001b0eeb38c83@mail.gmail.com>
References: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com>
	<df0b33100608231552ka6e6e99na18001b0eeb38c83@mail.gmail.com>
Message-ID: <ca471dc20608231610i276f2cf3mfb032b561fe94b3@mail.gmail.com>

IIUC this is how return currently works -- in some sense it's an
exception, but it can't be caught, and it won't escape from the
current frame. Ditto for break/continue.

The generator extensions are still very young, and I'm not against
changing them somewhat in 2.6 or py3k. Perhaps GeneratorExit can be
replaced by a treatment similar to the treatment of return. But I'm
not sure it helps with the original problem; you could still put a
yield in a finally clause that intercepts a return, just as today you
can write

def f():
  for i in range(10):
    print i
    try:
      return
    finally:
      break
  print 999

where the finally clause nullifies the return. In a generator, a yield
in the finally clause still needs to be dealt with somehow.


On 8/23/06, Igor Bukanov <igor.bukanov at gmail.com> wrote:
> On 8/23/06, Phillip J. Eby wrote:
> > Our original
> > assumption was that if they could implement throw() then they could clearly
> > implement close(), since close() was defined in terms of throw().  An
> > asynchronous return might be another matter.
>
> Such asynchronous return can always be implemented via a special
> hidden exception that is invisible to catch statements. I.e an
> implementation of generators  can still use ___GeneratorExit
> internally as long as it is not exposed to scripts.
>
> Regards, Igor
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Thu Aug 24 00:13:55 2006
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 24 Aug 2006 00:13:55 +0200
Subject: [Python-Dev] [Python-checkins] r51525 - in
	python/trunk:Lib/test/test_float.py Objects/floatobject.c
In-Reply-To: <1f7befae0608231500n475cfed2x6cbfba20e1a2b480@mail.gmail.com>
References: <20060823204203.3C4911E4009@bag.python.org>
	<1f7befae0608231500n475cfed2x6cbfba20e1a2b480@mail.gmail.com>
Message-ID: <1156371234.44ecd32300e03@www.domainfactory-webmail.de>

Zitat von Tim Peters <tim.peters at gmail.com>:

> Huh.  It's been a (mildly controversial, but intentional all the same)
> feature that Python tries to raise raise OverflowError on overflowing
> libm operations.  Doesn't work all that well, since   there's no
> consistency across platforms about when libm sets errno, or to what
> (when it does).

The other question is whether pow() is guaranteed to return inf
when it sets errno to ERANGE; I believe there is no such guarantee.

So I think the change should be reverted.

Regards,
Martin




From g.brandl at gmx.net  Thu Aug 24 01:13:54 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 24 Aug 2006 01:13:54 +0200
Subject: [Python-Dev] PEP 362 open issues
In-Reply-To: <bbaeab100608231559j14d2f7c3s37eef593a9f1809e@mail.gmail.com>
References: <bbaeab100608231559j14d2f7c3s37eef593a9f1809e@mail.gmail.com>
Message-ID: <ecinfj$epu$1@sea.gmane.org>

Brett Cannon wrote:
> I have been spending my Google sprint time on writing and implementing 
> PEP 362 (http://www.python.org/dev/peps/pep-0362/), which finally 
> defines Signature objects for functions.  With the implementation at a 
> place I am happy with, I wanted to ask about the open issues with the PEP.

While reading the PEP, I stumbled over the description of the bind() method.

As I understand the implementation patch, the parameter names will be keys and
the arguments values in the resulting dictionary, but this sentence looks to
me as if it describes it the opposite way (it also doesn't clarify that
"parameter" means "parameter name"):

* bind(*args, **kwargs) -> dict
     Create a mapping from argument to parameter for the signature (see
     `Open Issues`_ for question of how to handle tuples).

Another question: could it be helpful to make Parameter.default_value a
weak reference?

Georg


From aleaxit at gmail.com  Thu Aug 24 01:17:47 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 23 Aug 2006 16:17:47 -0700
Subject: [Python-Dev] [Python-checkins] r51525 - in python/trunk:
	Lib/test/test_float.py Objects/floatobject.c
In-Reply-To: <9e804ac0608231529y7c8b941fyf1cb21a7a4ef5a6b@mail.gmail.com>
References: <20060823204203.3C4911E4009@bag.python.org>
	<9e804ac0608231516k3a08f0c5m3c4375fe0a09a24a@mail.gmail.com>
	<9e804ac0608231529y7c8b941fyf1cb21a7a4ef5a6b@mail.gmail.com>
Message-ID: <09CE8417-9F1F-4DEC-82DE-43AA0DAFD53E@gmail.com>


On Aug 23, 2006, at 3:29 PM, Thomas Wouters wrote:

>
> Since Alex isn't on python-dev, forwarding this for his convenience  
> (he said he wanted to reply.)

Thanks!  I _am_ on Python-Dev (otherwise couldn't read what you're  
forwarding here), but not on Python-Checkins (where the discussion  
was started).

> Tim also replied to his checkin, but I didn't forward that message.

Yep, saw that (thanks Brett) -- if I read him right, Tim was  
basically suggesting better ways to check for infinity rather than  
saying it's better to keep today's inconsistency (he can surely  
correct me if I misread).  Tim is of course right, but PEP 754 (the  
fpconst module) would appear to be the right way to perform the  
checking, anyway.

> I don't think this can be right. The returnvalue of pow() in the  
> case of ERANGE isn't defined anywhere that I can find, at least.  
> How can we assume it is +Infinity? As I tried to say over the  
> visiphone, you can't even use compile-

That's how I read ISO/IEC 9899:1999 7.12.1.4 , which is describing  
C's <math> functions in general:

"""
If a floating result overflows and default rounding is in effect, or  
if the mathematical result is an exact infinity (for example log 
(0.0)), then the function returns the value of the macro HUGE_VAL,  
HUGE_VALF, or HUGE_VALL according to the return type, with the same  
sign as the correct value of the function;
"""
etc.

Yes, that's a C language standard that's just 7 years old, rather  
than the 16-years-old standard Python can "definitely" count on; but  
we could presumably test (and rely on) __STDC_IEC_559__ if we need  
features in Appendix F, or just on __STDC_VERSION__ being defined and  
 >= 199901L for more generic parts of the standard (such as the part  
I was quoting above from chapter 7).  People who build Python with C  
compilers not meeting these macro criteria but still "good enough" to  
afford a minimally functional implementation of IEEE 754 floats might  
run configure with some appropriate -with-whatever options to assert  
the fact, if needed (i.e., if either because of C 1990 Standard  
language, or realworld brokenness of important compilers, automatic  
tests are in fact unreliable).

And PEP 754 (and thus fpconst, or some other way for Python code to  
tell if it's running on a minimally functional implementation of IEEE  
754 floats, and if so test for and generate NaNs and infinities)  
should also be there so that the Python-coded unittests can properly  
decide whether to test for compliance.

> time or startup-time checks for the behaviour because 'undefined'  
> also means it need not be consistent, either. The

Yes, but the behavior is not undefined in standard C (1999 standard;  
I don't have access to the 1990 standard)

> best we could do, if we really wanted to return +inf instead of  
> raising OverflowError (which I don't, but don't really care about  
> either), is generate +Infinity in some guaranteed way. I'm sure Tim  
> can come up with a convenient, portable way < 0.9 wink>.

The macro Py_HUGE_VAL seems to be the documented way to do it.  And  
while I'd rather uniformly get infinity from operations producing  
infinity, what I'm really keen to avoid is having one such operation  
raise an exception while another one succeeds.

> But I'd prefer to keep it the way it was: x*x and x**2 don't always  
> do the same thing. Floats have a lot more confusing features like  
> that, such as 10*x - 9*x need not be x. I don't see the added value  
> of trying to make it consistent in just this case, even if it's  
> portable.

There's a huge conceptual gap between, on one side, two values  
produced by "roughly equivalent" expressions being only *ALMOST*  
equal rather than strictly equal, and one of the expressions raising  
an exception while the other one happily proceeds.  The former case  
is reasonably easy to handle -- that's why we have an  
assertAlmostEqual method in unittest.TestCase; the latter can be a  
nightmare.

The same goes for the other bug I recently added to the tracker, the  
""" x != y and [x] == [y] """ one.  Of course, according to this  
"more confusing features" philosophy, I guess it can be considered  
quite OK to leave in Python absurdities like this one, as well.  I  
abhor that whle "slippery slope" philosophy ("there's a tiny  
unavoidable problem in this corner so it's OK to leave a huge gaping  
hole all over the place") but I see no way to resolve such  
philosophical conflict without BDFL pronouncement.  So, I have  
reverted r51525; I now believe it can't be done right (with proper  
unittests) without PEP 754 or something like that, anyway.


Alex


From aleaxit at gmail.com  Thu Aug 24 01:24:11 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 23 Aug 2006 16:24:11 -0700
Subject: [Python-Dev] [Python-checkins] r51525 - in
	python/trunk:Lib/test/test_float.py Objects/floatobject.c
In-Reply-To: <1156371234.44ecd32300e03@www.domainfactory-webmail.de>
References: <20060823204203.3C4911E4009@bag.python.org>
	<1f7befae0608231500n475cfed2x6cbfba20e1a2b480@mail.gmail.com>
	<1156371234.44ecd32300e03@www.domainfactory-webmail.de>
Message-ID: <38FC57F6-7F1A-4F4B-B6A6-9198F2BC0FC4@gmail.com>


On Aug 23, 2006, at 3:13 PM, martin at v.loewis.de wrote:

> Zitat von Tim Peters <tim.peters at gmail.com>:
>
>> Huh.  It's been a (mildly controversial, but intentional all the  
>> same)
>> feature that Python tries to raise raise OverflowError on overflowing
>> libm operations.  Doesn't work all that well, since   there's no
>> consistency across platforms about when libm sets errno, or to what
>> (when it does).
>
> The other question is whether pow() is guaranteed to return inf
> when it sets errno to ERANGE; I believe there is no such guarantee.

Summarizing my longer mail that crossed with yours: the guarantee is  
there in the 1999 C Standard; I don't have access any more to a copy  
of the 1990 C Standard to check how things were back then.

> So I think the change should be reverted.

I have reverted the change, but I think it (or rather "something  
like" it -- *SOME* fix for x**2 raising OverflowError while x*x  
proceeds happily for a certain range of floats) is important and  
should go back in again.  Using Py_HUGE_VAL instead of trusting pow's  
result might be OK, for example, if the 1990 C Standard is that much  
weaker on this than 1999's and we need to stick to it anyway.  But  
the problem is that testing this sensibly requires fpconst (from PEP  
754) or the equivalent, otherwise it may be impossible to write  
unittest to show the presence or absence of this bug.


Alex


From brett at python.org  Thu Aug 24 02:30:55 2006
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Aug 2006 17:30:55 -0700
Subject: [Python-Dev] PEP 362 open issues
In-Reply-To: <ecinfj$epu$1@sea.gmane.org>
References: <bbaeab100608231559j14d2f7c3s37eef593a9f1809e@mail.gmail.com>
	<ecinfj$epu$1@sea.gmane.org>
Message-ID: <bbaeab100608231730q360db337xa525c88078a55a19@mail.gmail.com>

On 8/23/06, Georg Brandl <g.brandl at gmx.net> wrote:
>
> Brett Cannon wrote:
> > I have been spending my Google sprint time on writing and implementing
> > PEP 362 (http://www.python.org/dev/peps/pep-0362/), which finally
> > defines Signature objects for functions.  With the implementation at a
> > place I am happy with, I wanted to ask about the open issues with the
> PEP.
>
> While reading the PEP, I stumbled over the description of the bind()
> method.
>
> As I understand the implementation patch, the parameter names will be keys
> and
> the arguments values in the resulting dictionary, but this sentence looks
> to
> me as if it describes it the opposite way (it also doesn't clarify that
> "parameter" means "parameter name"):
>
> * bind(*args, **kwargs) -> dict
>      Create a mapping from argument to parameter for the signature (see
>      `Open Issues`_ for question of how to handle tuples).


You're right, it's a typo.

Another question: could it be helpful to make Parameter.default_value a
> weak reference?


Perhaps, but I don't think it is necessarily required.  I can change it if
others want to go that way, but for now I am happy with the way it is.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060823/d557359c/attachment.html 

From pje at telecommunity.com  Thu Aug 24 02:31:56 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 23 Aug 2006 20:31:56 -0400
Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary
In-Reply-To: <ca471dc20608231610i276f2cf3mfb032b561fe94b3@mail.gmail.com
 >
References: <df0b33100608231552ka6e6e99na18001b0eeb38c83@mail.gmail.com>
	<5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com>
	<df0b33100608231552ka6e6e99na18001b0eeb38c83@mail.gmail.com>
Message-ID: <5.1.1.6.0.20060823195222.022a1f40@sparrow.telecommunity.com>

At 04:10 PM 8/23/2006 -0700, Guido van Rossum wrote:
>IIUC this is how return currently works -- in some sense it's an
>exception, but it can't be caught, and it won't escape from the
>current frame. Ditto for break/continue.
>
>The generator extensions are still very young, and I'm not against
>changing them somewhat in 2.6 or py3k. Perhaps GeneratorExit can be
>replaced by a treatment similar to the treatment of return. But I'm
>not sure it helps with the original problem;

It depends on what you're referring to as "the original problem".  If you 
mean the problem Bob Ippolito opened a bug for, then yes, it does not solve 
that problem.  But I think we agree that *that* problem doesn't actually 
need solving.

However, it *does* solve the problem(s) that Igor Bukanov brought up, 
namely that GeneratorExit is unintuitive and unnecessary.  :)  It also 
fixes the problem of calling close() or GC'ing a generator written for an 
earlier version of Python, that yields inside a try-except.


>In a generator, a yield
>in the finally clause still needs to be dealt with somehow.

And we should keep the existing PEP-defined behavior for dealing with 
it.  That is, my understanding is that the only part of close()'s 
definition that would change is the 'throw(GeneratorExit)' bit.

I just wish this had been proposed some time much sooner, like maybe not 
long after I did the original implementation of 
throw()/close()/etc.  During the original discussion, I'd have been more 
skeptical of a return-based approach.  But knowing what I know now about 
the implementation, making it based on return instead would be trivial.  A 
proof-of-concept patch I just whipped up adds 7 lines to the hook code in 
ceval.c (to determine whether WHY_RETURN or WHY_EXCEPTION should be used), 
and 5 lines to gen_close() in genobject.c.


From pje at telecommunity.com  Thu Aug 24 02:43:31 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 23 Aug 2006 20:43:31 -0400
Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary
In-Reply-To: <df0b33100608231644h3842abbkb98f49a2bf11363b@mail.gmail.com
 >
References: <ca471dc20608231610i276f2cf3mfb032b561fe94b3@mail.gmail.com>
	<5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com>
	<df0b33100608231552ka6e6e99na18001b0eeb38c83@mail.gmail.com>
	<ca471dc20608231610i276f2cf3mfb032b561fe94b3@mail.gmail.com>
Message-ID: <5.1.1.6.0.20060823203943.026ce300@sparrow.telecommunity.com>

At 01:44 AM 8/24/2006 +0200, Igor Bukanov wrote:
>Regarding yield in the finally problem:
>
>The current implementation throws RuntimeException after the yield
>transfer the control back to the close method. If that is changed to
>reporting a bad behavior and rethrowing GeneratorExit or its hidden
>analog as suggested at the point of yield inside the generator, that
>would nicely unwind possible outer finally blocks and close the
>generator in any case.

This approach has already been proposed and rejected multiple times.  A 
badly-behaving generator is broken and asking it to handle the error of its 
own brokenness will lead only to grief.  Once the generator yields 
inappropriately, it has proven it cannot be trusted.  It is only 
compounding the generator's insanity to ask it to handle this error!


From sreeram at tachyontech.net  Thu Aug 24 12:58:33 2006
From: sreeram at tachyontech.net (K.S.Sreeram)
Date: Thu, 24 Aug 2006 16:28:33 +0530
Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array
	lookup?
In-Reply-To: <A1482118-BBCB-429A-AB3C-81B9E206082C@gmail.com>
References: <44ECC72F.10303@tachyontech.net>
	<A1482118-BBCB-429A-AB3C-81B9E206082C@gmail.com>
Message-ID: <44ED8659.7090602@tachyontech.net>

Alex Martelli wrote:
> 
> On Aug 23, 2006, at 2:22 PM, K.S.Sreeram wrote:
> 
>> Hi all,
>>
>> I noticed in Python/ceval.c that LOAD_GLOBAL uses a dictionary lookup,
>> and was wondering if that can be optimized to a simple array lookup.
>>
>> If i'm right there are 3 kinds of name lookups: locals, outer
>> scopes(closures), and globals. (not counting attribute lookup). Locals
>> are identified by, either the presence of assignments, or their presence
>> in the arg list. So all name lookups can be classified into the 3 types
>> at compile/load time.
>>
>> Since we know, at load time, which names are global.. Can't we simply
>> build a global name table and replace LOAD_GLOBALs with a lookup at the
>> corresponding index into the global name table?
> 
> At the time the function's body gets compiled, the global (or builtin)
> it's trying to access might or might not be there -- as long as it gets
> added afterwards, before the function's body gets _executed_, no problem
> (in today's scheme).  It's not obvious to me how you could compile a
> ``corresponding index'' into the LOAD_GLOBAL opcode, since that index is
> in general unknown at compile time.

How about this approach:
(disclaimer: this is just a rough sketch!)

At compile time:
---------------
- When compiling the module, a list of global names is built. This list
is populated in the order in which the names appear in the module.
References to the global will be replaced by references to the index in
this list.

- Instead of generating LOAD_GLOBAL and STORE_GLOBAL opcodes, we
generate LOAD_GLOBAL_BY_INDEX and STORE_GLOBAL_BY_INDEX, which are new
opcodes. The indexes used with the opcode will be the index of the name
from the list.

- This list of names should be stored in the pyc file so that the
ordering is remembered.


At load time:
-------------
- At module load time, we build an array of PyObject*, which is used to
store the values for the globals. The values will be NULL, until the
actual binding is created.

- For each name in the global_name_list in the pyc file, we pass the
pointer to the corresponding entry in the global table, to the dict.
    PyDict_SetLocationForName( module_dict, name, PyObject
**ptr_to_entry_table )
  This is done so that 'get' and 'set' of these names in the dict will
update/fetch from the global table directly.


This approach should result in faster access to globals defined within
the module. It will still work if we try to shadow any builtins. As far
as i see, this doesn't seem to change any existing semantics either.

This does not result in faster access to existing builtins. But a
similar scheme can be worked out for that too.

> 
>> The module's dict object will need to be special so that whenever a name
>> gets 'set', the global name table should get updated.
> 
> It seems that you'd need to chase down and modify all of the LOAD_GLOBAL
> opcodes too, at every such modification.   (the concept of modifying
> builtins becomes extremely scary...).  Considering the amortized speed
> of a dict lookup for an interned string (hash value cached and thus
> immediately available, equality comparison with other interned string a
> single machine-level operation), it's not clear to me that the huge
> complexity (and potential performance impact) of all this could ever
> possibly be justified.
> 
> A change in Python semantics allowing some level of "nailing down" of
> builtins (and perhaps globals too) *COULD* easily yield large
> performance benefits, but that's a subject for the Python-3000 mailing
> list; as long as builtins and globals stay as fluid as today, I'm
> skeptical on the optimization opportunities here.

[sreeram;]


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20060824/c3c8699e/attachment.pgp 

From ncoghlan at gmail.com  Thu Aug 24 13:55:16 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 24 Aug 2006 21:55:16 +1000
Subject: [Python-Dev] PEP 362 open issues
In-Reply-To: <bbaeab100608231730q360db337xa525c88078a55a19@mail.gmail.com>
References: <bbaeab100608231559j14d2f7c3s37eef593a9f1809e@mail.gmail.com>	<ecinfj$epu$1@sea.gmane.org>
	<bbaeab100608231730q360db337xa525c88078a55a19@mail.gmail.com>
Message-ID: <44ED93A4.3010601@gmail.com>

Brett Cannon wrote:
>     Another question: could it be helpful to make Parameter.default_value a
>     weak reference?
> 
> 
> Perhaps, but I don't think it is necessarily required.  I can change it 
> if others want to go that way, but for now I am happy with the way it is.

Leave it as a normal reference - if I have a reference to a Signature object, 
I'd like all it's attributes to stick around, thank you very much :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From martin at v.loewis.de  Thu Aug 24 16:30:11 2006
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 24 Aug 2006 16:30:11 +0200
Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple
	arraylookup?
In-Reply-To: <44ED8659.7090602@tachyontech.net>
References: <44ECC72F.10303@tachyontech.net>
	<A1482118-BBCB-429A-AB3C-81B9E206082C@gmail.com>
	<44ED8659.7090602@tachyontech.net>
Message-ID: <1156429811.44edb7f34dbde@www.domainfactory-webmail.de>

Zitat von "K.S.Sreeram" <sreeram at tachyontech.net>:

> How about this approach:
> (disclaimer: this is just a rough sketch!)

This is actually the problem. There are many fine details which
can affect performance and correctness. So about there are only
two sensible ideas to treat such ideas: either implement them,
or ignore them.

If you think your approach could work, please try to implement
it. It's not that anybody is objecting the goal; there is just
debate about the feasibility of the implementation.

So if you can come up with an implementation, we are in a much
better position to praise or criticise the approach: it then
becomes possible to study whether the implementation is really
compatible with the current Python, and whether it really does
improve performance.

Regards,
Martin





From nnorwitz at gmail.com  Thu Aug 24 17:43:38 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 24 Aug 2006 11:43:38 -0400
Subject: [Python-Dev] xrange accepting non-ints
Message-ID: <ee2a432c0608240843y7019d101w45ef59bc60bc11af@mail.gmail.com>

I've been working on enhancing xrange and there are a bunch of issues
to consider.  I've got pretty much complete implementations in both C
and Python.  Currently xrange is 2 objects:  range and the iter.
These only work on C longs.  Here's what I propose:

2.6:
 * Add deprecation warning if a float is passed to xrange (currently
silently truncates)
 * Allow int, long, float, or obj.__index__
 * Implement xrange in python
 * Implement iterator over C longs (or Py_ssize_t) in C
 * Implement iterator over Python longs in Python (* may lose __length_hint__)
 * Capture the values on construction, so mutating objects wouldn't
affect xrange

The idea is to expand xrange's capabilities so that it can replace range in 3k.

I've profiled various combinations.  Here are the various results
normalized doing xrange(0, 1e6, 1):

Run on all integer (32-bit) values for start, step, end:
C xrange and iter:  1
Py xrange w/C iter: 1
Py xrange w/Py iter (gen): 5-8
Py xrange w/Py iter (class): ~30

So having xrange in python is the same speed as if xrange is written
in C.  The important part is that the iterator needs to be written in
C for speed.  If we use a generator, something like:

        while value < end:
            yield value
            value += step

The result is ~5 times slower in a release build and 8 times slower in
a debug build than with an iterator implemented in C.  Using a
generator means that there is no __length_hint__.  If we go with a
full class that has a __length_hint__ the result was ~32 times slower
in a debug build.

The Python impl is about 1/10th the size of the C impl, though is
lacking some comments.

Run on Python longs the result is somewhat interesting.  The Python
based iterator is faster.  There's probably a bug in the C version,
but given that there is a lot of object allocation, I wouldn't expect
the C version to ever be much faster than a similar Python version.
Plus the Python version is trivial (same as above) for ints or longs.
The C version for longs is quite a bit of code.

Run on all Python longs (still 0..1e6, but sys.maxint..(sys.maxint +
1e6) is the same):
C xrange and iter:  1.4
Py xrange w/C iter: not tested
Py xrange w/Py iter (gen): 1
Py xrange w/Py iter (class): 4

Caveats:
 * The generator version above doesn't support floats.  We could
easily support floats with a  different calculation that would be
slightly more expensive, but not have accumulated error.
 * By using the generator version, __length_hint__ gets dropped.  This
means that converting the iterator into a sequence could be slightly
more costly as we have to increase the allocation.  This would only
happen if any of start, step, end weren't an int.
 * With a python implementation there is a little bit of bootstraping
that is necessary to get the iter implemented in C into the xrange
object implemented in Python
 * Since xrange is implemented in Python, it can be changed.
 * The Python code is much easier to understand than the C code (there
is at least one bug in the current C version where -sys.maxint -1
isn't always displayed properly).

Hopefully this is all understandable.  If I left anything out, Thomas
will remind me.

n

From guido at python.org  Thu Aug 24 17:58:19 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 24 Aug 2006 08:58:19 -0700
Subject: [Python-Dev] xrange accepting non-ints
In-Reply-To: <ee2a432c0608240843y7019d101w45ef59bc60bc11af@mail.gmail.com>
References: <ee2a432c0608240843y7019d101w45ef59bc60bc11af@mail.gmail.com>
Message-ID: <ca471dc20608240858x5394ae87kad3776d0a126aa48@mail.gmail.com>

On 8/24/06, Neal Norwitz <nnorwitz at gmail.com> wrote:
> I've been working on enhancing xrange and there are a bunch of issues
> to consider.  I've got pretty much complete implementations in both C
> and Python.  Currently xrange is 2 objects:  range and the iter.
> These only work on C longs.  Here's what I propose:
>
> 2.6:
>  * Add deprecation warning if a float is passed to xrange (currently
> silently truncates)
>  * Allow int, long, float, or obj.__index__

float? I thought the first bullet says no float?

>  * Implement xrange in python

Since xrange is used in performance critical apps that may be a bad
idea. Or maybe only if the args aren't all ints?

>  * Implement iterator over C longs (or Py_ssize_t) in C
>  * Implement iterator over Python longs in Python (* may lose __length_hint__)
>  * Capture the values on construction, so mutating objects wouldn't
> affect xrange

Right. So capture them as Python int or long only.

> The idea is to expand xrange's capabilities so that it can replace range in 3k.
>
> I've profiled various combinations.  Here are the various results
> normalized doing xrange(0, 1e6, 1):
>
> Run on all integer (32-bit) values for start, step, end:
> C xrange and iter:  1
> Py xrange w/C iter: 1
> Py xrange w/Py iter (gen): 5-8
> Py xrange w/Py iter (class): ~30
>
> So having xrange in python is the same speed as if xrange is written
> in C.

I'm not sure I believe this benchmark -- to measure the cost of xrange
itself (as opposed to the cost of iterating over the iterator) you
should test xrange(0) or xrange(1).

> The important part is that the iterator needs to be written in
> C for speed.  If we use a generator, something like:
>
>         while value < end:
>             yield value
>             value += step
>
> The result is ~5 times slower in a release build and 8 times slower in
> a debug build than with an iterator implemented in C.

Nobody cares about the speed of the debug build. :-)

> Using a
> generator means that there is no __length_hint__.  If we go with a
> full class that has a __length_hint__ the result was ~32 times slower
> in a debug build.

I don't mind not having the length hint for longs.

Doesn't current xrange also support a faster version of reversed?

> The Python impl is about 1/10th the size of the C impl, though is
> lacking some comments.
>
> Run on Python longs the result is somewhat interesting.  The Python
> based iterator is faster.  There's probably a bug in the C version,
> but given that there is a lot of object allocation, I wouldn't expect
> the C version to ever be much faster than a similar Python version.
> Plus the Python version is trivial (same as above) for ints or longs.
> The C version for longs is quite a bit of code.

If the longs are large enough, the addition is going to dominate the cost, yes.

> Run on all Python longs (still 0..1e6, but sys.maxint..(sys.maxint +
> 1e6) is the same):
> C xrange and iter:  1.4
> Py xrange w/C iter: not tested
> Py xrange w/Py iter (gen): 1
> Py xrange w/Py iter (class): 4
>
> Caveats:
>  * The generator version above doesn't support floats.  We could
> easily support floats with a  different calculation that would be
> slightly more expensive, but not have accumulated error.

Is there a good use case for supporting float? The problem with floats
is that even apart from accumulated error, it's still ambiguous. E.g.
will xrange(1.1, 1.9, 0.1) include the end point or not? That would
depend on the details of decimal-to-binary conversion.

>  * By using the generator version, __length_hint__ gets dropped.  This
> means that converting the iterator into a sequence could be slightly
> more costly as we have to increase the allocation.  This would only
> happen if any of start, step, end weren't an int.

Fine with me -- you can't do that at all at the moment. :-)

>  * With a python implementation there is a little bit of bootstraping
> that is necessary to get the iter implemented in C into the xrange
> object implemented in Python

Long-term, I'd rather see it implemented all in C. Short term, the
Python implementation is great to experiment.

>  * Since xrange is implemented in Python, it can be changed.
>  * The Python code is much easier to understand than the C code (there
> is at least one bug in the current C version where -sys.maxint -1
> isn't always displayed properly).
>
> Hopefully this is all understandable.  If I left anything out, Thomas
> will remind me.

I'm about to head out to Google now...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From thomas at python.org  Thu Aug 24 18:10:32 2006
From: thomas at python.org (Thomas Wouters)
Date: Thu, 24 Aug 2006 18:10:32 +0200
Subject: [Python-Dev] xrange accepting non-ints
In-Reply-To: <ca471dc20608240858x5394ae87kad3776d0a126aa48@mail.gmail.com>
References: <ee2a432c0608240843y7019d101w45ef59bc60bc11af@mail.gmail.com>
	<ca471dc20608240858x5394ae87kad3776d0a126aa48@mail.gmail.com>
Message-ID: <9e804ac0608240910y5c95734cx841f92d42ecece73@mail.gmail.com>

On 8/24/06, Guido van Rossum <guido at python.org> wrote:
>
> On 8/24/06, Neal Norwitz <nnorwitz at gmail.com> wrote:
> > I've been working on enhancing xrange and there are a bunch of issues
> > to consider.  I've got pretty much complete implementations in both C
> > and Python.  Currently xrange is 2 objects:  range and the iter.
> > These only work on C longs.  Here's what I propose:
> >
> > 2.6:
> >  * Add deprecation warning if a float is passed to xrange (currently
> > silently truncates)
> >  * Allow int, long, float, or obj.__index__
>
> float? I thought the first bullet says no float?


No, the bullet says 'add warning' :) xrange() currently accepts floats,
because it uses one of the integer getargs formats:

>>> xrange(1.2, 2.5, 1.9999)
xrange(1, 2)

>  * Implement xrange in python
>
> Since xrange is used in performance critical apps that may be a bad
> idea. Or maybe only if the args aren't all ints?


Is the cost of *calling* xrange() really a big issue? I don't think Neal
measured this, but he could. I'd imagine performance-critical apps call
xrange() once, then use that to iterate.

>
> > Caveats:
> >  * The generator version above doesn't support floats.  We could
> > easily support floats with a  different calculation that would be
> > slightly more expensive, but not have accumulated error.
>
> Is there a good use case for supporting float? The problem with floats
> is that even apart from accumulated error, it's still ambiguous. E.g.
> will xrange(1.1, 1.9, 0.1) include the end point or not? That would
> depend on the details of decimal-to-binary conversion.


Supporting floats is definately problematic. It would be nice if xrange()
supported arbitrary numeric types, though, like decimals. That would quench
the thirst people seem to have for float-ish xranges.


> >  * With a python implementation there is a little bit of bootstraping
> > that is necessary to get the iter implemented in C into the xrange
> > object implemented in Python
>
> Long-term, I'd rather see it implemented all in C. Short term, the
> Python implementation is great to experiment.


Why, other than performance? It's a lot simpler and much easier to get right
in Python, which is quite good for maintenance, too.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/eca0c56f/attachment-0001.htm 

From fredrik at pythonware.com  Thu Aug 24 18:14:01 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 24 Aug 2006 18:14:01 +0200
Subject: [Python-Dev] xrange accepting non-ints
In-Reply-To: <ee2a432c0608240843y7019d101w45ef59bc60bc11af@mail.gmail.com>
References: <ee2a432c0608240843y7019d101w45ef59bc60bc11af@mail.gmail.com>
Message-ID: <eckj88$toi$1@sea.gmane.org>

Neal Norwitz wrote:

> I've profiled various combinations.  Here are the various results
> normalized doing xrange(0, 1e6, 1):
> 
> Run on all integer (32-bit) values for start, step, end:
> C xrange and iter:  1
> Py xrange w/C iter: 1

in real life, loops are a lot shorter than that.

if you take that into account, you don't even have to run the benchmark 
to realize that calling a Python function and checking the arguments 
before calling a C function takes more time than calling a C function.

even if you skip the "check the arguments" part, you take a 5% hit:

 > timeit -s"def myxrange(a,xrange=xrange): return xrange
   (a)" "for i in myxrange(100): pass"
100000 loops, best of 3: 5.28 usec per loop

 > timeit "for i in xrange(100): pass"
100000 loops, best of 3: 4.98 usec per loop

 > timeit -s"def myxrange(a,b=None,c=None,xrange=xrange):
   return xrange(a,b,c)" "for i in myxrange(0,100,1): pass"
100000 loops, best of 3: 5.58 usec per loop

 > timeit "for i in xrange(0,100,1): pass"
100000 loops, best of 3: 5.27 usec per loop

I doubt adding more code to the myxrange function will speed it up...

</F>


From tim.peters at gmail.com  Thu Aug 24 18:29:50 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 24 Aug 2006 12:29:50 -0400
Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple array
	lookup?
In-Reply-To: <A1482118-BBCB-429A-AB3C-81B9E206082C@gmail.com>
References: <44ECC72F.10303@tachyontech.net>
	<A1482118-BBCB-429A-AB3C-81B9E206082C@gmail.com>
Message-ID: <1f7befae0608240929x1f839d59wc9887748fd4143b3@mail.gmail.com>

Note that there are already three PEPs related to speeding dict-based
namespace access; start with:

    http://www.python.org/dev/peps/pep-0280/

which references the other two.

The "normal path" through dict lookups got faster since there was a
rash of those, to the extent that more complication elsewhere got much
less attractive.  It's possible that dict lookups got slower again
since /then/, though.

From sreeram at tachyontech.net  Thu Aug 24 19:23:32 2006
From: sreeram at tachyontech.net (K.S.Sreeram)
Date: Thu, 24 Aug 2006 22:53:32 +0530
Subject: [Python-Dev] Can LOAD_GLOBAL be optimized to a simple
	array	lookup?
In-Reply-To: <1f7befae0608240929x1f839d59wc9887748fd4143b3@mail.gmail.com>
References: <44ECC72F.10303@tachyontech.net>	<A1482118-BBCB-429A-AB3C-81B9E206082C@gmail.com>
	<1f7befae0608240929x1f839d59wc9887748fd4143b3@mail.gmail.com>
Message-ID: <44EDE094.7000201@tachyontech.net>

Tim Peters wrote:
> Note that there are already three PEPs related to speeding dict-based
> namespace access; start with:
> 
>     http://www.python.org/dev/peps/pep-0280/
> 
> which references the other two.
> 
> The "normal path" through dict lookups got faster since there was a
> rash of those, to the extent that more complication elsewhere got much
> less attractive.  It's possible that dict lookups got slower again
> since /then/, though.

Thanks! Looks like i'm only proposing what is already there in pep-267
by Jeremy Hylton.
http://www.python.org/dev/peps/pep-0267/

cheers
[sreeram;]

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20060824/76d6c88a/attachment.pgp 

From nnorwitz at gmail.com  Thu Aug 24 20:52:12 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 24 Aug 2006 14:52:12 -0400
Subject: [Python-Dev] for 2.5 issues
Message-ID: <ee2a432c0608241152s1f5af2bfl8b054a8381d660ee@mail.gmail.com>

I don't want to make any more changes to 2.5 unless we absolutely have
to.  I also don't want to lose fixes.  How about for anything that
should be resolved in 2.5, but wait for 2.5.1 we set the tracker item
to:  Group 2.5, Resolution: Later, Priority 7.

Then it should be easy to find these things.

Thanks,
n

From brett at python.org  Thu Aug 24 21:21:06 2006
From: brett at python.org (Brett Cannon)
Date: Thu, 24 Aug 2006 12:21:06 -0700
Subject: [Python-Dev] draft for bug guidelines
Message-ID: <bbaeab100608241221i426b135bkf7345c9701f0af81@mail.gmail.com>

Below is the current draft of a set of bug guidelines for people to follow
when they want to file a bug.  The hope is that when we get an email asking
"how do I file a bug?" we can point them towards these sets of guidelines
for most issues.

Let me know about any errors and such.  This will (hopefully) end up going
up on python.org/dev/ eventually.

-------------------------


About
///////////////////////////////////////////////////////////
These sets of guidelines are to help you file a bug report for the
Python programming language on SourceForge_.  If your bug is not for
the language but for a third-party application, please report the bug
to that third-party.

*Please make sure to follow every step as it will make the lives
of the Python developers much easier!!!*

.. contents::

Steps to File a Bug Report
///////////////////////////////////////////////////////////

Get a SourceForge account
=======================================

In order to file a bug report, you must have an account_ on
SourceForge_.  We realize some people would like to have anonymous
bug reporting for various reasons (anonymity, ease or reporting,
etc.).  But the Python developers made the decision to turn off
anonymous posting so as to make sure you are notified by email when
any action is taken on your report.  This can be very important if a
Python developer needs more information from you about the bug.


Start a new bug
=======================================

You must be logged into SourceForge to file a bug!  See `Get a
SourceForge account`_ if you do not have one.

Go to the `SourceForge bug page`_ to start a new bug report.  There
you will find a link called `Submit New`_.  Click on it and it will
allow you to fill out a new bug report.

Once you click on the link, you are presented with a page that has
several fields for you to fill in.  Here is what to do for each field:

* Private
    Leave this unchecked.
* Category
    Set this to the area that the bug is related to (e.g.,
    documentation, build, etc.).
* Group
    Usually this is set to the major.minor version of Python that you
    found the bug in.
* Summary
    A one-liner describing the problem so as to make it easy for
    developers to spot whether they have the expertise needed to work
    on the bug; it also ends up being displayed as the title on index
    and search pages.
* Detailed Description
    Following sections of this document discuss what should go in
    here.
* Upload and Attach a File
    If you are going to upload a file, either type in the path to the
    file or click the ``Browse...`` button to find the file.
* File Description
    A one-liner describing the file (e.g., how it is different from
    previously uploaded files); no date info is needed since the
    upload is timestamped.


Specify the Python version
=======================================

It is important that we have the most accurate version number of the
interpreter you are using in order to best diagnose the issue.  There
are two ways to get us the version information.

If you can run your Python interpreter, execute the following lines at
at a shell prompt and paste the result into the
``Detailed Description`` field of the bug report::

    python -c "import sys; print sys.version"

If you are running a version of Python older than 2.5 and are working
from a source checkout of Python, the please also report the
Subversion revision number for the root of your checkout::

    svnversion .

If your bug is preventing you from running the interpreter, execute
Python with the ``-V`` command-line flag and paste the output::

    python -V


Special settings for your Python interpreter
============================================

Sometimes your environment influences a bug and thus needs to be
reported to help find the problem.  This means we need to have
reported:

* Operating System
* Environment Variables
    + ``PYTHONSTARTUP``
        If this is set and might be causing the issue, please either
        upload the file or state what it does.
    + ``PYTHONCASEOK``
        If your bug is on Windows and involves importing, please
        report if this environment variable is set or not.
* ``site-packages``
    If you have any third-party packages installed that may be
    contributing to the bug, please report those.
* Custom Patches
    Any differences between your code and the code the Python
    developers work off of needs to be reported.


Sample code to reproduce bug
========================================

If you can, please upload a file the demonstrates the bug.  The more
succinct the better!


Submit!
========================================

At this point you should have a detailed bug report for developers to
work off of.  Click the ``Submit`` button and read on to see what you
should do after the bug is reported.


Respond to requests from developers
========================================

No matter how detailed the bug report, there is always the chance
that a developer will need more information to fix a bug.  Please be
prompt in replying to requests for information by submitting a reply
on the bug report.

You may be asked to test out a patch.  It is very important that you
help with this, especially if the bug is not reproducible by the
developer working on it.


Write a patch!
========================================

If you are so inclined, patches for your own bug reports are always
helpful!  Please make sure to reference the tracker item number in the
patch details.  Also add a follow-up to the bug referencing the
patch.



..  _SourceForge: http://www.sourceforge.net/
..  _SourceForge bug page:
http://sourceforge.net/tracker/?group_id=5470&atid=105470
..  _account: http://sourceforge.net/account/newuser_emailverify.php
..  _Submit New:
http://sourceforge.net/tracker/?func=add&group_id=5470&atid=105470
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/a246130c/attachment.html 

From brett at python.org  Thu Aug 24 21:22:42 2006
From: brett at python.org (Brett Cannon)
Date: Thu, 24 Aug 2006 12:22:42 -0700
Subject: [Python-Dev] draft of patch guidelines
Message-ID: <bbaeab100608241222id248e9bn173f2a9aa82b7398@mail.gmail.com>

Below is a draft for a set of patch guidelines.  This is meant for that
times where people ask "how do a create a patch for a change I made" or
where to point people if they created a patch but it comes up short (no
tests, etc.).  Hopefully this will go up on python.org/dev/ .

Let me know of any errors and such.

--------------------------

About
/////

This set of guidelines is meant for people who have written code that
they wish to have committed to the Python programming language.  This
means that this set of guidelines assumes you have already written
code and thus just want to know the steps required to generate the
patch necessary to submit your change for addition in Python.  The
assumption you are using a UNIX-style shell is also made.

If you need help with any of these steps, please ask for it!  Usually
you can create a new patch missing a few steps and the Python
developers will try to help you complete it so that your  patch can
eventuallly be accepted.


Guidelines for committing code to Python will be contained in a
separate document.

.. contents::

Steps to Creating a Patch
/////////////////////////

Read the Developer Intro to understand the scope of your proposed change
========================================================================

Changes to Python come in various levels of scope.  You could be
fixing a simple bug in the standard library.  You might be trying to
add a new feature to a module.  You might even be suggesting a new
feature to the language.  All three have a different amount of scope
and thus require varying amounts of work to get approval.

Please read the `Developer Intro`_ doc to make sure you realize the
proper scope of what you are doing.  It also tells you the rough steps
needed for varying types of changes to be accepted.


Add the appropriate unit tests
==============================

Python prides itself on being a very stable piece of software.  We
maintain this by having a regression test suite that contains a huge
amount of unit tests that exercise both the language and the standard
library.

Any change you make to Python requires a unit test.  If your patch is
to fix a bug, then a test exercising that bug and your fix is needed.
If it is a new feature then an entirely new regression test might be
needed.  The important thing is that your new code is exercised within
Python's test suite.


Add the proper document changes
===============================

If your patch will change or add new semantics, you must update the
documentation.  If your code clarifies some ambiguous behaviour, you
must update the docs.  The only time you do not need to touch the
documentation is if your patch fixes a bug or adds new unit tests.


Make your code follow the style guidelines
==========================================

`PEP 7`_ and `PEP 8`_ are the style guidelines for C and Python code,
respectively.  Your code must conform to the style guidelines before
it is accepted.


Generate a patch
================

In general, people get their copy of the Python source from two
locations: a svn checkout or from downloaded source.  Both have
slightly different steps in terms of generating patches.

Working from a svn checkout is the best way to create a patch.  It
makes it easier on the Python developers since there is less chance of
any merge problems.  It also raises the chances that a developer will
be willing to put the time and effort into working with you and your
patch.  If you need help with working with svn and Python's
repository, see the `Developer FAQ`_.  Working from downloaded source
is so inferior to working with a svn checkout that it will not be
discussed here (although patches *will* be accepted against downloaded
code).

First step is to update your svn checkout with ``svn update``.  Once
you have done that and resolved the merge conflicts, generate a patch
containing the files you have changed.  You can see what files you
have touched with ``svn status``.  If it lists only modified files you
want to include in your patch, you can then generate your patch::

    svn diff > PATCH_FILE

Make sure to name ``PATCH_FILE`` to a reasonable name and be prepared
to upload it in a later step.  Also, please note what revision number
you are working against.  This can be found by executing::

    python -c "import sys; print sys.version"


Create a tracker item on SourceForge
====================================

First make sure you have a `SourceForge account`_.  Once you have one
and havve logged in, go to the `SF patch tracker`_ for Python and
create a new tracker item.

In the new tracker item, set the item's fields as so:

* Private
    Leave unchecked.
* Category
    Set to what area of Python your patch affects.
* Group
    Set to the version of Python that the patch was generated against.
* Assigned To
    Leave as-is.
* Priority:
    Leave as-is.
* Summary
    Briefly describe what the patch does.
* Detailed Description
    Describe in more detail what the patch does.  Make sure to mention
    the revision number the patch is against here.  Also mention any
    bugs that the patch fixes.
* Upload and Attach a File
    Point to the patch you generated.
* File Description
    Mention it is the initial version of the patch.  If you end up
    uploading newer versions of your patch you then specify how it is
    different from the previous one.


Reference the patch in proper bug reports
=========================================

If your patch fixes a bug, add a follow-up to that bug report
referencing the patch.  If there is not bug report but your patch does
fix a bug, then quickly create a bug report.


Wait for a developer to contact you
===================================

At this point you need to wait for a Python developer to come along
and look at your patch.  This might be a while as all Python
developers are volunteers and thus have a limited amount of time.  Do
not be surprised if it takes weeks or more before your patch is
examined.

There is a standing offer that if you review five other patches (by
commenting on its correctness, helping it meet these guidelines,
etc.), then a developer will definitely have a look at any patch you
specify.  Just email python-dev with what five patches you reviewed
(and what you did for the review) and the patch you want examined.



.. _Developer Intro: http://www.python.org/dev/intro/
.. _PEP 7: http://www.python.org/dev/peps/pep-0007/
.. _PEP 8: http://www.python.org/dev/peps/pep-0008/
.. _Developer FAQ: http://www.python.org/dev/faq
.. _SourceForge account:
http://sourceforge.net/account/newuser_emailverify.php
.. _SF patch tracker:
http://sourceforge.net/tracker/?group_id=5470&atid=305470
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/85d3792f/attachment.htm 

From guido at python.org  Thu Aug 24 21:22:52 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 24 Aug 2006 12:22:52 -0700
Subject: [Python-Dev] Need help with test_mutants.py
Message-ID: <ca471dc20608241222m399c3a4p9d2d1bb123fac926@mail.gmail.com>

There's a unit test "test_mutants" which I don't understand. If anyone
remembers what it's doing, please contact me -- after ripping out
dictionary ordering in Py3k, it stops working. In particular, the code
in test_one() requires changes, but I don't know how... Please help!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From phd at mail2.phd.pp.ru  Thu Aug 24 21:29:45 2006
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Thu, 24 Aug 2006 23:29:45 +0400
Subject: [Python-Dev] draft for bug guidelines
In-Reply-To: <bbaeab100608241221i426b135bkf7345c9701f0af81@mail.gmail.com>
References: <bbaeab100608241221i426b135bkf7345c9701f0af81@mail.gmail.com>
Message-ID: <20060824192945.GA32475@phd.pp.ru>

On Thu, Aug 24, 2006 at 12:21:06PM -0700, Brett Cannon wrote:
> Start a new bug

   "Before starting a new bug please try to search if the bug has already
been reported. It it has - do not start a new report, add your comments to
the existing report."

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From phd at mail2.phd.pp.ru  Thu Aug 24 21:33:58 2006
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Thu, 24 Aug 2006 23:33:58 +0400
Subject: [Python-Dev] draft of patch guidelines
In-Reply-To: <bbaeab100608241222id248e9bn173f2a9aa82b7398@mail.gmail.com>
References: <bbaeab100608241222id248e9bn173f2a9aa82b7398@mail.gmail.com>
Message-ID: <20060824193358.GB32475@phd.pp.ru>

On Thu, Aug 24, 2006 at 12:22:42PM -0700, Brett Cannon wrote:
> Read the Developer Intro to understand the scope of your proposed change

   "Search through the PEPs, developer mailing lists and patches. Has a
similar patch already been proposed? Has it been accepted, postponed or
rejected? Learn the reasons why it's been postponed or rejected."

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From tim.peters at gmail.com  Thu Aug 24 21:46:30 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 24 Aug 2006 15:46:30 -0400
Subject: [Python-Dev] Need help with test_mutants.py
In-Reply-To: <ca471dc20608241222m399c3a4p9d2d1bb123fac926@mail.gmail.com>
References: <ca471dc20608241222m399c3a4p9d2d1bb123fac926@mail.gmail.com>
Message-ID: <1f7befae0608241246m7e6e54b9o265362752ab5dd61@mail.gmail.com>

[Guido]
> There's a unit test "test_mutants" which I don't understand. If anyone
> remembers what it's doing, please contact me -- after ripping out
> dictionary ordering in Py3k,

Is any form of dictionary comparison still supported, and, if so, what
does "dict1 cmp_op dict2" mean now?

> it stops working.

Traceback?

> In particular, the code in test_one() requires changes, but I don't
> know how... Please help!

The keys and values of dict1 and dict2 are filled with objects of a
user-defined class whose __cmp__ method randomly mutates dict1 and
dict2.  dict1 and dict2 are initially forced to have the same number
of elements, so in current Python:

        c = cmp(dict1, dict2)

triggers a world of pain, with the internal dict code doing fancy
stuff comparing keys and values.  However, every key and value
comparison /may/ mutate the dicts in arbitrary ways, so this is
testing whether the dict comparison implementation blows up
(segfaults, etc) when the dicts it's comparing mutate during
comparison.

If it's only ordering comparisons that have gone away for dicts, then,
e.g., replacing

    c = cmp(dict1, dict2)

with

    c = dict1 == dict2

instead will still meet the test's intent.

No particular /result/ is expected.  The test passes if and only if
Python doesn't crash.  When the test was introduced, it uncovered at
least six distinct failure (crashing) modes across the first 20 times
it was run, so it's well worth keeping around in some form.

From brett at python.org  Thu Aug 24 22:46:35 2006
From: brett at python.org (Brett Cannon)
Date: Thu, 24 Aug 2006 13:46:35 -0700
Subject: [Python-Dev] draft of patch guidelines
In-Reply-To: <20060824193358.GB32475@phd.pp.ru>
References: <bbaeab100608241222id248e9bn173f2a9aa82b7398@mail.gmail.com>
	<20060824193358.GB32475@phd.pp.ru>
Message-ID: <bbaeab100608241346v5d9cf3e8pc86b1895431833d1@mail.gmail.com>

On 8/24/06, Oleg Broytmann <phd at oper.phd.pp.ru> wrote:
>
> On Thu, Aug 24, 2006 at 12:22:42PM -0700, Brett Cannon wrote:
> > Read the Developer Intro to understand the scope of your proposed change
>
>    "Search through the PEPs, developer mailing lists and patches. Has a
> similar patch already been proposed? Has it been accepted, postponed or
> rejected? Learn the reasons why it's been postponed or rejected."
>
> Oleg.
> --
>      Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
>            Programmers don't die, they just GOSUB without RETURN.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/7e37de70/attachment.html 

From brett at python.org  Thu Aug 24 22:49:25 2006
From: brett at python.org (Brett Cannon)
Date: Thu, 24 Aug 2006 13:49:25 -0700
Subject: [Python-Dev] draft for bug guidelines
In-Reply-To: <20060824192945.GA32475@phd.pp.ru>
References: <bbaeab100608241221i426b135bkf7345c9701f0af81@mail.gmail.com>
	<20060824192945.GA32475@phd.pp.ru>
Message-ID: <bbaeab100608241349s1e8c3c12see196234794e81d4@mail.gmail.com>

Made it the first step.  =)

-Brett

On 8/24/06, Oleg Broytmann <phd at oper.phd.pp.ru> wrote:
>
> On Thu, Aug 24, 2006 at 12:21:06PM -0700, Brett Cannon wrote:
> > Start a new bug
>
>    "Before starting a new bug please try to search if the bug has already
> been reported. It it has - do not start a new report, add your comments to
> the existing report."
>
> Oleg.
> --
>      Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
>            Programmers don't die, they just GOSUB without RETURN.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/685da05b/attachment.htm 

From guido at python.org  Thu Aug 24 23:03:25 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 24 Aug 2006 14:03:25 -0700
Subject: [Python-Dev] Need help with test_mutants.py
In-Reply-To: <1f7befae0608241246m7e6e54b9o265362752ab5dd61@mail.gmail.com>
References: <ca471dc20608241222m399c3a4p9d2d1bb123fac926@mail.gmail.com>
	<1f7befae0608241246m7e6e54b9o265362752ab5dd61@mail.gmail.com>
Message-ID: <ca471dc20608241403o4482f826g818f6b0b3fbc7ec0@mail.gmail.com>

On 8/24/06, Tim Peters <tim.peters at gmail.com> wrote:
> [Guido]
> > There's a unit test "test_mutants" which I don't understand. If anyone
> > remembers what it's doing, please contact me -- after ripping out
> > dictionary ordering in Py3k,
>
> Is any form of dictionary comparison still supported, and, if so, what
> does "dict1 cmp_op dict2" mean now?

Only == and != are supported between dicts. All the work is done by
dict_equal().

> > it stops working.
>
> Traceback?

Not particularly interesting: without changes, the code immediately
bombs like this:

trying w/ lengths 90 90 .
Traceback (most recent call last):
  File "../Lib/test/test_mutants.py", line 152, in <module>
    test(100)
  File "../Lib/test/test_mutants.py", line 149, in test
    test_one(random.randrange(1, 100))
  File "../Lib/test/test_mutants.py", line 135, in test_one
    c = cmp(dict1, dict2)
TypeError: unorderable types: dict() > dict()

> > In particular, the code in test_one() requires changes, but I don't
> > know how... Please help!
>
> The keys and values of dict1 and dict2 are filled with objects of a
> user-defined class whose __cmp__ method randomly mutates dict1 and
> dict2.  dict1 and dict2 are initially forced to have the same number
> of elements, so in current Python:
>
>         c = cmp(dict1, dict2)
>
> triggers a world of pain, with the internal dict code doing fancy
> stuff comparing keys and values.  However, every key and value
> comparison /may/ mutate the dicts in arbitrary ways, so this is
> testing whether the dict comparison implementation blows up
> (segfaults, etc) when the dicts it's comparing mutate during
> comparison.
>
> If it's only ordering comparisons that have gone away for dicts, then,
> e.g., replacing
>
>     c = cmp(dict1, dict2)
>
> with
>
>     c = dict1 == dict2
>
> instead will still meet the test's intent.

I made that change, and changed class Horrid to define __eq__ instead
of __cmp__. Since dict_equal() only invokes PyObject_RichCompareBool()
with op==Py_EQ that should be all that's needed.

Now when I run it, it spits out an apaprently infinite number of dots.
Putting a print in that __eq__ method suggests it is never called. Do
you understand this?

If I change Horrid.__hash__ to always return 42, I get output like this:

trying w/ lengths 12 14
trying w/ lengths 48 52
trying w/ lengths 19 18
trying w/ lengths 10 9
trying w/ lengths 48 46
trying w/ lengths 58 55
trying w/ lengths 50 48
trying w/ lengths 45 50
trying w/ lengths 19 19 .
Traceback (most recent call last):
  File "../Lib/test/test_mutants.py", line 158, in <module>
    test(100)
  File "../Lib/test/test_mutants.py", line 155, in test
    test_one(random.randrange(1, 100))
  File "../Lib/test/test_mutants.py", line 141, in test_one
    c = dict1 == dict2
  File "../Lib/test/test_mutants.py", line 99, in __eq__
    return self.i == other.i
AttributeError: 'Horrid' object has no attribute 'i'
Segmentation fault

But it doesn't always end with a segfault -- most of the time, the
AttributeError is the last thing printed.

> No particular /result/ is expected.  The test passes if and only if
> Python doesn't crash.  When the test was introduced, it uncovered at
> least six distinct failure (crashing) modes across the first 20 times
> it was run, so it's well worth keeping around in some form.

Well, it looks like it did provoke another crash, so I'll play with it
some more. Thanks!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Aug 24 23:49:15 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 24 Aug 2006 14:49:15 -0700
Subject: [Python-Dev] Need help with test_mutants.py
In-Reply-To: <ca471dc20608241403o4482f826g818f6b0b3fbc7ec0@mail.gmail.com>
References: <ca471dc20608241222m399c3a4p9d2d1bb123fac926@mail.gmail.com>
	<1f7befae0608241246m7e6e54b9o265362752ab5dd61@mail.gmail.com>
	<ca471dc20608241403o4482f826g818f6b0b3fbc7ec0@mail.gmail.com>
Message-ID: <ca471dc20608241449l14238481qd0a2c00d65a8685d@mail.gmail.com>

On 8/24/06, Guido van Rossum <guido at python.org> wrote:
> I made that change, and changed class Horrid to define __eq__ instead
> of __cmp__. Since dict_equal() only invokes PyObject_RichCompareBool()
> with op==Py_EQ that should be all that's needed.
>
> Now when I run it, it spits out an apaprently infinite number of dots.
> Putting a print in that __eq__ method suggests it is never called. Do
> you understand this?
>
> If I change Horrid.__hash__ to always return 42, I get output like this:
>
> trying w/ lengths 12 14
> trying w/ lengths 48 52
> trying w/ lengths 19 18
> trying w/ lengths 10 9
> trying w/ lengths 48 46
> trying w/ lengths 58 55
> trying w/ lengths 50 48
> trying w/ lengths 45 50
> trying w/ lengths 19 19 .
> Traceback (most recent call last):
>   File "../Lib/test/test_mutants.py", line 158, in <module>
>     test(100)
>   File "../Lib/test/test_mutants.py", line 155, in test
>     test_one(random.randrange(1, 100))
>   File "../Lib/test/test_mutants.py", line 141, in test_one
>     c = dict1 == dict2
>   File "../Lib/test/test_mutants.py", line 99, in __eq__
>     return self.i == other.i
> AttributeError: 'Horrid' object has no attribute 'i'
> Segmentation fault
>
> But it doesn't always end with a segfault -- most of the time, the
> AttributeError is the last thing printed.

As a follow up to this story line, this appeared to be a refcount bug
in dict_equal(). I believe the same bug is probably present in 2.5; it
isn't triggered by test_mutants.py because that only exercises
dict_compare, not dict_richcompare, and only the latter can call
dict_equal (when op==Py_EQ or op==Py_NE). The bug is here:

                        PyObject *key = a->ma_table[i].me_key;
                        /* temporarily bump aval's refcount to ensure
it stays
                           alive until we're done with it */
                        Py_INCREF(aval);
                        bval = PyDict_GetItem((PyObject *)b, key);

The fix is to put Py_INCREF(key) // Py_DECREF(key) around the call to
PyDict_GetItem(). Apparently what can happen is that the only
reference to the key is in the dict, and the evil mutation from the
comparison delete the object before the comparison is completely done
with it.

Should I attempt to reproduce this bug in 2.5 and fix it?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Aug 25 00:06:44 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 24 Aug 2006 15:06:44 -0700
Subject: [Python-Dev] Need help with test_mutants.py
In-Reply-To: <ca471dc20608241449l14238481qd0a2c00d65a8685d@mail.gmail.com>
References: <ca471dc20608241222m399c3a4p9d2d1bb123fac926@mail.gmail.com>
	<1f7befae0608241246m7e6e54b9o265362752ab5dd61@mail.gmail.com>
	<ca471dc20608241403o4482f826g818f6b0b3fbc7ec0@mail.gmail.com>
	<ca471dc20608241449l14238481qd0a2c00d65a8685d@mail.gmail.com>
Message-ID: <ca471dc20608241506x569abbb0ra3b0a073659a9ea5@mail.gmail.com>

On 8/24/06, Guido van Rossum <guido at python.org> wrote:
> Should I attempt to reproduce this bug in 2.5 and fix it?

Couldn't help myself. The fix is python.org/sf/1546288 . I set the
priority to 8 which means Neal and Anthony will look at it. It's
probably okay to reduce the priority to 7 and fix it in 2.5.1. I
suspect pre-2.5 versions may have the same bug, as long as they have
dict_equal().

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Aug 25 01:08:26 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 24 Aug 2006 16:08:26 -0700
Subject: [Python-Dev] zip -> izip; is __length_hint__ required?
Message-ID: <ca471dc20608241608n7d8ed1c2r5aebb2f77634d5ab@mail.gmail.com>

At today's sprint, Brian Holmes contributed a patch that implements
zip as an interator, a la izip. When reviewing Brian's code, I noticed
that he added an implementation of __length_hint__. My gut feeling is
that this isn't particularly useful given that zip() is almost
exclusively used iteratively, and rarely if ever converted to a list
or a tuple. (The one common exception is in the test suite, but there
it's almost always a short list, and 3 out of 5 were actually tests
for zip or izip.)

Should we rip it out or keep it?

Also, the existing convention for calling __length_hint__ (e.g. in
_PyObject_LengthHint() in abstract.c) seems to be to use
PyObject_CallMethod() and suppress TypeError and AttributeError coming
out of the call. It would seem to make much more sense to check
whether the attribute exists without calling it, and once it exists,
just call it and not suppress any exceptions that come out of it. Is
there any reason why this shouldn't work?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From chad at zetaweb.com  Fri Aug 25 05:01:47 2006
From: chad at zetaweb.com (Chad Whitacre)
Date: Thu, 24 Aug 2006 23:01:47 -0400
Subject: [Python-Dev] draft of patch guidelines
In-Reply-To: <bbaeab100608241222id248e9bn173f2a9aa82b7398@mail.gmail.com>
References: <bbaeab100608241222id248e9bn173f2a9aa82b7398@mail.gmail.com>
Message-ID: <44EE681B.7010601@zetaweb.com>

Brett,

 > Below is a draft for a set of patch guidelines.

Thanks for getting around to this!


 > Wait for a developer to contact you
 > ===================================
 >
 > At this point you need to wait for a Python developer to come along
 > and look at your patch.  This might be a while as all Python
 > developers are volunteers and thus have a limited amount of time.  Do
 > not be surprised if it takes weeks or more before your patch is
 > examined.
 >
 > There is a standing offer that if you review five other patches (by
 > commenting on its correctness, helping it meet these guidelines,
 > etc.), then a developer will definitely have a look at any patch you
 > specify.  Just email python-dev with what five patches you reviewed
 > (and what you did for the review) and the patch you want examined.


May I suggest the following wording for this section? This is basically 
the feedback I found most helpful a few weeks ago; it's more specific 
about how notification and the 5-for-1 rule work.

=================

When you submit your patch, the tracker notifies a mailing list that 
most core Python developers subscribe to. But remember that Open Source 
runs on volunteerism and meritocracy. If you are a new Python 
contributor, then your patch may languish for weeks or more before 
anyone volunteers a response.

Python addresses this barrier to entry with a "5-for-1 rule." Pick five 
other patches or bugs, try to apply or reproduce them, and then report 
your results back to the tracker. Then, send a note to python-dev 
listing the five you reviewed and the patch id of your own patch, and 
one of Python's "patch angels" will get the ball rolling on your patch.

=================





chad

From brett at python.org  Fri Aug 25 06:41:37 2006
From: brett at python.org (Brett Cannon)
Date: Thu, 24 Aug 2006 21:41:37 -0700
Subject: [Python-Dev] draft of patch guidelines
In-Reply-To: <44EE681B.7010601@zetaweb.com>
References: <bbaeab100608241222id248e9bn173f2a9aa82b7398@mail.gmail.com>
	<44EE681B.7010601@zetaweb.com>
Message-ID: <bbaeab100608242141q5403bad4v3fbe73fb6ba41bf0@mail.gmail.com>

On 8/24/06, Chad Whitacre <chad at zetaweb.com> wrote:
>
> Brett,
>
> > Below is a draft for a set of patch guidelines.
>
> Thanks for getting around to this!


Welcome.

> Wait for a developer to contact you
> > ===================================
> >
> > At this point you need to wait for a Python developer to come along
> > and look at your patch.  This might be a while as all Python
> > developers are volunteers and thus have a limited amount of time.  Do
> > not be surprised if it takes weeks or more before your patch is
> > examined.
> >
> > There is a standing offer that if you review five other patches (by
> > commenting on its correctness, helping it meet these guidelines,
> > etc.), then a developer will definitely have a look at any patch you
> > specify.  Just email python-dev with what five patches you reviewed
> > (and what you did for the review) and the patch you want examined.
>
>
> May I suggest the following wording for this section? This is basically
> the feedback I found most helpful a few weeks ago; it's more specific
> about how notification and the 5-for-1 rule work.
>
> =================
>
> When you submit your patch, the tracker notifies a mailing list that
> most core Python developers subscribe to. But remember that Open Source
> runs on volunteerism and meritocracy. If you are a new Python
> contributor, then your patch may languish for weeks or more before
> anyone volunteers a response.
>
> Python addresses this barrier to entry with a "5-for-1 rule." Pick five
> other patches or bugs, try to apply or reproduce them, and then report
> your results back to the tracker. Then, send a note to python-dev
> listing the five you reviewed and the patch id of your own patch, and
> one of Python's "patch angels" will get the ball rolling on your patch.
>
> =================



Changed the wording slightly, but I mostly ended up using your text:

When you submit your patch, the tracker notifies a mailing list that
most core Python developers subscribe to of the creation of your new
patch. But remember that Open Source runs on volunteerism and
meritocracy. Your patch may languish for weeks or more before anyone
volunteers a response because of limited time on the part of Python
developers.

Python addresses this barrier to entry with a "5-for-1 rule." Pick
five other patches or bugs, review them, and then report your results
back to the tracker. Then, send a note to python-dev listing the five
you reviewed and the patch id of your own patch, and a Python
developer will take a look at your patch.


-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060824/1cbfe71a/attachment.htm 

From guido at python.org  Fri Aug 25 19:44:53 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 25 Aug 2006 10:44:53 -0700
Subject: [Python-Dev] Removing anachronisms from logging module
Message-ID: <ca471dc20608251044k41e8b71coa763bdd2bb67dcb@mail.gmail.com>

The logging module currently contains some weird-looking code that is
there to maintain 1.5.2 compatibility (*). I don't think that's needed
any more; neither does Vinay. I propose as a project for someone to
clean it up; perhaps compatibility could be maintained with 2.2 but
that's probably as far back as it needs to go. Linux distros all come
with 2.2 or newer these days.

The logging module is currently listed in PEP 291 as requiring
compatibility with 1.5.2; my proposal is to remove this or replace it
with 2.2 (Vinay even proposed 2.4).

(*) I found an example of code testing "if string.find(s, t) != 0",
thinking it was a bug attempting to write "if t in s", but which Vinay
identified as a 1.5.2 idiom for "if not s.startswith(t)"...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Sat Aug 26 00:16:38 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat, 26 Aug 2006 00:16:38 +0200
Subject: [Python-Dev] Removing anachronisms from logging module
In-Reply-To: <ca471dc20608251044k41e8b71coa763bdd2bb67dcb@mail.gmail.com>
References: <ca471dc20608251044k41e8b71coa763bdd2bb67dcb@mail.gmail.com>
Message-ID: <ecnss5$6mm$1@sea.gmane.org>

Guido van Rossum wrote:

> (*) I found an example of code testing "if string.find(s, t) != 0",
> thinking it was a bug attempting to write "if t in s", but which Vinay
> identified as a 1.5.2 idiom for "if not s.startswith(t)"...

and as noted on the py3k list, not a very good one, given that the usual 
way to write that under 1.5.2 is a lot faster (*).

</F>

*) even faster than "startswith", in many cases.


From guido at python.org  Sat Aug 26 00:46:49 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 25 Aug 2006 15:46:49 -0700
Subject: [Python-Dev] Removing anachronisms from logging module
In-Reply-To: <ecnss5$6mm$1@sea.gmane.org>
References: <ca471dc20608251044k41e8b71coa763bdd2bb67dcb@mail.gmail.com>
	<ecnss5$6mm$1@sea.gmane.org>
Message-ID: <ca471dc20608251546p6f2e1718p6adf0bc19c51d040@mail.gmail.com>

I prefer to focus on "clearer" and "less error-prone" rather than
"faster" in most cases.

I do agree it's a minor embarrassment that s.startswith(t) is slower
than s[:len(t)]==t; but I think we should make startswith() faster
rather than recommending the other idiom.

(Also, for longer substrings startswith wins -- but admittedly rather
long, like 300 chracters.)

--Guido

On 8/25/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Guido van Rossum wrote:
>
> > (*) I found an example of code testing "if string.find(s, t) != 0",
> > thinking it was a bug attempting to write "if t in s", but which Vinay
> > identified as a 1.5.2 idiom for "if not s.startswith(t)"...
>
> and as noted on the py3k list, not a very good one, given that the usual
> way to write that under 1.5.2 is a lot faster (*).
>
> </F>
>
> *) even faster than "startswith", in many cases.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Sat Aug 26 01:01:38 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat, 26 Aug 2006 01:01:38 +0200
Subject: [Python-Dev] Removing anachronisms from logging module
References: <ca471dc20608251044k41e8b71coa763bdd2bb67dcb@mail.gmail.com><ecnss5$6mm$1@sea.gmane.org>
	<ca471dc20608251546p6f2e1718p6adf0bc19c51d040@mail.gmail.com>
Message-ID: <ecnvgk$d6j$1@sea.gmane.org>

Guido van Rossum wrote:

> I prefer to focus on "clearer" and "less error-prone" rather than
> "faster" in most cases.

well, given that "startwith" is by far the most common typo I make
when writing Python programs these days, I'm not so sure about that
error-proneness thing, but never mind...

> I do agree it's a minor embarrassment that s.startswith(t) is slower
> than s[:len(t)]==t; but I think we should make startswith() faster
> rather than recommending the other idiom.

absolutely (see the py3k list for some related discussion).

</F>




From amk at amk.ca  Sat Aug 26 17:58:22 2006
From: amk at amk.ca (A.M. Kuchling)
Date: Sat, 26 Aug 2006 11:58:22 -0400
Subject: [Python-Dev] Removing anachronisms from logging module
In-Reply-To: <ca471dc20608251044k41e8b71coa763bdd2bb67dcb@mail.gmail.com>
References: <ca471dc20608251044k41e8b71coa763bdd2bb67dcb@mail.gmail.com>
Message-ID: <20060826155822.GA5291@Siri.local>

On Fri, Aug 25, 2006 at 10:44:53AM -0700, Guido van Rossum wrote:
> any more; neither does Vinay. I propose as a project for someone to
> clean it up ...

There's a list of simple tasks in the wiki at
<http://wiki.python.org/moin/SimpleTodo>.

--amk

From howarth at bromo.msbb.uc.edu  Sat Aug 26 21:06:00 2006
From: howarth at bromo.msbb.uc.edu (Jack Howarth)
Date: Sat, 26 Aug 2006 15:06:00 -0400 (EDT)
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
Message-ID: <20060826190600.0E75911002B@bromo.msbb.uc.edu>

       I discovered that gcc 4.2 exposes a flaw with
signed integer overflows in python. This bug and the
necessary fix has been discussed in detail on the gcc
mailing list. I have filed a detailed bug report and
the recommended patch proposed by the gcc developers.
This problem should be addressed BEFORE python 2.5 is
released. The bug report is...

[ 1545668 ] gcc trunk (4.2) exposes a signed integer overflows

in the python sourceforge bug tracker. Thanks in advance
for attempting to fix this before Python 2.5 is released.
             Jack

From guido at python.org  Sat Aug 26 23:01:18 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 26 Aug 2006 14:01:18 -0700
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
Message-ID: <ca471dc20608261401s97dea6w8c6287507062573f@mail.gmail.com>

On 8/26/06, Jack Howarth <howarth at bromo.msbb.uc.edu> wrote:
>        I discovered that gcc 4.2 exposes a flaw with
> signed integer overflows in python. This bug and the
> necessary fix has been discussed in detail on the gcc
> mailing list. I have filed a detailed bug report and
> the recommended patch proposed by the gcc developers.
> This problem should be addressed BEFORE python 2.5 is
> released. The bug report is...
>
> [ 1545668 ] gcc trunk (4.2) exposes a signed integer overflows
>
> in the python sourceforge bug tracker. Thanks in advance
> for attempting to fix this before Python 2.5 is released.

I'm not sure I follow why this isn't considered a regression in GCC.
Clearly, on all current hardware, x == -x is also true for the most
negative int (0x80000000 on a 32-bit box). Why is GCC attempting to
break our code (and then blaming us for it!) by using the C standard's
weaselwords that signed integer overflow is undefined, despite that it
has had a traditional meaning on 2's complement hardware for many
decades? If GCC starts to enforce everything that the C standard says
is undefined then very few programs will still work...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From howarth at bromo.msbb.uc.edu  Sat Aug 26 23:14:44 2006
From: howarth at bromo.msbb.uc.edu (Jack Howarth)
Date: Sat, 26 Aug 2006 17:14:44 -0400 (EDT)
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
Message-ID: <20060826211444.68419110019@bromo.msbb.uc.edu>

Guido,
    You'll never win that argument with the gcc developers. If you
rely on undefined behavior in the c language standard, they have
in the past, and will continue to, feel free to ignore those cases.
If you plan on ignoring this issue, just be prepared to see a 
testcase failure in the python testsuite when python is built with
gcc 4.2.
            Jack

From david.nospam.hopwood at blueyonder.co.uk  Sun Aug 27 01:17:17 2006
From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood)
Date: Sun, 27 Aug 2006 00:17:17 +0100
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <ca471dc20608261401s97dea6w8c6287507062573f@mail.gmail.com>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
	<ca471dc20608261401s97dea6w8c6287507062573f@mail.gmail.com>
Message-ID: <44F0D67D.9000007@blueyonder.co.uk>

Guido van Rossum wrote:
> On 8/26/06, Jack Howarth <howarth at bromo.msbb.uc.edu> wrote:
> 
>>       I discovered that gcc 4.2 exposes a flaw with
>>signed integer overflows in python. This bug and the
>>necessary fix has been discussed in detail on the gcc
>>mailing list. I have filed a detailed bug report and
>>the recommended patch proposed by the gcc developers.
>>This problem should be addressed BEFORE python 2.5 is
>>released. The bug report is...
>>
>>[ 1545668 ] gcc trunk (4.2) exposes a signed integer overflows
>>
>>in the python sourceforge bug tracker. Thanks in advance
>>for attempting to fix this before Python 2.5 is released.
> 
> I'm not sure I follow why this isn't considered a regression in GCC.
> Clearly, on all current hardware, x == -x is also true for the most
> negative int (0x80000000 on a 32-bit box). Why is GCC attempting to
> break our code (and then blaming us for it!) by using the C standard's
> weaselwords that signed integer overflow is undefined, despite that it
> has had a traditional meaning on 2's complement hardware for many
> decades? If GCC starts to enforce everything that the C standard says
> is undefined then very few programs will still work...

+1. It's clearly undefined behaviour, but that doesn't mean that it is a
good idea for gcc to be making changes that could silently break a lot of
code, in pursuit of a microoptimization that is unlikely to significantly
help performance.

CPython should be fixed anyway. The correct fix is
"if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)".

-- 
David Hopwood <david.nospam.hopwood at blueyonder.co.uk>



From thomas at python.org  Sun Aug 27 01:54:08 2006
From: thomas at python.org (Thomas Wouters)
Date: Sat, 26 Aug 2006 19:54:08 -0400
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <44F0D67D.9000007@blueyonder.co.uk>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
	<ca471dc20608261401s97dea6w8c6287507062573f@mail.gmail.com>
	<44F0D67D.9000007@blueyonder.co.uk>
Message-ID: <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com>

On 8/26/06, David Hopwood <david.nospam.hopwood at blueyonder.co.uk> wrote:

> CPython should be fixed anyway. The correct fix is
> "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)".


Why not just "... && x == LONG_MIN"?

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060826/11fa9509/attachment.html 

From dberlin at dberlin.org  Sun Aug 27 01:36:34 2006
From: dberlin at dberlin.org (Daniel Berlin)
Date: Sat, 26 Aug 2006 19:36:34 -0400
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <20060826211444.68419110019@bromo.msbb.uc.edu>
References: <20060826211444.68419110019@bromo.msbb.uc.edu>
Message-ID: <44F0DB02.6030804@dberlin.org>

Jack Howarth wrote:
> Guido,
>     You'll never win that argument with the gcc developers. If you
> rely on undefined behavior in the c language standard, they have
> in the past, and will continue to, feel free to ignore those cases.
> If you plan on ignoring this issue, just be prepared to see a 
> testcase failure in the python testsuite when python is built with
> gcc 4.2.

In addition, i'm surprised that you don't see these failures on other
compilers.  I know XLC, for example, would treat signed integer overflow
as undefined at all opt levels as well.
I would not be surprised to find Intel's compiler doing the same thing.

Speaking as a gcc optimization person, if we were to treat signed
integer overflow as wrapping, you would inhibit a very large number of
loop optimizations on a very large class of loops (due to a number of
reasons, from no longer being able to prove termination of loops or
estimation of number of iterations, to other things).

Heck, XLC at -O3+ will ignore *unsigned* integer wraparound of loop
indices unless you use -qstrict_induction.
As the docs for this option say, using that option can cause severe
performance degradation (I happen to know the numbers from when i worked
at IBM, and they are surprisingly high).

GCC is willing to take the performance hit relative to other compilers
on things like that to be a standard conforming compiler, just like we
are willing to optimize code based on the assumption that you do not
invoke undefined behavior.

--Dan

From tim.peters at gmail.com  Sun Aug 27 02:37:46 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 26 Aug 2006 20:37:46 -0400
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
	<ca471dc20608261401s97dea6w8c6287507062573f@mail.gmail.com>
	<44F0D67D.9000007@blueyonder.co.uk>
	<9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com>
Message-ID: <1f7befae0608261737p595b805fnee9d65762fe1af19@mail.gmail.com>

[David Hopwood]
>> CPython should be fixed anyway. The correct fix is
>> "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)".

Note that this was already suggested in the bug report.

[Thomas Wouters]
> Why not just "... && x == LONG_MIN"?

In full,

    if (y == -1 && x == LONG_MIN)

"should work" too.  In practice we try to avoid numeric symbols from
platform header files because so many platforms have screwed these up
over the centuries (search for LONG_BIT or HUGE_VAL ;-)), and because
it's better (when possible) not to tie the code to that `x` was
specifically declared as type "long" (e.g., just more stuff that will
break if Python decides to make its short int of type PY_LONG_LONG
instead).  In this specific case, there may also have been a desire to
avoid generating a memory load for a fat constant.  However, since
this is integer division, in real life (outside the test suite) we'll
never go beyond the "y == -1" test.

From david.nospam.hopwood at blueyonder.co.uk  Sun Aug 27 03:18:37 2006
From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood)
Date: Sun, 27 Aug 2006 02:18:37 +0100
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>	
	<ca471dc20608261401s97dea6w8c6287507062573f@mail.gmail.com>	
	<44F0D67D.9000007@blueyonder.co.uk>
	<9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com>
Message-ID: <44F0F2ED.3020907@blueyonder.co.uk>

Thomas Wouters wrote:
> On 8/26/06, David Hopwood <david.nospam.hopwood at blueyonder.co.uk> wrote:
> 
>> CPython should be fixed anyway. The correct fix is
>> "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)".
> 
> Why not just "... && x == LONG_MIN"?

Because the intent is to check that x / y does not overflow a long, and
x == LONG_MIN would not cause an overflow on 1's complement or sign-magnitude
systems.

(CPython has probably only been tested on 2's complement systems anyway,
but if we're going to be pedantic about depending only on things in the
C standard...)

-- 
David Hopwood <david.nospam.hopwood at blueyonder.co.uk>



From tim.peters at gmail.com  Sun Aug 27 05:24:51 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 26 Aug 2006 23:24:51 -0400
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <44F0F2ED.3020907@blueyonder.co.uk>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
	<ca471dc20608261401s97dea6w8c6287507062573f@mail.gmail.com>
	<44F0D67D.9000007@blueyonder.co.uk>
	<9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com>
	<44F0F2ED.3020907@blueyonder.co.uk>
Message-ID: <1f7befae0608262024i4c712c3et12da59644607f526@mail.gmail.com>

[David Hopwood]
> (CPython has probably only been tested on 2's complement systems anyway,

Definitely so.  Are there any boxes using 1's-comp or sign-magnitude
integers anymore?  Python assumes 2's-comp in many places.

> but if we're going to be pedantic about depending only on things in the
> C standard...)

No, in that respect we're driven by the silliest decisions made by C
compiler writers ;-)

From kbk at shore.net  Sun Aug 27 05:27:23 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Sat, 26 Aug 2006 23:27:23 -0400 (EDT)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200608270327.k7R3RNcT027925@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  407 open ( +3) /  3393 closed (+17) /  3800 total (+20)
Bugs    :  888 open (+28) /  6145 closed (+14) /  7033 total (+42)
RFE     :  232 open ( +3) /   236 closed ( +1) /   468 total ( +4)

New / Reopened Patches
______________________

most missing from Windows distribution  (2006-08-16)
CLOSED http://python.org/sf/1541412  opened by  Jim Jewett

buffer overrun in repr() for unicode strings  (2006-08-16)
CLOSED http://python.org/sf/1541585  opened by  Simon Law

work around clocks with low resolution (uuid.py)  (2006-08-17)
       http://python.org/sf/1541863  opened by  Hirokazu Yamamoto

patch in bug 1542016  (2006-08-17)
CLOSED http://python.org/sf/1542019  opened by  Santiago Gala

fix crash with continue in nested try/finally  (2006-08-18)
       http://python.org/sf/1542451  opened by  Neal Norwitz

Improve dynamic linking support on AIX  (2006-08-18)
       http://python.org/sf/1542544  opened by  G?ran Uddeborg

Tweak pydoc to speak about with and CONTEXTMANAGERS  (2006-08-18)
       http://python.org/sf/1542681  opened by  Santiago Gala

Fix the vc8 solution files  (2006-08-18)
       http://python.org/sf/1542946  opened by  baus

urllib2 regression  (2006-08-19)
CLOSED http://python.org/sf/1542948  opened by  John J Lee

tarfile.py: fix for bug 1543303  (2006-08-21)
CLOSED http://python.org/sf/1543897  opened by  Lars Gust?bel

Socket module is not thread-safe  (2006-08-21)
       http://python.org/sf/1544279  opened by  Maxim Sobolev

Implementation of PEP 362  (2006-08-22)
       http://python.org/sf/1544909  opened by  Brett Cannon

Fixes SocketServer Bug 1531963  (2006-08-23)
       http://python.org/sf/1545011  opened by  Damon Kohler

new splicetee module  (2006-08-23)
       http://python.org/sf/1545262  opened by  Omar AitMous

new directio module  (2006-08-23)
       http://python.org/sf/1545275  opened by  Omar AitMous

xrange that supports longs, etc  (2006-08-24)
       http://python.org/sf/1546078  opened by  Neal Norwitz

crash in dict_equal  (2006-08-24)
       http://python.org/sf/1546288  opened by  Guido van Rossum

Create a real zip iterator object; not using itertools.izip  (2006-08-24)
CLOSED http://python.org/sf/1546297  opened by  Brian Holmes

broken error reporting when filename specified by -s missing  (2006-08-25)
       http://python.org/sf/1546372  opened by  lplatypus

Patches Closed
______________

most missing from Windows distribution  (2006-08-16)
       http://python.org/sf/1541412  closed by  jimjjewett

buffer overrun in repr() for unicode strings  (2006-08-16)
       http://python.org/sf/1541585  closed by  gbrandl

broken shortcut keys  (2006-08-15)
       http://python.org/sf/1540874  closed by  kbk

patch in bug 1542016  (2006-08-17)
       http://python.org/sf/1542019  closed by  gbrandl

urllib2 regression  (2006-08-19)
       http://python.org/sf/1542948  closed by  gbrandl

Backports from trunk to release24-maint  (2006-08-15)
       http://python.org/sf/1540329  closed by  gbrandl

tarfile.py: fix for bug 1543303  (2006-08-21)
       http://python.org/sf/1543897  closed by  nnorwitz

Add some explication to PEP 3100  (2006-07-13)
       http://python.org/sf/1522038  closed by  bcannon

Create a real zip iterator object; not using itertools.izip  (2006-08-24)
       http://python.org/sf/1546297  closed by  gvanrossum

Add readinto method to StringIO and cStringIO  (2006-08-12)
       http://python.org/sf/1539381  closed by  bcannon

Remove the repr()-backticks  (2006-06-04)
       http://python.org/sf/1500623  closed by  bcannon

Move reduce() to functools  (2006-06-28)
       http://python.org/sf/1513870  closed by  gvanrossum

New / Reopened Bugs
___________________

infinite __str__ recursion in thread causes seg fault  (2003-07-31)
       http://python.org/sf/780714  reopened by  gbrandl

tools and demo missing from windows  (2006-08-16)
       http://python.org/sf/1541420  opened by  Jim Jewett

bug in python 2.4.3 for windows?  (2006-08-16)
CLOSED http://python.org/sf/1541566  opened by  tinkiwinky

Compiler command percent-sign causes format string error  (2006-08-16)
CLOSED http://python.org/sf/1541642  opened by  Matthew Cowles

bsddb can't use unicode filenames  (2006-08-17)
       http://python.org/sf/1541671  opened by  Zooko O'Whielacronx

Incorrect example calls to PyObject_SetItem  (2006-08-17)
CLOSED http://python.org/sf/1541682  opened by  John Machin

Recently introduced sgmllib regexp bug hangs Python  (2006-08-17)
       http://python.org/sf/1541697  opened by  John J Lee

Exceptions don't call _PyObject_GC_UNTRACK(self)  (2006-08-17)
       http://python.org/sf/1542051  opened by  ?iga Seilnacht

global variable: multiple id()-addresses   (2006-08-17)
CLOSED http://python.org/sf/1542166  opened by  Frank R. Schaefer

Nested finally in generators don't follow PEP 342  (2006-08-17)
       http://python.org/sf/1542308  opened by  Bob Ippolito

httplib reads one byte per system call  (2006-08-18)
       http://python.org/sf/1542407  opened by  Zoyd Wheeler

python-2.5c1.msi contains ICE validation errors and warnings  (2006-08-18)
       http://python.org/sf/1542432  opened by  Bob Arnson

Bug in definition of PyImport_ImportModuleEx macro  (2006-08-18)
CLOSED http://python.org/sf/1542693  opened by  Chris Stawarz

idle in python 2.5c1 freezes on macos 10.3.9  (2006-08-18)
       http://python.org/sf/1542949  opened by  David Strozzi

tarfile in mode w|gz adds padding that annoys gunzip  (2006-08-19)
CLOSED http://python.org/sf/1543303  opened by  alexis

"from __future__ import foobar;" causes wrong SyntaxError  (2006-08-20)
       http://python.org/sf/1543306  opened by  daniel hahler

Operator precedence inconsistent for complex literal  (2006-08-20)
CLOSED http://python.org/sf/1543347  opened by  [N/A]

test_tempfile fails on cygwin  (2006-08-20)
       http://python.org/sf/1543467  opened by  Miki Tebeka

test_subprocess fails on cygwin  (2006-08-20)
       http://python.org/sf/1543469  opened by  Miki Tebeka

md5 sums are different between Solaris and Windows XP SP1  (2006-08-21)
       http://python.org/sf/1543801  reopened by  sunmountain

md5 sums are different between Solaris and Windows XP SP1  (2006-08-21)
       http://python.org/sf/1543801  opened by  Stefan Sonnenberg

ctypes unit test fails (test_macholib.py) under MacOS 10.4.7  (2006-08-21)
       http://python.org/sf/1544102  opened by  M. J. Fromberger

test_anydbm segmentation fault  (2006-08-21)
       http://python.org/sf/1544106  opened by  Clay Spence

python compiler support for with stmt  (2006-08-21)
       http://python.org/sf/1544277  opened by  Neal Norwitz

Fix Lib/test/test___all__.py  (2006-08-21)
       http://python.org/sf/1544295  opened by  Hasan Diwan

checking size of int... configure: error: cannot compute siz  (2006-08-21)
       http://python.org/sf/1544306  opened by  arrecostao

_ctypes fails to build on Solaris x86 32-bit (Sun compiler)  (2006-08-21)
       http://python.org/sf/1544339  opened by  Case Van Horsen

Bad result of calculation  (2006-08-22)
CLOSED http://python.org/sf/1544381  opened by  Jean-Christophe BERTOLINI

x!=y and [x]=[y] (!)  (2006-08-22)
       http://python.org/sf/1544762  opened by  Alex Martelli

setup() keyword have to be list (doesn't work with tuple)  (2006-08-23)
       http://python.org/sf/1545341  opened by  STINNER Victor

New-style classes fail to cleanup attributes  (2006-08-23)
       http://python.org/sf/1545463  opened by  Alexander Belopolsky

inconsistent treatment of NULs in int()  (2006-08-23)
       http://python.org/sf/1545497  opened by  Neal Norwitz

distutils home scheme lacks python versioning  (2006-08-24)
       http://python.org/sf/1545658  opened by  John Levon

distutils needs vendor-packages support  (2006-08-24)
       http://python.org/sf/1545659  opened by  John Levon

gcc trunk (4.2) exposes a signed integer overflows  (2006-08-24)
       http://python.org/sf/1545668  opened by  Jack Howarth

structmember T_LONG won't accept a python long  (2006-08-24)
       http://python.org/sf/1545696  opened by  Roger Upole

Incomplete info in 7.18.1 ZipFile Objects  (2006-08-24)
       http://python.org/sf/1545836  opened by  Taco

array.array borks on deepcopy  (2006-08-24)
       http://python.org/sf/1545837  opened by  V?clav Haisman

PyString_FromString() clarification  (2006-08-24)
       http://python.org/sf/1546052  opened by  Christian Walther

bcannon secure branch issues  (2006-08-24)
CLOSED http://python.org/sf/1546203  opened by  Jim Jewett

subprocess.Popen can't read file object as stdin after seek  (2006-08-25)
       http://python.org/sf/1546442  opened by  GaryD

String methods don't support explicit None arguments  (2006-08-25)
       http://python.org/sf/1546585  opened by  Nick Coghlan

urlparse.urljoin odd behaviour  (2006-08-25)
       http://python.org/sf/1546628  opened by  Andres Riancho

Bugs Closed
___________

Python 2.5 svn crash in _elementtree.c  (2006-08-04)
       http://python.org/sf/1534630  closed by  effbot

bug in python 2.4.3 for windows?  (2006-08-16)
       http://python.org/sf/1541566  closed by  gbrandl

Compiler command percent-sign causes format string error  (2006-08-16)
       http://python.org/sf/1541642  deleted by  mdcowles

Incorrect example calls to PyObject_SetItem  (2006-08-17)
       http://python.org/sf/1541682  closed by  gbrandl

global variable: multiple id()-addresses   (2006-08-17)
       http://python.org/sf/1542166  closed by  gbrandl

Bug in definition of PyImport_ImportModuleEx macro  (2006-08-18)
       http://python.org/sf/1542693  closed by  nnorwitz

tarfile in mode w|gz adds padding that annoys gunzip  (2006-08-19)
       http://python.org/sf/1543303  closed by  nnorwitz

Operator precedence inconsistent for complex literal  (2006-08-20)
       http://python.org/sf/1543347  closed by  gbrandl

md5 sums are different between Solaris and Windows XP SP1  (2006-08-21)
       http://python.org/sf/1543801  closed by  sunmountain

Bad result of calculation  (2006-08-22)
       http://python.org/sf/1544381  closed by  gbrandl

bcannon secure branch issues  (2006-08-24)
       http://python.org/sf/1546203  closed by  bcannon

New / Reopened RFE
__________________

Optimizations for cgi.FieldStorage methods  (2006-08-16)
       http://python.org/sf/1541463  opened by  Bob Kline

wsgi.org link in wsgiref  (2006-08-18)
       http://python.org/sf/1542920  opened by  Ian Bicking

RFE Closed
__________

extend strptime to understand logging timestamps  (2004-08-10)
       http://python.org/sf/1006786  closed by  bcannon


From howarth at bromo.msbb.uc.edu  Sun Aug 27 16:36:04 2006
From: howarth at bromo.msbb.uc.edu (Jack Howarth)
Date: Sun, 27 Aug 2006 10:36:04 -0400 (EDT)
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
Message-ID: <20060827143604.EB1A0110019@bromo.msbb.uc.edu>

    I believe some of the others here might be interested in
some other postings on the gcc mailing list regarding this issue
(which weren't cross-posted here)...

http://gcc.gnu.org/ml/gcc/2006-08/msg00516.html
http://gcc.gnu.org/ml/gcc/2006-08/msg00517.html

It makes clear that the impact of this change in gcc was considered
and the reasoning on their decision to follow the standard so closely.

http://gcc.gnu.org/ml/gcc/2005-06/msg01238.html

Just so other beside Guido see those.
              Jack

From RHettinger at ewtllc.com  Sun Aug 27 17:26:10 2006
From: RHettinger at ewtllc.com (Raymond Hettinger)
Date: Sun, 27 Aug 2006 08:26:10 -0700
Subject: [Python-Dev] zip -> izip; is __length_hint__ required?
References: <ca471dc20608241608n7d8ed1c2r5aebb2f77634d5ab@mail.gmail.com>
Message-ID: <B6FAC926EFE7B348B12F29CF7E4A93D47FB0A2@hammer.office.bhtrader.com>

Yes, please rip that out.  The patch should be a direct copy of the code in the itertools module.  The use of length-hint was deliberately left-out of izip().
 
Also, yes it would be fine to alter the code in abstract.c to LBYL instead of suppressing exceptions.
 
 
Raymond

________________________________

From: gvanrossum at gmail.com on behalf of Guido van Rossum
Sent: Thu 8/24/2006 4:08 PM
To: Raymond Hettinger; python-dev at python.org; Brian Holmes
Subject: zip -> izip; is __length_hint__ required?



At today's sprint, Brian Holmes contributed a patch that implements
zip as an interator, a la izip. When reviewing Brian's code, I noticed
that he added an implementation of __length_hint__. My gut feeling is
that this isn't particularly useful given that zip() is almost
exclusively used iteratively, and rarely if ever converted to a list
or a tuple. (The one common exception is in the test suite, but there
it's almost always a short list, and 3 out of 5 were actually tests
for zip or izip.)

Should we rip it out or keep it?

Also, the existing convention for calling __length_hint__ (e.g. in
_PyObject_LengthHint() in abstract.c) seems to be to use
PyObject_CallMethod() and suppress TypeError and AttributeError coming
out of the call. It would seem to make much more sense to check
whether the attribute exists without calling it, and once it exists,
just call it and not suppress any exceptions that come out of it. Is
there any reason why this shouldn't work?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060827/1360bfc8/attachment.htm 

From chad at zetaweb.com  Sun Aug 27 20:12:28 2006
From: chad at zetaweb.com (Chad Whitacre)
Date: Sun, 27 Aug 2006 14:12:28 -0400
Subject: [Python-Dev] draft of patch guidelines
In-Reply-To: <bbaeab100608252322w76f95620nfd46f5ece24f4fa@mail.gmail.com>
References: <324634B71B159D469BCEB616678A6B94F94BA7@ingdexs1.ingdirect.com>
	<bbaeab100608252322w76f95620nfd46f5ece24f4fa@mail.gmail.com>
Message-ID: <44F1E08C.3080805@zetaweb.com>

Brett,

 >> I liked the "patch angel" term in Chad's version. Stating "a Python
 >> developer will take a look at your patch" smacks of a guarantee,
 >> while Chad's use of "patch angel" and "get the ball rolling" better
 >> conveyed the fact that this 5-for-1 rule is simply a practice of some
 >> of the volunteers.
 >
 > Eh, I still don't like the term.  Too hokey.  But I did toss in a
 > "try" to not make it sound like a guarantee.

Yeah, kind of hokey. I included it because it's succinct and accurate, 
and because I assumed there's history to the term. Putting it in quotes 
helped w/ the goofiness, I thought.

I believe Skip used the term: any comment, Skip?



chad

From chad at zetaweb.com  Sun Aug 27 20:41:21 2006
From: chad at zetaweb.com (Chad Whitacre)
Date: Sun, 27 Aug 2006 14:41:21 -0400
Subject: [Python-Dev] draft of patch guidelines
In-Reply-To: <bbaeab100608242141q5403bad4v3fbe73fb6ba41bf0@mail.gmail.com>
References: <bbaeab100608241222id248e9bn173f2a9aa82b7398@mail.gmail.com>	
	<44EE681B.7010601@zetaweb.com>
	<bbaeab100608242141q5403bad4v3fbe73fb6ba41bf0@mail.gmail.com>
Message-ID: <44F1E751.7080002@zetaweb.com>

Brett,

 > When you submit your patch, the tracker notifies a mailing list that
 > most core Python developers subscribe to of the creation of your new
 > patch.

Isn't "of the creation of your new patch" redundant? What else would it 
be notifying the list of?


 > Your patch may languish for weeks or more before anyone volunteers a
 > response because of limited time on the part of Python developers.

I think this is less accurate. Patches languish because of limited time 
*and* because newbies don't have any social capital w/in the Python 
community. New patch contributors are volunteers too, so they understand 
that constraint. Their big problem is their outsider status, to which 
the patch angels/5-for-1 system is an elegant solution. And that's what 
should be emphasized in this FAQ.

In other words, I still prefer my wording. But since I'm a newbie with 
little merit, I'll be content with whatever you decide. :)



chad

From brett at python.org  Mon Aug 28 02:17:32 2006
From: brett at python.org (Brett Cannon)
Date: Sun, 27 Aug 2006 17:17:32 -0700
Subject: [Python-Dev] draft of patch guidelines
In-Reply-To: <44F1E751.7080002@zetaweb.com>
References: <bbaeab100608241222id248e9bn173f2a9aa82b7398@mail.gmail.com>
	<44EE681B.7010601@zetaweb.com>
	<bbaeab100608242141q5403bad4v3fbe73fb6ba41bf0@mail.gmail.com>
	<44F1E751.7080002@zetaweb.com>
Message-ID: <bbaeab100608271717t11e15552n5fac44354678e263@mail.gmail.com>

On 8/27/06, Chad Whitacre <chad at zetaweb.com> wrote:
>
> Brett,
>
> > When you submit your patch, the tracker notifies a mailing list that
> > most core Python developers subscribe to of the creation of your new
> > patch.
>
> Isn't "of the creation of your new patch" redundant? What else would it
> be notifying the list of?


Yep, slightly.

> Your patch may languish for weeks or more before anyone volunteers a
> > response because of limited time on the part of Python developers.
>
> I think this is less accurate. Patches languish because of limited time
> *and* because newbies don't have any social capital w/in the Python
> community. New patch contributors are volunteers too, so they understand
> that constraint. Their big problem is their outsider status, to which
> the patch angels/5-for-1 system is an elegant solution. And that's what
> should be emphasized in this FAQ.


But that makes us sound elitist.

In other words, I still prefer my wording. But since I'm a newbie with
> little merit, I'll be content with whatever you decide. :)


OK.  =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060827/4ed0dd87/attachment.htm 

From david.nospam.hopwood at blueyonder.co.uk  Mon Aug 28 02:46:10 2006
From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood)
Date: Mon, 28 Aug 2006 01:46:10 +0100
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <20060827143604.EB1A0110019@bromo.msbb.uc.edu>
References: <20060827143604.EB1A0110019@bromo.msbb.uc.edu>
Message-ID: <44F23CD2.5090609@blueyonder.co.uk>

Jack Howarth wrote:
>     I believe some of the others here might be interested in
> some other postings on the gcc mailing list regarding this issue
> (which weren't cross-posted here)...
> 
> http://gcc.gnu.org/ml/gcc/2006-08/msg00516.html
> http://gcc.gnu.org/ml/gcc/2006-08/msg00517.html
> 
> It makes clear that the impact of this change in gcc was considered
> and the reasoning on their decision to follow the standard so closely.
> 
> http://gcc.gnu.org/ml/gcc/2005-06/msg01238.html
> 
> Just so other beside Guido see those.

This discussion just highlights how different the perspective of much of
the C community (especially its language committee and compiler writers)
is on error handling and language safety, relative to the perspective of
the community that uses Python and other memory-safe languages.

The C perspective is that programs have to be correct, and if they're
not correct, it basically doesn't matter what happens.

The safe language perspective is that of course we try to write correct
programs, but if a program is not correct, then it *really matters* what
the error behaviour is. Ideally, it should be something that facilitates
debugging and that doesn't silently propagate an incorrect result.
Undefined behaviour is right out.

For example, from <http://gcc.gnu.org/ml/gcc/2006-08/msg00522.html>:

# The gcc developers always face two competing constituencies: do not
# change the compiler behaviour so that existing code which relies on
# undefined behaviour continues to work, and do change the compiler
# behaviour so that new code which does not rely on undefined behaviour
# runs faster.  In general, being compiler developers, we come down on
# the side of making code which does not rely on undefined behaviour run
# faster.

... and rarely even consider whether the behaviour *should* have been
undefined or not.


(There are many people who use C but do not agree with that perspective
on error handling -- I'm one of them. These people often devise elaborate
error handling frameworks, and compile at low optimization levels when
it doesn't hurt performance.)

-- 
David Hopwood <david.nospam.hopwood at blueyonder.co.uk>



From igor.bukanov at gmail.com  Wed Aug 23 16:06:01 2006
From: igor.bukanov at gmail.com (Igor Bukanov)
Date: Wed, 23 Aug 2006 16:06:01 +0200
Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary
In-Reply-To: <44EC3CF5.8030502@gmail.com>
References: <df0b33100608220617y5760c9f8q8fe713b2e1fc6c7b@mail.gmail.com>
	<44EC3CF5.8030502@gmail.com>
Message-ID: <df0b33100608230706t25084276i945aa6e0623f3ac5@mail.gmail.com>

On 8/23/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Simpler is in the eye of the beholder - the current close() merely uses
> throw() to raise a particular kind of exception 'asynchronously' (which is
> already a familiar concept due to KeyboardInterrupt). What you're suggesting
> is a completely new flow control concept that doesn't exist anywhere else in
> the language.

Yes, this is a new feature, but IMO the extra complexity it brings to
the implementation is worth simplifications for the user.

>
> I suggested during the PEP 352 discussions that GeneratorExit should inherit
> directly from BaseException, so that the fix to your example would be to
> replace the bare "except:" with "except Exception:"

That is, the idea was to make writing code that catches GeneratorExit
difficult, right? But then note that if the code never catches
GeneratorExit, then throwing GeneratorExit after the yield is exactly
equivalent to just returning after the yield. In either case only
finally blocks would be executed and close then would ignore
GeneratorExit or StopIteration exceptions.

So this proposal is just an extension of the idea "it should be
difficult to use catch to ignore GeneratorExit" to simpler "No catch
can catch GeneratorExit".

> (because it can swallow
> KeyboardInterrupt the version you posted is buggy regardless of how g.close()
> works).

So there are 4 kinds of exceptions now in Python:

1. Normal exceptions that are OK to catch and ignore.
2. Interrupts like KeyboardInterrupt that can be caught and ignored,
but this is not recommended and it is easy to avoid catching them
accidentally.
3. Interrupts like GeneratorExit that can be caught and ignored, but
this is not recommended and where one has to write a lot of code to
avoid catching them in a generic catch block.
4. Control transfer "exceptions" like break and continue across
finally blocks. With enough efforts one can catch and ignore them, but
then one know what he is doing:

for i in range(3):
  print "i:", i
  try:
    try:
      break
    finally:
      raise Exception
  except Exception:
    print "Ignoring break"

print "Done"

Ideally it would be nice to merge 2-3-4 into a single class of special
exceptions where the normal catch statement never sees them and extra
efforts are necessary to ignore them, but with this proposal at least
the list would not contain the item 3.

Please also note that these comments are born after playing with
generators in JS 1.7, the next JavaScript that Firefox 2.0 would
implement. The current JS implementation tries to follow PEP 352, but
since the only catch available in JS1.7 is just catch-all as before,
the problems with accidentally catching GeneratorExit are much more
visible in JS [1]. Thus changing close in JS to just return after
yield would address the issues nicely but perhaps this can help Python
users as well.

Regards, Igor

[1]  https://bugzilla.mozilla.org/show_bug.cgi?id=349331

From igor.bukanov at gmail.com  Thu Aug 24 00:52:09 2006
From: igor.bukanov at gmail.com (Igor Bukanov)
Date: Thu, 24 Aug 2006 00:52:09 +0200
Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary
In-Reply-To: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com>
Message-ID: <df0b33100608231552ka6e6e99na18001b0eeb38c83@mail.gmail.com>

On 8/23/06, Phillip J. Eby wrote:
> Our original
> assumption was that if they could implement throw() then they could clearly
> implement close(), since close() was defined in terms of throw().  An
> asynchronous return might be another matter.

Such asynchronous return can always be implemented via a special
hidden exception that is invisible to catch statements. I.e an
implementation of generators  can still use ___GeneratorExit
internally as long as it is not exposed to scripts.

Regards, Igor

From igor.bukanov at gmail.com  Thu Aug 24 01:44:21 2006
From: igor.bukanov at gmail.com (Igor Bukanov)
Date: Thu, 24 Aug 2006 01:44:21 +0200
Subject: [Python-Dev] GeneratorExit is unintuitive and uneccessary
In-Reply-To: <ca471dc20608231610i276f2cf3mfb032b561fe94b3@mail.gmail.com>
References: <5.1.1.6.0.20060823102150.03a90638@sparrow.telecommunity.com>
	<df0b33100608231552ka6e6e99na18001b0eeb38c83@mail.gmail.com>
	<ca471dc20608231610i276f2cf3mfb032b561fe94b3@mail.gmail.com>
Message-ID: <df0b33100608231644h3842abbkb98f49a2bf11363b@mail.gmail.com>

Regarding yield in the finally problem:

The current implementation throws RuntimeException after the yield
transfer the control back to the close method. If that is changed to
reporting a bad behavior and rethrowing GeneratorExit or its hidden
analog as suggested at the point of yield inside the generator, that
would nicely unwind possible outer finally blocks and close the
generator in any case.

With that close-as-return proposal it does not even require to change
the interpreter as the close method can simply loop like in
generator.triggerRetrunAfterTheYield
while generatorYielding:
  reportBadYield
  generator.triggerRetrunAfterTheYield

Of cause, a bad generator can still subvert even this in the same way
as return can be canceled, but with such changes one has to put
significant non-trivial efforts in writing such code.

Regards, Igor

On 8/24/06, Guido van Rossum  wrote:
> IIUC this is how return currently works -- in some sense it's an
> exception, but it can't be caught, and it won't escape from the
> current frame. Ditto for break/continue.
>
> The generator extensions are still very young, and I'm not against
> changing them somewhat in 2.6 or py3k. Perhaps GeneratorExit can be
> replaced by a treatment similar to the treatment of return. But I'm
> not sure it helps with the original problem; you could still put a
> yield in a finally clause that intercepts a return, just as today you
> can write
>
> def f():
>   for i in range(10):
>     print i
>     try:
>       return
>     finally:
>       break
>   print 999
>
> where the finally clause nullifies the return. In a generator, a yield
> in the finally clause still needs to be dealt with somehow.
>
>
> On 8/23/06, Igor Bukanov <igor.bukanov at gmail.com> wrote:
> > On 8/23/06, Phillip J. Eby wrote:
> > > Our original
> > > assumption was that if they could implement throw() then they could clearly
> > > implement close(), since close() was defined in terms of throw().  An
> > > asynchronous return might be another matter.
> >
> > Such asynchronous return can always be implemented via a special
> > hidden exception that is invisible to catch statements. I.e an
> > implementation of generators  can still use ___GeneratorExit
> > internally as long as it is not exposed to scripts.
> >
> > Regards, Igor
> >
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

From anthony at interlink.com.au  Mon Aug 28 03:54:43 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon, 28 Aug 2006 11:54:43 +1000
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
Message-ID: <200608281154.45219.anthony@interlink.com.au>

On Sunday 27 August 2006 05:06, Jack Howarth wrote:
>        I discovered that gcc 4.2 exposes a flaw with
> signed integer overflows in python. This bug and the
> necessary fix has been discussed in detail on the gcc
> mailing list. I have filed a detailed bug report and
> the recommended patch proposed by the gcc developers.
> This problem should be addressed BEFORE python 2.5 is
> released. The bug report is...
>
> [ 1545668 ] gcc trunk (4.2) exposes a signed integer overflows
>
> in the python sourceforge bug tracker. Thanks in advance
> for attempting to fix this before Python 2.5 is released.
>              Jack

Regardless of whether we consider gcc's behaviour to be correct or not, I do 
agree we need a fix for this in 2.5 final. That should also be backported to 
release24-maint for the 2.4.4 release, and maybe release23-maint, as Barry 
recently started talking about cutting a 2.3.6. 

Can I nominate Tim, with his terrifying knowledge of C compiler esoterica, as 
the person to pick the best fix? 

Thanks,
Anthony

From chad at zetaweb.com  Mon Aug 28 04:10:34 2006
From: chad at zetaweb.com (Chad Whitacre)
Date: Sun, 27 Aug 2006 22:10:34 -0400
Subject: [Python-Dev] draft of patch guidelines
In-Reply-To: <bbaeab100608271717t11e15552n5fac44354678e263@mail.gmail.com>
References: <bbaeab100608241222id248e9bn173f2a9aa82b7398@mail.gmail.com>	
	<44EE681B.7010601@zetaweb.com>	
	<bbaeab100608242141q5403bad4v3fbe73fb6ba41bf0@mail.gmail.com>	
	<44F1E751.7080002@zetaweb.com>
	<bbaeab100608271717t11e15552n5fac44354678e263@mail.gmail.com>
Message-ID: <44F2509A.1030506@zetaweb.com>

Brett,

>> I think this is less accurate. Patches languish because of limited time
>> *and* because newbies don't have any social capital w/in the Python
>> community. New patch contributors are volunteers too, so they understand
>> that constraint. Their big problem is their outsider status, to which
>> the patch angels/5-for-1 system is an elegant solution. And that's what
>> should be emphasized in this FAQ.
> 
> But that makes us sound elitist.

To be honest I took away the opposite lesson. New patch contributors 
acutely feel their outsider status; this is to be expected in any social 
system. The fact that there is a clear, established path for newcomers 
to follow -- the 5-for-1 rule -- is precisely what makes the Python 
community non-elitist in this instance. Transparent documentation on 
this point would further erode any elitism.



chad

From tim.peters at gmail.com  Mon Aug 28 04:32:52 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 27 Aug 2006 22:32:52 -0400
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <200608281154.45219.anthony@interlink.com.au>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
	<200608281154.45219.anthony@interlink.com.au>
Message-ID: <1f7befae0608271932r19358b5fsb36372f3cdb2097e@mail.gmail.com>

[Anthony Baxter]
> Regardless of whether we consider gcc's behaviour to be correct or not,

It is correct, but more to the point it's, umm, /there/ ;-)

> I do agree we need a fix for this in 2.5 final. That should also be backported to
> release24-maint for the 2.4.4 release, and maybe release23-maint, as Barry
> recently started talking about cutting a 2.3.6.
>
> Can I nominate Tim, with his terrifying knowledge of C compiler esoterica, as
> the person to pick the best fix?

It's a bitch.  Changing to

	if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)

is the obvious fix, but violates our "no warnings" policy:  the MS
compiler warns about applying unary minus to an unsigned operand -- it
"looks insane" to /their/ compiler writers ;-).  Elegant patch below
-- LOL.

Would be nice if someone verified it worked on a box where it matters.
 Would also be nice if people checked to see whether their compiler(s)
warn about something else now.

IIndex: Objects/intobject.c
===================================================================
--- Objects/intobject.c (revision 51618)
+++ Objects/intobject.c (working copy)
@@ -564,8 +564,14 @@
                                "integer division or modulo by zero");
                return DIVMOD_ERROR;
        }
-       /* (-sys.maxint-1)/-1 is the only overflow case. */
-       if (y == -1 && x < 0 && x == -x)
+       /* (-sys.maxint-1)/-1 is the only overflow case.  x is the most
+        * negative long iff x < 0 and, on a 2's-complement box, x == -x.
+        * However, -x is undefined (by C) if x /is/ the most negative long
+        * (it's a signed overflow case), and some compilers care.  So we cast
+        * x to unsigned long first.  However, then other compilers warn about
+        * applying unary minus to an unsigned operand.  Hence the weird "0-".
+        */
+       if (y == -1 && x < 0 && (unsigned long)x == 0-(unsigned long)x)
                return DIVMOD_OVERFLOW;
        xdivy = x / y;
        xmody = x - xdivy * y;

From p.f.moore at gmail.com  Mon Aug 28 12:18:38 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 28 Aug 2006 11:18:38 +0100
Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bit late,
	I know...)
Message-ID: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com>

One little addition to the elementtree docs. In the overview section,
adding a paragraph explaining best practice for importing the module
might be useful.

Some suggested text, for the "overview" section:

"""
The ElementTree module comes in two forms - a pure-python version
(xml.etree.ElementTree) and a C-coded implementation
(xml.etree.cElementTree) which is faster. To import the faster code if
possible, but fall back to the Python implementation, you can use

try:
    from xml.etree import cElementTree as ET
except ImportError:
    from xml.etree import ElementTree as ET

ElementTree is also available as an external module for older Python
versions. For portability to these versions, this pattern can be
extended to

try:
    from xml.etree import cElementTree as ET
except ImportError:
    try:
        from xml.etree import ElementTree as ET
    except ImportError:
        try:
            import cElementTree as ET
        except ImportError:
            import ElementTree as ET

"""

I'd put a patch on SF, but guess what? It's down again :-(

Paul.

PS This actually begs the question - are there platforms where
xml.etree.cElementTree is not available? If not, is there a need for
both versions? If there are, the wording above should probably be
modified to reflect this.

From fredrik at pythonware.com  Mon Aug 28 13:03:43 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 28 Aug 2006 13:03:43 +0200
Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bit
	late, I know...)
References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com>
Message-ID: <ecuii7$khs$1@sea.gmane.org>

Paul Moore wrote:

> One little addition to the elementtree docs. In the overview section,
> adding a paragraph explaining best practice for importing the module
> might be useful.

good idea.

> PS This actually begs the question - are there platforms where
> xml.etree.cElementTree is not available?

not really; I usually recommend checking for

    xml.etree.cElementTree
    cElementTree
    elementtree.ElementTree

in that order, if you just want to get the "best" implementation.

(there are some subtle differences between the C implementations
and the Python implementations, so people who do non-standard
stuff may want to use xml.etree.ElementTree, but I don't think that
has to be mentioned in the overview)

</F> 




From rowen at cesmail.net  Tue Aug 29 00:24:19 2006
From: rowen at cesmail.net (Russell E. Owen)
Date: Mon, 28 Aug 2006 15:24:19 -0700
Subject: [Python-Dev] distutils patch
Message-ID: <rowen-EA4D4C.15241928082006@sea.gmane.org>

A colleague stumbled across distutils bug/misfeature that he found had 
been reported with an associated patch:
<http://sourceforge.net/tracker/index.php?func=detail&aid=1183712&group_i
d=5470&atid=305470>

and I'm wondering if there's any plans to accept the patch or implement 
some other fix (such as raising an exception if one tries to use a blank 
package name). The current behavior is clearly unexpected and confusing. 
If there's something I can do to help get a fix of some kind into python 
2.5, I'm happy to do it.

Regards,

-- Russell


From martin at v.loewis.de  Tue Aug 29 01:01:16 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 29 Aug 2006 01:01:16 +0200
Subject: [Python-Dev] distutils patch
In-Reply-To: <rowen-EA4D4C.15241928082006@sea.gmane.org>
References: <rowen-EA4D4C.15241928082006@sea.gmane.org>
Message-ID: <44F375BC.20603@v.loewis.de>

Russell E. Owen schrieb:
> A colleague stumbled across distutils bug/misfeature that he found had 
> been reported with an associated patch:
> <http://sourceforge.net/tracker/index.php?func=detail&aid=1183712&group_i
> d=5470&atid=305470>
> 
> and I'm wondering if there's any plans to accept the patch or implement 
> some other fix (such as raising an exception if one tries to use a blank 
> package name). The current behavior is clearly unexpected and confusing. 
> If there's something I can do to help get a fix of some kind into python 
> 2.5, I'm happy to do it.

As you can see from my comment, I think it really is a bug in the
application. Without reviewing it again, I think disallowing empty
package names might be the right thing; this can only be done in 2.6,
though.

Regards,
Martin


From greg.ewing at canterbury.ac.nz  Tue Aug 29 04:05:22 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 29 Aug 2006 14:05:22 +1200
Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bit
 late, I know...)
In-Reply-To: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com>
References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com>
Message-ID: <44F3A0E2.60006@canterbury.ac.nz>

Paul Moore wrote:

> The ElementTree module comes in two forms - a pure-python version
> (xml.etree.ElementTree) and a C-coded implementation
> (xml.etree.cElementTree) which is faster.

If this is to be in the stdlib, is there any chance
of tidying up the convoluted, uninituitive and
non-pep8-compliant module naming structure?

--
Greg

From steve at holdenweb.com  Tue Aug 29 06:36:05 2006
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 29 Aug 2006 05:36:05 +0100
Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bit
	late, I know...)
In-Reply-To: <44F3A0E2.60006@canterbury.ac.nz>
References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com>
	<44F3A0E2.60006@canterbury.ac.nz>
Message-ID: <ed0g6k$m01$5@sea.gmane.org>

Greg Ewing wrote:
> Paul Moore wrote:
> 
> 
>>The ElementTree module comes in two forms - a pure-python version
>>(xml.etree.ElementTree) and a C-coded implementation
>>(xml.etree.cElementTree) which is faster.
> 
> 
> If this is to be in the stdlib, is there any chance
> of tidying up the convoluted, uninituitive and
> non-pep8-compliant module naming structure?
> 
I'm guessing the answer is "no" since the objection was only raised when 
the code had been incorporated into a release candidate. It was probably 
"no" from the beginning given that there was already substantial usage 
of the code before it was adopted for the stdlib.

Standards, apparently, are for *other* people :-)

It would be good if 3.0 was *much* more hard-nosed about naming 
conventions. How can we expect the community as a whole to take them 
seriously if we don't take them seriously ourselves?

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd          http://www.holdenweb.com
Skype: holdenweb       http://holdenweb.blogspot.com
Recent Ramblings     http://del.icio.us/steve.holden


From steve at holdenweb.com  Tue Aug 29 06:38:36 2006
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 29 Aug 2006 05:38:36 +0100
Subject: [Python-Dev] draft for bug guidelines
In-Reply-To: <bbaeab100608241349s1e8c3c12see196234794e81d4@mail.gmail.com>
References: <bbaeab100608241221i426b135bkf7345c9701f0af81@mail.gmail.com>	<20060824192945.GA32475@phd.pp.ru>
	<bbaeab100608241349s1e8c3c12see196234794e81d4@mail.gmail.com>
Message-ID: <ed0gbb$m01$6@sea.gmane.org>

Brett Cannon wrote:
> Made it the first step.  =)
> 
> -Brett
> 
> On 8/24/06, *Oleg Broytmann* <phd at oper.phd.pp.ru 
> <mailto:phd at oper.phd.pp.ru>> wrote:
> 
>     On Thu, Aug 24, 2006 at 12:21:06PM -0700, Brett Cannon wrote:
>      > Start a new bug
> 
>        "Before starting a new bug please try to search if the bug has
>     already
>     been reported. It it has - do not start a new report, add your
>     comments to
>     the existing report."
> 
You probably want to include some advice about how. Isn't this rather an 
inappropriate time to be writing SF user notes, given that you are 
planning to provide us with an alternative to SF by the end of the year?

The last thing we need is a "how to submit a bug" article that sends 
people to the wrong repository ...

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd          http://www.holdenweb.com
Skype: holdenweb       http://holdenweb.blogspot.com
Recent Ramblings     http://del.icio.us/steve.holden


From fredrik at pythonware.com  Tue Aug 29 09:59:08 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 29 Aug 2006 09:59:08 +0200
Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bitlate,
	I know...)
References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com><44F3A0E2.60006@canterbury.ac.nz>
	<ed0g6k$m01$5@sea.gmane.org>
Message-ID: <ed0s4c$lnr$1@sea.gmane.org>

Steve Holden wrote:

> Standards, apparently, are for *other* people :-)

ET was written years before the "certain modules should use camelcase" stuff
was removed from PEP 8.  as a refresher for those of you who have trouble
remembering how things were back in early 2004, here's GvR's original style
guide:

    http://www.python.org/doc/essays/styleguide/

    Modules that export a single class (or a number of closely related
    classes, plus some additional support) are often named in MixedCase,
    with the module name being the same as the class name (e.g. the
    standard StringIO module). Modules that export a bunch of functions
    are usually named in all lowercase. /.../

    There is an emerging convention that when an extension module written
    in C or C++ has an accompanying Python module that provides a higher
    level (e.g. more object oriented) interface, the Python module's name
    CapWords, while the C/C++ module is named in all lowercase and has a
    leading underscore (e.g. Tkinter/_tkinter).

</F> 




From greg.ewing at canterbury.ac.nz  Tue Aug 29 11:19:10 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 29 Aug 2006 21:19:10 +1200
Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bitlate,
 I know...)
In-Reply-To: <ed0s4c$lnr$1@sea.gmane.org>
References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com>
	<44F3A0E2.60006@canterbury.ac.nz> <ed0g6k$m01$5@sea.gmane.org>
	<ed0s4c$lnr$1@sea.gmane.org>
Message-ID: <44F4068E.7030205@canterbury.ac.nz>

Fredrik Lundh wrote:

> ET was written years before the "certain modules should use camelcase" stuff
> was removed from PEP 8.

Maybe so, but I was hoping that additions to the
stdlib in this day and age might be adapted to follow
modern conventions instead of just being plonked in
as-is.

--
Greg

From ncoghlan at iinet.net.au  Tue Aug 29 11:29:28 2006
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue, 29 Aug 2006 19:29:28 +1000
Subject: [Python-Dev] Adding an rslice() builtin?
Message-ID: <44F408F8.1070403@iinet.net.au>

A discussion on the py3k list reminded me that translating a forward slice 
into a reversed slice is significantly less than obvious to many people. Not 
only do you have to negate the step value and swap the start and stop values, 
but you also need to subtract one from each of the step values, and ensure the 
new start value was actually in the original slice:

   reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step]

An rslice builtin would make the latter version significantly easier to read:

   seq[rslice(start, stop, step)]

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From steve at holdenweb.com  Tue Aug 29 11:37:54 2006
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 29 Aug 2006 10:37:54 +0100
Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bitlate,
	I know...)
In-Reply-To: <44F4068E.7030205@canterbury.ac.nz>
References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com>	<44F3A0E2.60006@canterbury.ac.nz>
	<ed0g6k$m01$5@sea.gmane.org>	<ed0s4c$lnr$1@sea.gmane.org>
	<44F4068E.7030205@canterbury.ac.nz>
Message-ID: <44F40AF2.30504@holdenweb.com>

Greg Ewing wrote:
> Fredrik Lundh wrote:
> 
> 
>>ET was written years before the "certain modules should use camelcase" stuff
>>was removed from PEP 8.
> 
> 
> Maybe so, but I was hoping that additions to the
> stdlib in this day and age might be adapted to follow
> modern conventions instead of just being plonked in
> as-is.
> 
Unfortunately this doesn't work if you want to encourage existing users 
to migrate to the built-in version.

However, let's hope that Python 3000 will have enforcers patrolling the 
perimeter of the library with shotguns. It does make us look rather dumb 
when we set rules that our own code doesn't obey - or change the rules 
after we've already encouraged contributions under other terms.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd          http://www.holdenweb.com
Skype: holdenweb       http://holdenweb.blogspot.com
Recent Ramblings     http://del.icio.us/steve.holden

From greg.ewing at canterbury.ac.nz  Tue Aug 29 11:37:47 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 29 Aug 2006 21:37:47 +1200
Subject: [Python-Dev] Adding an rslice() builtin?
In-Reply-To: <44F408F8.1070403@iinet.net.au>
References: <44F408F8.1070403@iinet.net.au>
Message-ID: <44F40AEB.7010405@canterbury.ac.nz>

Nick Coghlan wrote:

>    reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step]
> 
> An rslice builtin would make the latter version significantly easier to read:
> 
>    seq[rslice(start, stop, step)]

How would this deal with omitted start and/or stop values?

--
Greg

From steve at holdenweb.com  Tue Aug 29 11:37:54 2006
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 29 Aug 2006 10:37:54 +0100
Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bitlate,
	I know...)
In-Reply-To: <44F4068E.7030205@canterbury.ac.nz>
References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com>	<44F3A0E2.60006@canterbury.ac.nz>
	<ed0g6k$m01$5@sea.gmane.org>	<ed0s4c$lnr$1@sea.gmane.org>
	<44F4068E.7030205@canterbury.ac.nz>
Message-ID: <44F40AF2.30504@holdenweb.com>

Greg Ewing wrote:
> Fredrik Lundh wrote:
> 
> 
>>ET was written years before the "certain modules should use camelcase" stuff
>>was removed from PEP 8.
> 
> 
> Maybe so, but I was hoping that additions to the
> stdlib in this day and age might be adapted to follow
> modern conventions instead of just being plonked in
> as-is.
> 
Unfortunately this doesn't work if you want to encourage existing users 
to migrate to the built-in version.

However, let's hope that Python 3000 will have enforcers patrolling the 
perimeter of the library with shotguns. It does make us look rather dumb 
when we set rules that our own code doesn't obey - or change the rules 
after we've already encouraged contributions under other terms.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd          http://www.holdenweb.com
Skype: holdenweb       http://holdenweb.blogspot.com
Recent Ramblings     http://del.icio.us/steve.holden


From p.f.moore at gmail.com  Tue Aug 29 14:46:33 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 29 Aug 2006 13:46:33 +0100
Subject: [Python-Dev] Adding an rslice() builtin?
In-Reply-To: <44F40AEB.7010405@canterbury.ac.nz>
References: <44F408F8.1070403@iinet.net.au> <44F40AEB.7010405@canterbury.ac.nz>
Message-ID: <79990c6b0608290546h2542fd25raae256ebb3691b84@mail.gmail.com>

On 8/29/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
>
> >    reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step]
> >
> > An rslice builtin would make the latter version significantly easier to read:
> >
> >    seq[rslice(start, stop, step)]
>
> How would this deal with omitted start and/or stop values?

More generally, given start/stop/step don't have to be numbers, how
can this work in general?

I don't actually see a need for this, given that
reversed(seq[start:stop:step]) covers all of the real use cases I can
think of...

Paul.

From guido at python.org  Tue Aug 29 15:45:05 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Aug 2006 06:45:05 -0700
Subject: [Python-Dev] Adding an rslice() builtin?
In-Reply-To: <44F408F8.1070403@iinet.net.au>
References: <44F408F8.1070403@iinet.net.au>
Message-ID: <ca471dc20608290645v18ac0b3dg8df48c77a27c7a9c@mail.gmail.com>

That discussion on py3k is far from finished.

We've also had a similar discussion in the past and ended up with
reversed(), which solves the problem on the other end (using a forward
slice but iterating backwards).

Also, when I saw your subject I thought rslice() was related to
rfind(), so the name is at best questuinable.

Let's drop this until we've got clarity on what Py3k actually will do.

--Guido

On 8/29/06, Nick Coghlan <ncoghlan at iinet.net.au> wrote:
> A discussion on the py3k list reminded me that translating a forward slice
> into a reversed slice is significantly less than obvious to many people. Not
> only do you have to negate the step value and swap the start and stop values,
> but you also need to subtract one from each of the step values, and ensure the
> new start value was actually in the original slice:
>
>    reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step]
>
> An rslice builtin would make the latter version significantly easier to read:
>
>    seq[rslice(start, stop, step)]
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>              http://www.boredomandlaziness.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Tue Aug 29 16:45:03 2006
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 29 Aug 2006 07:45:03 -0700
Subject: [Python-Dev] Doc suggestion for Elementtree (for 2.5? a bitlate,
	I know...)
In-Reply-To: <44F4068E.7030205@canterbury.ac.nz>
References: <79990c6b0608280318s39c0ba41uda8377f3e8efc11@mail.gmail.com>
	<44F3A0E2.60006@canterbury.ac.nz> <ed0g6k$m01$5@sea.gmane.org>
	<ed0s4c$lnr$1@sea.gmane.org> <44F4068E.7030205@canterbury.ac.nz>
Message-ID: <20060829144503.GA26161@panix.com>

On Tue, Aug 29, 2006, Greg Ewing wrote:
> Fredrik Lundh wrote:
>> 
>> ET was written years before the "certain modules should use camelcase" stuff
>> was removed from PEP 8.
> 
> Maybe so, but I was hoping that additions to the stdlib in this day
> and age might be adapted to follow modern conventions instead of just
> being plonked in as-is.

You have a point, but I think that for external libraries with a large
following the best we can do is set things up so that it's both PEP8
compliant *and* has aliases to the existing setup.  From my POV, it's
critical to encourage people to switch to the stdlib version if possible
(but often writing code that works with the external library is the only
way to support multiple Python versions).

That parenthetical bit is the real killer, and I don't think even Py3K
can completely overcome it if PEP8 continues to evolve.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it."  --Brian W. Kernighan

From ncoghlan at gmail.com  Tue Aug 29 16:54:40 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Aug 2006 00:54:40 +1000
Subject: [Python-Dev] Adding an rslice() builtin?
In-Reply-To: <44F40AEB.7010405@canterbury.ac.nz>
References: <44F408F8.1070403@iinet.net.au> <44F40AEB.7010405@canterbury.ac.nz>
Message-ID: <44F45530.80108@gmail.com>

Greg Ewing wrote:
> Nick Coghlan wrote:
> 
>>    reversed(seq[start:stop:step]) becomes 
>> seq[(stop-1)%abs(step):start-1:-step]
>>
>> An rslice builtin would make the latter version significantly easier 
>> to read:
>>
>>    seq[rslice(start, stop, step)]
> 
> How would this deal with omitted start and/or stop values?

Badly! (negative indices are screwed, too)

This would be an awful lot easier if we could just subclass slice, and do the 
calculation in the indices() method where we have access to len(seq) in order 
to deal with an omitted stop value and negative indices.

Ah well, never mind. I'll take it back to the Py3k list, and see if I can find 
some small changes we can make so that slice() doesn't feel like a bolted-on 
hack that exists solely to avoid the need for a new special method :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From barry at python.org  Tue Aug 29 17:22:58 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 29 Aug 2006 11:22:58 -0400
Subject: [Python-Dev] Interest in a Python 2.3.6?
Message-ID: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org>

I am considering producing a Python 2.3.6 release, which would of  
course only be a bug fix maintenance release.  The primary reason is  
that not all OS distributions have upgraded to Python 2.4 and I think  
it's worthwhile for us to bless a release that fixes known critical  
bugs.  I'm willing to be the release manager for 2.3.6, but I'm  
hoping you all will be able to help identify the most important bugs  
that need fixing.

First, is there interest in getting a 2.3.6 release out?  I'd propose  
keeping things simple, by picking a date and releasing what we've got  
at that date (assuming of course all the unit tests pass).  It's  
probably a good idea to add a wiki page tracking the fixes we want to  
get in there.

2.5 final is slated for 12-Sep-2006 and I know that Anthony is  
planning for a 2.4.4 soon after that.  I'm thinking that we'd try to  
do a 2.3.6 a couple of weeks after 2.4.4 so that people who care  
about it aren't stacked up with fixing too many branches at once.  My  
first thought was to shoot for Monday October 9th.

What are the potential 2.3.6 fixes?  Nothing on this page:

http://www.python.org/download/releases/2.3.5/bugs/

seems critical to me.  I know that I've added some important email  
package fixes that are already in Subversion.  There are tons of bugs  
and patches reported against 2.3 in SF (lament: why won't SF just / 
tell/ me how many there are?), but I have no idea atm which, if any,  
are important enough to go into a 2.3.6.  I haven't yet done an svn  
diff to see what changes are already in there.

I don't have the cycles to fix things myself, so it would be up to  
everyone to help commit fixes.  I'll ride herd a bit if necessary.

Thoughts?  I don't want to waste my time if nobody thinks a 2.3.6  
would be useful, but I'm happy to do it if there's community  
support.  I'll also need the usual help with Windows installers and  
documentation updates.

Cheers,
-Barry

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060829/d2b3650e/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 304 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060829/d2b3650e/attachment.pgp 

From g.brandl at gmx.net  Tue Aug 29 17:29:08 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 29 Aug 2006 17:29:08 +0200
Subject: [Python-Dev] Interest in a Python 2.3.6?
In-Reply-To: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org>
References: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org>
Message-ID: <ed1mg4$jcg$1@sea.gmane.org>

Barry Warsaw wrote:
>   I am considering producing a Python 2.3.6 release, which would of 
> course only be a bug fix maintenance release.  The primary reason is 
> that not all OS distributions have upgraded to Python 2.4 and I think 
> it's worthwhile for us to bless a release that fixes known critical 
> bugs.  I'm willing to be the release manager for 2.3.6, but I'm hoping 
> you all will be able to help identify the most important bugs that need 
> fixing.
> 
> First, is there interest in getting a 2.3.6 release out?  I'd propose 
> keeping things simple, by picking a date and releasing what we've got at 
> that date (assuming of course all the unit tests pass).  It's probably a 
> good idea to add a wiki page tracking the fixes we want to get in there.
> 
> 2.5 final is slated for 12-Sep-2006 and I know that Anthony is planning 
> for a 2.4.4 soon after that.  I'm thinking that we'd try to do a 2.3.6 a 
> couple of weeks after 2.4.4 so that people who care about it aren't 
> stacked up with fixing too many branches at once.  My first thought was 
> to shoot for Monday October 9th.
> 
> What are the potential 2.3.6 fixes?  Nothing on this page:
> 
> http://www.python.org/download/releases/2.3.5/bugs/
> 
> seems critical to me.  I know that I've added some important email 
> package fixes that are already in Subversion.  There are tons of bugs 
> and patches reported against 2.3 in SF (lament: why won't SF just /tell/ 
> me how many there are?), but I have no idea atm which, if any, are 
> important enough to go into a 2.3.6.

Most of them are also in the 2.4 releases...

>  I haven't yet done an svn diff to 
> see what changes are already in there.

There's one problem: it was thought for a long time that there wouldn't be
any more 2.3 releases, so bug fixes were applied only in the head and 2.4
branch.

If there will be a 2.3.6, it might be necessary to look at all 2.4 branch
fixes and decide which of them are worth backporting.

Georg


From jcarlson at uci.edu  Tue Aug 29 18:13:07 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 29 Aug 2006 09:13:07 -0700
Subject: [Python-Dev] Interest in a Python 2.3.6?
In-Reply-To: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org>
References: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org>
Message-ID: <20060829084804.1B06.JCARLSON@uci.edu>


Barry Warsaw <barry at python.org> wrote:
> I am considering producing a Python 2.3.6 release, which would of  
> course only be a bug fix maintenance release.  The primary reason is  
> that not all OS distributions have upgraded to Python 2.4 and I think  
> it's worthwhile for us to bless a release that fixes known critical  
> bugs.  I'm willing to be the release manager for 2.3.6, but I'm  
> hoping you all will be able to help identify the most important bugs  
> that need fixing.
> 
> What are the potential 2.3.6 fixes?  Nothing on this page:

These are both issues in 2.3 and 2.4:
http://www.python.org/sf/780714
http://www.python.org/sf/1548687

Those are all I have off the top of my head.

 - Josiah


From david.nospam.hopwood at blueyonder.co.uk  Tue Aug 29 18:44:40 2006
From: david.nospam.hopwood at blueyonder.co.uk (David Hopwood)
Date: Tue, 29 Aug 2006 17:44:40 +0100
Subject: [Python-Dev] Adding an rslice() builtin?
In-Reply-To: <44F408F8.1070403@iinet.net.au>
References: <44F408F8.1070403@iinet.net.au>
Message-ID: <44F46EF8.9000909@blueyonder.co.uk>

Nick Coghlan wrote:
> A discussion on the py3k list reminded me that translating a forward slice 
> into a reversed slice is significantly less than obvious to many people. Not 
> only do you have to negate the step value and swap the start and stop values, 
> but you also need to subtract one from each of the step values, and ensure the 
> new start value was actually in the original slice:
> 
>    reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step]
> 
> An rslice builtin would make the latter version significantly easier to read:
> 
>    seq[rslice(start, stop, step)]

Or slice.reversed().

-- 
David Hopwood <david.nospam.hopwood at blueyonder.co.uk>



From exarkun at divmod.com  Tue Aug 29 19:09:57 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 29 Aug 2006 13:09:57 -0400
Subject: [Python-Dev] Adding an rslice() builtin?
In-Reply-To: <44F46EF8.9000909@blueyonder.co.uk>
Message-ID: <20060829170957.1717.385586324.divmod.quotient.33876@ohm>

On Tue, 29 Aug 2006 17:44:40 +0100, David Hopwood <david.nospam.hopwood at blueyonder.co.uk> wrote:
>Nick Coghlan wrote:
>> A discussion on the py3k list reminded me that translating a forward slice
>> into a reversed slice is significantly less than obvious to many people. Not
>> only do you have to negate the step value and swap the start and stop values,
>> but you also need to subtract one from each of the step values, and ensure the
>> new start value was actually in the original slice:
>>
>>    reversed(seq[start:stop:step]) becomes seq[(stop-1)%abs(step):start-1:-step]
>>
>> An rslice builtin would make the latter version significantly easier to read:
>>
>>    seq[rslice(start, stop, step)]
>
>Or slice.reversed().
>

Better, slice.reversed(length).

Jean-Paul

From brett at python.org  Tue Aug 29 19:15:51 2006
From: brett at python.org (Brett Cannon)
Date: Tue, 29 Aug 2006 10:15:51 -0700
Subject: [Python-Dev] draft for bug guidelines
In-Reply-To: <ed0gbb$m01$6@sea.gmane.org>
References: <bbaeab100608241221i426b135bkf7345c9701f0af81@mail.gmail.com>
	<20060824192945.GA32475@phd.pp.ru>
	<bbaeab100608241349s1e8c3c12see196234794e81d4@mail.gmail.com>
	<ed0gbb$m01$6@sea.gmane.org>
Message-ID: <bbaeab100608291015m1d2a07a7re51bb828ff553a98@mail.gmail.com>

On 8/28/06, Steve Holden <steve at holdenweb.com> wrote:
>
> Brett Cannon wrote:
> > Made it the first step.  =)
> >
> > -Brett
> >
> > On 8/24/06, *Oleg Broytmann* <phd at oper.phd.pp.ru
> > <mailto:phd at oper.phd.pp.ru>> wrote:
> >
> >     On Thu, Aug 24, 2006 at 12:21:06PM -0700, Brett Cannon wrote:
> >      > Start a new bug
> >
> >        "Before starting a new bug please try to search if the bug has
> >     already
> >     been reported. It it has - do not start a new report, add your
> >     comments to
> >     the existing report."
> >
> You probably want to include some advice about how. Isn't this rather an
> inappropriate time to be writing SF user notes, given that you are
> planning to provide us with an alternative to SF by the end of the year?


No because we have people reporting bugs between now and the end of the year
which is four months away.

The last thing we need is a "how to submit a bug" article that sends
> people to the wrong repository ...


But the whole point is for these things to end up on
www.python.org/dev/where I will update them as soon as we have a new
tracker selected and
before the new tracker is even used.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060829/5cc582fa/attachment.htm 

From arigo at tunes.org  Tue Aug 29 22:10:22 2006
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 29 Aug 2006 22:10:22 +0200
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <1f7befae0608261737p595b805fnee9d65762fe1af19@mail.gmail.com>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
	<ca471dc20608261401s97dea6w8c6287507062573f@mail.gmail.com>
	<44F0D67D.9000007@blueyonder.co.uk>
	<9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com>
	<1f7befae0608261737p595b805fnee9d65762fe1af19@mail.gmail.com>
Message-ID: <20060829201022.GA22579@code0.codespeak.net>

Hi Tim,

On Sat, Aug 26, 2006 at 08:37:46PM -0400, Tim Peters wrote:
> [Thomas Wouters]
> > Why not just "... && x == LONG_MIN"?

> it's better (when possible) not to tie the code to that `x` was
> specifically declared as type "long" (e.g., just more stuff that will
> break if Python decides to make its short int of type PY_LONG_LONG
> instead).

The proposed "correct fix" breaks this goal too:

> >> "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)".

                               ^^^^^^^^^^^^^        ^^^^^^^^^^^^^


A bientot,

Armin

From barry at python.org  Tue Aug 29 22:42:27 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 29 Aug 2006 16:42:27 -0400
Subject: [Python-Dev] Interest in a Python 2.3.6?
In-Reply-To: <ed1mg4$jcg$1@sea.gmane.org>
References: <05D21426-7FF5-4AEF-B757-DF0BEC5D0D74@python.org>
	<ed1mg4$jcg$1@sea.gmane.org>
Message-ID: <4212B8D1-3152-4245-94C5-852EBFB820B3@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Aug 29, 2006, at 11:29 AM, Georg Brandl wrote:

> There's one problem: it was thought for a long time that there  
> wouldn't be
> any more 2.3 releases, so bug fixes were applied only in the head  
> and 2.4
> branch.
>
> If there will be a 2.3.6, it might be necessary to look at all 2.4  
> branch
> fixes and decide which of them are worth backporting.

Right, we'll definitely have to do that, and maybe not much more.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRPSmuXEjvBPtnXfVAQKyegQAgpzLpPddKE03wg3orKvlHPJXFKpbaLvd
4gIRIgvqwGjKwjxY4s8Gn55axHJM8YBwJ+IbCzYg3nmBB3JvljuKNxTZtz4w781Y
BS/apc8Ng+2wv/W8Eo2zVbbRformZa24H/d8Gq5Mdst1yTrcAwwcO7k1hHjAl9w8
HFIWjechAVg=
=4wXb
-----END PGP SIGNATURE-----

From laca at sun.com  Tue Aug 29 23:04:25 2006
From: laca at sun.com (Laszlo (Laca) Peter)
Date: Tue, 29 Aug 2006 17:04:25 -0400
Subject: [Python-Dev] 32-bit and 64-bit python on Solaris
Message-ID: <1156885465.1622.42.camel@ferrari.sfbay.sun.com>

Hi,

I work in the team that delivers python on Solaris.  Recently we've
been getting requests for delivering python in 64-bit as well
as in 32-bit.  As you probably know, Solaris can run 64-bit and
32-bit binaries on the same system, but of course you can't mix and
match shared objects with different ISAs.  This seems to apply to
python bytecode as well: the pyc files generated by a 64-bit build
of python are incompatible with those generated by the 32-bit python.
Note the caveat at
http://aspn.activestate.com/ASPN/docs/ActivePython/2.3/python/lib/module-marshal.html

I guess my first question is if there are any plans to make the
bytecodes for different ISAs compatible.  That would make most of
our problems magically go away (;

If not, then the issue we have is how to separate the 32-bit and
64-bit pyc's on the same system so that:
 - loading the right pyc is transparent to the user
 - you can't accidentally load a pyc with the wrong ISA
   (by setting PYTHONPATH, for example)
So duplicating the libdir is not an option.

The usual technique used for Solaris shared objects is placing the
64-bit objects in a 64/ subdirectory, while the 32-bit objects stay
where they are.
We considered various solutions.  Our current proposal is this:

 .../some/dir/foo.py          <- python source
 .../some/dir/foo.pyc         <- 32-bit bytecode
 .../some/dir/foo.pyc64       <- 64-bit bytecode
 .../some/dir/foo.pyo         <- optimised 32-bit bytecode
 .../some/dir/foo.pyo64       <- optimised 64-bit bytecode

The 64-bit python would always look for pyc64/pyo64, so this would
take care of accidentally loading the wrong ISA.  PYTHONPATH could
be the same for the 32-bit and the 64-bit python.

Shared objects:

 .../some/dir/foo.so       <- 32-bit shared object
 .../some/dir/64/foo.so    <- 64-bit shared object

This is kinda inconsistent, but simple enough.  The 64-bit python
would automatically look for shared objects in the 64/ subdir.
ld.so.1 makes sure that the wrong share object is never loaded.

So I'm looking for advice and wondering if it would be possible to
support multiple ISAs out of the box.  Would you be willing to accept
patches that implement the above (or whatever solution we agree to)
on Solaris?

Thanks,
Laca



From guido at python.org  Tue Aug 29 23:42:43 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 29 Aug 2006 14:42:43 -0700
Subject: [Python-Dev] Small Py3k task: fix modulefinder.py
Message-ID: <ca471dc20608291442p3d92790ema7aa35f85d38156a@mail.gmail.com>

Is anyone familiar enough with modulefinder.py to fix its breakage in
Py3k? It chokes in a nasty way (exceeding the recursion limit) on the
relative import syntax. I suspect this is also a problem for 2.5, when
people use that syntax; hence the cross-post. There's no unittest for
modulefinder.py, but I believe py2exe depends on it (and of course
freeze.py, but who uses that still?)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.peters at gmail.com  Wed Aug 30 00:57:12 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 29 Aug 2006 18:57:12 -0400
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <20060829201022.GA22579@code0.codespeak.net>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
	<ca471dc20608261401s97dea6w8c6287507062573f@mail.gmail.com>
	<44F0D67D.9000007@blueyonder.co.uk>
	<9e804ac0608261654y39cd2613i57b65a021c375263@mail.gmail.com>
	<1f7befae0608261737p595b805fnee9d65762fe1af19@mail.gmail.com>
	<20060829201022.GA22579@code0.codespeak.net>
Message-ID: <1f7befae0608291557l5b04a8f6wd1371e62a5c9c69c@mail.gmail.com>

[Thomas Wouters]
>>> Why not just "... && x == LONG_MIN"?

[Tim Peters]
>> it's better (when possible) not to tie the code to that `x` was
>> specifically declared as type "long" (e.g., just more stuff that will
>> break if Python decides to make its short int of type PY_LONG_LONG
>> instead).

[Armin Rigo]
> The proposed "correct fix" breaks this goal too:
>
> > >> "if (y == -1 && x < 0 && (unsigned long)x == -(unsigned long)x)".
>
>                                ^^^^^^^^^^^^^        ^^^^^^^^^^^^^

Yup, although as noted before the proposed fix actually writes

    == 0-(unsigned long)x

at the tail end instead (to avoid compiler warnings at least under MS C).

It doesn't run afoul of the other criterion you snipped from the start
of the quoted paragraph:

    In practice we try to avoid numeric symbols from platform header files
    because so many platforms have screwed these up over the centuries
    (search for LONG_BIT or HUGE_VAL ;-)), ....

This is a wrong time in the release process to take on chance on
discovering a flaky LONG_MIN on some box, so I want to keep the code
as much as possible like what's already there (which worked fine for >
10 years on all known boxes) for now.

Speaking of which, I saw no feedback on the proposed patch in

    http://mail.python.org/pipermail/python-dev/2006-August/068502.html

so I'll just check that in tomorrow.

From tim.peters at gmail.com  Wed Aug 30 01:07:33 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 29 Aug 2006 19:07:33 -0400
Subject: [Python-Dev] 32-bit and 64-bit python on Solaris
In-Reply-To: <1156885465.1622.42.camel@ferrari.sfbay.sun.com>
References: <1156885465.1622.42.camel@ferrari.sfbay.sun.com>
Message-ID: <1f7befae0608291607p513f0201wda2465a5f4a73310@mail.gmail.com>

[Laszlo (Laca) Peter]
> I work in the team that delivers python on Solaris.  Recently we've
> been getting requests for delivering python in 64-bit as well
> as in 32-bit.  As you probably know, Solaris can run 64-bit and
> 32-bit binaries on the same system, but of course you can't mix and
> match shared objects with different ISAs.  This seems to apply to
> python bytecode as well: the pyc files generated by a 64-bit build
> of python are incompatible with those generated by the 32-bit python.
> Note the caveat at
> http://aspn.activestate.com/ASPN/docs/ActivePython/2.3/python/lib/module-marshal.html

Which caveat, specifically?  As it says there, the only known problem
was fixed in Python 2.2:

    This behavior is new in Python 2.2. In earlier versions, all but the least-
    significant 32 bits of the value were lost, and a warning message
was printed

> I guess my first question is if there are any plans to make the
> bytecodes for different ISAs compatible.  That would make most of
> our problems magically go away (;

I suspect they already have ;-)  There are no plans to make marshal
store a Python long object on a 64-bit box for integers that fit in 64
points but not in 32 bits, and there would be very little point to
doing so.  As the referenced page says, you get the same numeric value
regardless.  It's /possible/ to write Python code to detect the
difference in type, but real code wouldn't do that.

> ...

From rhettinger at ewtllc.com  Wed Aug 30 02:20:34 2006
From: rhettinger at ewtllc.com (Raymond Hettinger)
Date: Tue, 29 Aug 2006 17:20:34 -0700
Subject: [Python-Dev] Py2.5 issue:  decimal context manager misimplemented,
 misdesigned, and misdocumented
Message-ID: <44F4D9D2.2040804@ewtllc.com>

I would like to see the changes to the decimal module reverted for the 
Py2.5 release.

Currently, the code in the decimal module implements the context manager 
as a separate class instead of incorporating it directly in 
decimal.Context.  This makes the API unnecessarily complex and is not 
pretty compared to the code it was intended to replace.

Worse still, the implementation saves a reference to the context instead 
of making a copy of it.  Remember decimal.Context objects are mutable -- 
the current implementation does not fulfill its contract to restore the 
context to its original state at the conclusion of the with-statement.

The right way to do it was presented in PEP343.  The implementation was 
correct and the API was simple.

Additionally:

* The examples in WhatsNew don't work because the implementation uses a 
different method name to fetch to context (this is a shallow error 
except that the name in WhatsNew is better and we don't really want to 
have a new method for this).  It doesn't bode well that none of the 
release candidate end users noticed this discrepancy -- it means they 
are not trying out the examples.

* The implementation's doc string examples were not tested and don't 
work (this is a deep error).  One reads:

    with decimal.getcontext() as ctx:
        ctx.prec += 2
        s = ...
    return +s


To get this to work with the current implementation, it should read

    with decimal.getcontext().copy().get_manager() as ctx:
        ctx.prec += 2
        s = ...
    return +s

This is horrid.  Please either revert the patch or fix it to match PEP-343.


Raymond


From pje at telecommunity.com  Wed Aug 30 02:40:00 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 29 Aug 2006 20:40:00 -0400
Subject: [Python-Dev] Py2.5 issue: decimal context manager
 misimplemented, misdesigned, and misdocumented
In-Reply-To: <44F4D9D2.2040804@ewtllc.com>
Message-ID: <5.1.1.6.0.20060829203433.00a04748@sparrow.telecommunity.com>

At 05:20 PM 8/29/2006 -0700, Raymond Hettinger wrote:
>* The implementation's doc string examples were not tested and don't
>work (this is a deep error).  One reads:
>
>     with decimal.getcontext() as ctx:
>         ctx.prec += 2
>         s = ...
>     return +s
>
>
>To get this to work with the current implementation, it should read
>
>     with decimal.getcontext().copy().get_manager() as ctx:
>         ctx.prec += 2
>         s = ...
>     return +s
>
>This is horrid.  Please either revert the patch or fix it to match PEP-343.

Actually, as I read the code, that would be:

     with decimal.getcontext().get_manager() as ctx:

Which is still horrible, but unfortunately Guido has already pronounced 
that __context__ must be removed from PEP 343, which is what caused this 
abomination to come about.

The PEP currently offers the idea of a 'localcontext()' API that provides a 
nicer spelling, but it appears nobody implemented it.


From laca at sun.com  Wed Aug 30 02:42:23 2006
From: laca at sun.com (Laszlo (Laca) Peter)
Date: Tue, 29 Aug 2006 20:42:23 -0400
Subject: [Python-Dev] 32-bit and 64-bit python on Solaris
In-Reply-To: <1f7befae0608291607p513f0201wda2465a5f4a73310@mail.gmail.com>
References: <1156885465.1622.42.camel@ferrari.sfbay.sun.com>
	<1f7befae0608291607p513f0201wda2465a5f4a73310@mail.gmail.com>
Message-ID: <1156898543.1622.52.camel@ferrari.sfbay.sun.com>

On Tue, 2006-08-29 at 19:07 -0400, Tim Peters wrote:
> > I work in the team that delivers python on Solaris.  Recently we've
> > been getting requests for delivering python in 64-bit as well
> > as in 32-bit.  As you probably know, Solaris can run 64-bit and
> > 32-bit binaries on the same system, but of course you can't mix and
> > match shared objects with different ISAs.  This seems to apply to
> > python bytecode as well: the pyc files generated by a 64-bit build
> > of python are incompatible with those generated by the 32-bit
> python.
> > Note the caveat at
> >
> http://aspn.activestate.com/ASPN/docs/ActivePython/2.3/python/lib/module-marshal.html
> 
> Which caveat, specifically?  As it says there, the only known problem
> was fixed in Python 2.2:
> 
>     This behavior is new in Python 2.2. In earlier versions, all but
> the least-
>     significant 32 bits of the value were lost, and a warning message
> was printed

Hmm... I must be going blind reading that wrong.
So it's expected to Just Work and if not, that's a bug.
That's cool, thanks.

Now all that is left is taking care of the 64-bit shared objects.

Thanks,
Laca



From rhettinger at ewtllc.com  Wed Aug 30 02:51:13 2006
From: rhettinger at ewtllc.com (Raymond Hettinger)
Date: Tue, 29 Aug 2006 17:51:13 -0700
Subject: [Python-Dev] Py2.5 issue: decimal context manager
 misimplemented, misdesigned, and misdocumented
In-Reply-To: <44F4D9D2.2040804@ewtllc.com>
References: <44F4D9D2.2040804@ewtllc.com>
Message-ID: <44F4E101.6080309@ewtllc.com>

Raymond Hettinger wrote:

>I would like to see the changes to the decimal module reverted for the 
>Py2.5 release.
>
>Currently, the code in the decimal module implements the context manager 
>as a separate class instead of incorporating it directly in 
>decimal.Context. 
>  
>

That should read "... as a separate class instantiated by a brand-new 
Context method instead of implementing it as a module level function as 
shown in section 9 of PEP343".

The API in PEP343 is much cleaner.  Also, it doesn't have the copy vs 
reference bug.



Raymond


From rhettinger at ewtllc.com  Wed Aug 30 02:57:12 2006
From: rhettinger at ewtllc.com (Raymond Hettinger)
Date: Tue, 29 Aug 2006 17:57:12 -0700
Subject: [Python-Dev] Py2.5 issue: decimal context manager
 misimplemented, misdesigned, and misdocumented
In-Reply-To: <5.1.1.6.0.20060829203433.00a04748@sparrow.telecommunity.com>
References: <5.1.1.6.0.20060829203433.00a04748@sparrow.telecommunity.com>
Message-ID: <44F4E268.9000703@ewtllc.com>

Phillip J. Eby wrote:

> At 05:20 PM 8/29/2006 -0700, Raymond Hettinger wrote:
>
>> * The implementation's doc string examples were not tested and don't
>> work (this is a deep error).  One reads:
>>
>>     with decimal.getcontext() as ctx:
>>         ctx.prec += 2
>>         s = ...
>>     return +s
>>
>>
>> To get this to work with the current implementation, it should read
>>
>>     with decimal.getcontext().copy().get_manager() as ctx:
>>         ctx.prec += 2
>>         s = ...
>>     return +s
>>
>> This is horrid.  Please either revert the patch or fix it to match 
>> PEP-343.
>
>
> Actually, as I read the code, that would be:
>
>     with decimal.getcontext().get_manager() as ctx:

Given the current mis-implementation, the copy() step is absolutely 
necessary.  Since context objects are mutable, the current context would 
never get it precision and flags restored.

Try running the example and printing out the current context precision 
before and after the with-suite.  You'll see that the context has 
changed (which defeats the whole purpose).

>
> Which is still horrible, but unfortunately Guido has already 
> pronounced that __context__ must be removed from PEP 343, which is 
> what caused this abomination to come about.
>
> The PEP currently offers the idea of a 'localcontext()' API that 
> provides a nicer spelling, but it appears nobody implemented it.
>

Right.  The PEP version was correct and desirable.  But at this point, 
it is better to have nothing at all.  Users can still write their own (a 
la example 8 in the PEP).

I do not want the decimal API to be forever mucked-up; hence, we should 
revert the change.  Since it is buggy, that makes the decision an easy 
one to swallow.


Raymond


From anthony at interlink.com.au  Wed Aug 30 04:41:41 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 30 Aug 2006 12:41:41 +1000
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <1f7befae0608291557l5b04a8f6wd1371e62a5c9c69c@mail.gmail.com>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
	<20060829201022.GA22579@code0.codespeak.net>
	<1f7befae0608291557l5b04a8f6wd1371e62a5c9c69c@mail.gmail.com>
Message-ID: <200608301241.46420.anthony@interlink.com.au>

On Wednesday 30 August 2006 08:57, Tim Peters wrote:
> Speaking of which, I saw no feedback on the proposed patch in
>
>     http://mail.python.org/pipermail/python-dev/2006-August/068502.html
>
> so I'll just check that in tomorrow.

Fine with me!

thanks,
Anthony

-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From anthony at interlink.com.au  Wed Aug 30 04:42:26 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 30 Aug 2006 12:42:26 +1000
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
In-Reply-To: <1f7befae0608291557l5b04a8f6wd1371e62a5c9c69c@mail.gmail.com>
References: <20060826190600.0E75911002B@bromo.msbb.uc.edu>
	<20060829201022.GA22579@code0.codespeak.net>
	<1f7befae0608291557l5b04a8f6wd1371e62a5c9c69c@mail.gmail.com>
Message-ID: <200608301242.28648.anthony@interlink.com.au>

On Wednesday 30 August 2006 08:57, Tim Peters wrote:
> Speaking of which, I saw no feedback on the proposed patch in
>
>     http://mail.python.org/pipermail/python-dev/2006-August/068502.html
>
> so I'll just check that in tomorrow.

This should also be backported to release24-maint and release23-maint. Let me 
know if you can't do the backport...


-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From pje at telecommunity.com  Wed Aug 30 05:59:00 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 29 Aug 2006 23:59:00 -0400
Subject: [Python-Dev] Py2.5 issue: decimal context manager
 misimplemented, misdesigned, and misdocumented
In-Reply-To: <44F4E268.9000703@ewtllc.com>
References: <5.1.1.6.0.20060829203433.00a04748@sparrow.telecommunity.com>
	<5.1.1.6.0.20060829203433.00a04748@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20060829235607.04aaa878@sparrow.telecommunity.com>

At 05:57 PM 8/29/2006 -0700, Raymond Hettinger wrote:
>Phillip J. Eby wrote:
>
>>At 05:20 PM 8/29/2006 -0700, Raymond Hettinger wrote:
>>
>>>* The implementation's doc string examples were not tested and don't
>>>work (this is a deep error).  One reads:
>>>
>>>     with decimal.getcontext() as ctx:
>>>         ctx.prec += 2
>>>         s = ...
>>>     return +s
>>>
>>>
>>>To get this to work with the current implementation, it should read
>>>
>>>     with decimal.getcontext().copy().get_manager() as ctx:
>>>         ctx.prec += 2
>>>         s = ...
>>>     return +s
>>>
>>>This is horrid.  Please either revert the patch or fix it to match PEP-343.
>>
>>
>>Actually, as I read the code, that would be:
>>
>>     with decimal.getcontext().get_manager() as ctx:
>
>Given the current mis-implementation, the copy() step is absolutely 
>necessary.  Since context objects are mutable, the current context would 
>never get it precision and flags restored.
>
>Try running the example and printing out the current context precision 
>before and after the with-suite.  You'll see that the context has changed 
>(which defeats the whole purpose).

No need; now that you've explained the problem I see why the code is 
wrong.  This is definitely a bug in the decimal module.  It looks like the 
code is correct at first glance, but the .copy() definitely needs to be in 
the ContextManager class, not the get_manager() method.  Yuck.


From nmm1 at cus.cam.ac.uk  Wed Aug 30 12:23:26 2006
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Wed, 30 Aug 2006 11:23:26 +0100
Subject: [Python-Dev] gcc 4.2 exposes signed integer overflows
Message-ID: <E1GINEE-0007Lw-9C@draco.cus.cam.ac.uk>

"Tim Peters" <tim.peters at gmail.com> wrote:
>
> This is a wrong time in the release process to take on chance on
> discovering a flaky LONG_MIN on some box, so I want to keep the code
> as much as possible like what's already there (which worked fine for >
> 10 years on all known boxes) for now.

No, it didn't.  I reported a bug a couple of years back.

A blanket rule not to use symbols is clearly wrong, but there are
good reasons not to want to rely on LONG_MIN (or INT_MIN for that
matter).  Because of some incredibly complex issues (which I only
know some of), it hasn't been consistently -(1+LONG_MAX) on twos'
complement machines.  There are good reasons for making it -LONG_MAX,
but they aren't the ones that actually cause it to be so.

There are, however, very good reasons for using BOTH tests.  I.e.
if I have a C system which defines LONG_MIN to be -LONG_MAX because
it uses -(1+LONG_MAX) for an integer NaN indicator in some contexts,
you really DON'T want to create such a value.  I don't know if there
are any such C systems, but there have been other languages that did.

I hope that Guido wasn't saying that Python should EVER rely on
signed integer overflow wrapping in twos' complement.  Despite the
common usage, Java and all that, it is perhaps the worst systematic
architectural change to have happened in 30 years, and accounts for
a good 30% of all hard bugs in many classes of program.  Simple
buffer overflow is fairly easy to avoid by good programming style;
integer overflow causing trashing of unpredictable data isn't.

Any decent programming language (like Python!) regards integer
overflow as an error, and the moves to make C copy Java semantics
are yet another step away from software engineering in the direction
of who-gives-a-damn hacking.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From ncoghlan at gmail.com  Wed Aug 30 13:11:52 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 30 Aug 2006 21:11:52 +1000
Subject: [Python-Dev] Py2.5 issue: decimal context manager
 misimplemented, misdesigned, and misdocumented
In-Reply-To: <44F4D9D2.2040804@ewtllc.com>
References: <44F4D9D2.2040804@ewtllc.com>
Message-ID: <44F57278.5090408@gmail.com>

Raymond Hettinger wrote:
> I would like to see the changes to the decimal module reverted for the 
> Py2.5 release.

I believe you may be overreacting - I don't consider the current behaviour 
buggy and the module level API can be added later.

That said, the docstring is definitely wrong, and I can't find any unit tests 
for the feature (I thought there were some in test_with, but I appear to be 
mistaken).

> Currently, the code in the decimal module implements the context manager 
> as a separate class instead of incorporating it directly in 
> decimal.Context.  This makes the API unnecessarily complex and is not 
> pretty compared to the code it was intended to replace.

The removal of the __context__ method made it impossible to permit context 
objects to be used directly in with statements. Even when that was the case, 
the separate ContextManager class was necessary in order to correctly handle 
the restoration as context objects may be shared between threads or nested 
within a single thread [1]. The localcontext() function in PEP 343 does 
exactly the same thing - it merely uses a generator context instead of a 
direct implementation of __enter__ and __exit__.

The current syntax (the get_manager() method) can easily be made prettier in 
the future by adding a sugar function at the module level:

   def localcontext(ctx=None):
       if ctx is None: ctx = getcontext()
       return ctx.get_manager()

> Worse still, the implementation saves a reference to the context instead 
> of making a copy of it.  Remember decimal.Context objects are mutable -- 
> the current implementation does not fulfill its contract to restore the 
> context to its original state at the conclusion of the with-statement.

The implementation doesn't forget that. The context to restore is determined 
by calling getcontext() in the __enter__ method. The restored context has 
nothing to do with the context passed to the ContextManager constructor.

 >>> from decimal import getcontext()
 >>> getcontext().prec
28
 >>> with getcontext().get_manager() as ctx:
...   ctx.prec += 2
...
 >>> getcontext().prec
28

The only ways to break it are to call ContextManager directly with an existing 
context that someone else already has a reference to:

 >>> from decimal import getcontext()
 >>> getcontext().prec
28
 >>> with ContextManager(getcontext()) as ctx:
...   ctx.prec += 2
...
 >>> getcontext().prec
30

Or to deliberately reuse a ContextManager instance:

 >>> mgr = getcontext().get_manager()
 >>> with mgr as ctx:
...    ctx.prec += 2
...    print ctx.prec
...
32
 >>> with mgr as ctx:
...    ctx.prec += 2
...    print ctx.prec
...
34

Cheers,
Nick.

[1] In fact, get_manager() is merely a new name for the old __context__ 
method. This name was suggested by Neal after I initially called the method 
context_manager(), but was never separately discussed on python-dev (the 
original discussion occurred in one of the massive PEP 343 threads).

http://mail.python.org/pipermail/python-checkins/2006-May/052083.html


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From rhettinger at ewtllc.com  Wed Aug 30 14:35:51 2006
From: rhettinger at ewtllc.com (Raymond Hettinger)
Date: Wed, 30 Aug 2006 05:35:51 -0700
Subject: [Python-Dev] Py2.5 issue: decimal context manager
 misimplemented, misdesigned, and misdocumented
In-Reply-To: <44F57278.5090408@gmail.com>
References: <44F4D9D2.2040804@ewtllc.com> <44F57278.5090408@gmail.com>
Message-ID: <44F58627.5040504@ewtllc.com>

Nick Coghlan wrote:

> Raymond Hettinger wrote:
>
>> I would like to see the changes to the decimal module reverted for 
>> the Py2.5 release.
>
>
> I believe you may be overreacting - I don't consider the current 
> behaviour buggy and the module level API can be added later.


My preference is to remove the method entirely and then implement the 
approach in PEP 343 by adding a module level "localcontext()" function 
in Py2.6.

If you keep the method, then at least:
* fix the docstring (and make it doctestable)
* rename the method to .localcontext()
* move the .copy() step to inside the contextmanager instead of its caller
* update the WhatsNew example to match
* add a unittest




From rhettinger at ewtllc.com  Wed Aug 30 14:58:51 2006
From: rhettinger at ewtllc.com (Raymond Hettinger)
Date: Wed, 30 Aug 2006 05:58:51 -0700
Subject: [Python-Dev] Py2.5 issue: decimal context manager
 misimplemented, misdesigned, and misdocumented
In-Reply-To: <44F58627.5040504@ewtllc.com>
References: <44F4D9D2.2040804@ewtllc.com> <44F57278.5090408@gmail.com>
	<44F58627.5040504@ewtllc.com>
Message-ID: <44F58B8B.5060301@ewtllc.com>

An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060830/9476b59b/attachment.htm 

From ncoghlan at gmail.com  Thu Aug 31 12:08:36 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 31 Aug 2006 20:08:36 +1000
Subject: [Python-Dev] Py2.5 issue: decimal context manager
 misimplemented, misdesigned, and misdocumented
In-Reply-To: <44F4D9D2.2040804@ewtllc.com>
References: <44F4D9D2.2040804@ewtllc.com>
Message-ID: <44F6B524.6060504@gmail.com>

Raymond Hettinger wrote:
> The right way to do it was presented in PEP343.  The implementation was 
> correct and the API was simple.

Raymond's persuaded me that he's right on the API part at the very least. The 
current API was a mechanical replacement of the initial __context__ based API 
with a normal method, whereas I should have reverted back to the module-level 
localcontext() function from PEP343 and thrown the method on Context objects 
away entirely.

I can fix it on the trunk (and add those missing tests!), but I'll need 
Anthony and/or Neal's permission to backport it and remove the get_manager() 
method from Python 2.5 before we get stuck with it forever.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mugunth at gmail.com  Thu Aug 31 12:36:54 2006
From: mugunth at gmail.com (Muguntharaj Subramanian)
Date: Thu, 31 Aug 2006 16:06:54 +0530
Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8
Message-ID: <8dd9fd0608310336q45d2d3d3re203e871c7b384b8@mail.gmail.com>

Hi All,
I tried building python 2.5c1 using VC8.

Getting the following errors while building pythoncore_pgo:

   Creating library pythoncore_pgo/python25.lib and object
pythoncore_pgo/python25.exp
config.obj : error LNK2001: unresolved external symbol _init_types
.\pythoncore_pgo/python25.dll : fatal error LNK1120: 1 unresolved externals

I checked pythoncore_pgo.vcproj file, it alreadu contains:

RelativePath="..\Modules\_typesmodule.c"

Can anyone give me some clue to solve this issue.


Thanks & Regards,
Mugunth

http://mugunth.blogspot.com
http://thamizha.com
http://tamilblogs.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060831/1b491f77/attachment.htm 

From fredrik at pythonware.com  Thu Aug 31 12:55:12 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 31 Aug 2006 12:55:12 +0200
Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8
In-Reply-To: <8dd9fd0608310336q45d2d3d3re203e871c7b384b8@mail.gmail.com>
References: <8dd9fd0608310336q45d2d3d3re203e871c7b384b8@mail.gmail.com>
Message-ID: <ed6f6h$la4$1@sea.gmane.org>

Muguntharaj Subramanian wrote:
> Hi All,
> I tried building python 2.5c1 using VC8.
> 
> Getting the following errors while building pythoncore_pgo:
> 
>    Creating library pythoncore_pgo/python25.lib and object 
> pythoncore_pgo/python25.exp
> config.obj : error LNK2001: unresolved external symbol _init_types
> .\pythoncore_pgo/python25.dll : fatal error LNK1120: 1 unresolved externals
> 
> I checked pythoncore_pgo.vcproj file, it alreadu contains:
> 
> RelativePath="..\Modules\_typesmodule.c"
> 
> Can anyone give me some clue to solve this issue.

googling for error messages is always a good idea:

http://thread.gmane.org/gmane.comp.python.devel/83341

(see the last post in that thread for what I believe is a workaround)

</F>


From mugunth at gmail.com  Thu Aug 31 13:46:23 2006
From: mugunth at gmail.com (Muguntharaj Subramanian)
Date: Thu, 31 Aug 2006 17:16:23 +0530
Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8
In-Reply-To: <ed6f6h$la4$1@sea.gmane.org>
References: <8dd9fd0608310336q45d2d3d3re203e871c7b384b8@mail.gmail.com>
	<ed6f6h$la4$1@sea.gmane.org>
Message-ID: <8dd9fd0608310446o6008240x8bfa852b41595eab@mail.gmail.com>

>
>
> > I checked pythoncore_pgo.vcproj file, it alreadu contains:
> >
> > RelativePath="..\Modules\_typesmodule.c"
> >
> > Can anyone give me some clue to solve this issue.
>
> googling for error messages is always a good idea:
>
> http://thread.gmane.org/gmane.comp.python.devel/83341
>
> (see the last post in that thread for what I believe is a workaround)
>
> </F>
>


Yes, I have already done a bit of searching before posting that message.
I have seen that previous post.

That error mentioned in that post was in "pythoncore" module.
My error is while compiling "pythoncore_pgo" module.
And the workaround solution mentioned there is already there in the
pythoncore_pgo.vcproj file. Thats why i mentioned RelativePath="..\Modules\
>
> _typesmodule.c" is already there.



I am still getting the same error. I am not able to find the cause yet. Any
clue regarding this.

Thanks & Regards,
Mugunth



http://mugunth.blogspot.com
http://thamizha.com
http://tamilblogs.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060831/5c211830/attachment.html 

From kristjan at ccpgames.com  Thu Aug 31 13:46:46 2006
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_V=2E_J=F3nsson?=)
Date: Thu, 31 Aug 2006 11:46:46 -0000
Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8
Message-ID: <129CEF95A523704B9D46959C922A280002FE98F0@nemesis.central.ccp.cc>

Hello all.
I am working on updating the PCBuild8 directory in the trunk.  This should fix this issue, unify platform builds for win32, x64 and I64, and provide better support for PGO builds.
Hopefully when this is checked in, I can backport it to 2.5, since PCBuild8 is not an "official" platform for 2.5

Cheers,
Kristjan 

> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org 
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] 
> On Behalf Of Fredrik Lundh
> Sent: 31. ?g?st 2006 10:55
> To: python-dev at python.org
> Subject: Re: [Python-Dev] Error while building 2.5rc1 
> pythoncore_pgo on VC8
> 
> Muguntharaj Subramanian wrote:
> > Hi All,
> > I tried building python 2.5c1 using VC8.
> > 
> > Getting the following errors while building pythoncore_pgo:
> > 
> >    Creating library pythoncore_pgo/python25.lib and object 
> > pythoncore_pgo/python25.exp config.obj : error LNK2001: unresolved 
> > external symbol _init_types .\pythoncore_pgo/python25.dll : fatal 
> > error LNK1120: 1 unresolved externals
> > 
> > I checked pythoncore_pgo.vcproj file, it alreadu contains:
> > 
> > RelativePath="..\Modules\_typesmodule.c"
> > 
> > Can anyone give me some clue to solve this issue.
> 
> googling for error messages is always a good idea:
> 
> http://thread.gmane.org/gmane.comp.python.devel/83341
> 
> (see the last post in that thread for what I believe is a workaround)
> 
> </F>
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/kristjan%40c
cpgames.com
> 

From fredrik at pythonware.com  Thu Aug 31 13:58:21 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 31 Aug 2006 13:58:21 +0200
Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8
In-Reply-To: <8dd9fd0608310446o6008240x8bfa852b41595eab@mail.gmail.com>
References: <8dd9fd0608310336q45d2d3d3re203e871c7b384b8@mail.gmail.com>	<ed6f6h$la4$1@sea.gmane.org>
	<8dd9fd0608310446o6008240x8bfa852b41595eab@mail.gmail.com>
Message-ID: <ed6ist$28v$1@sea.gmane.org>

Muguntharaj Subramanian wrote:

> That error mentioned in that post was in "pythoncore" module.
> My error is while compiling "pythoncore_pgo" module.

iirc, that's a partially experimental alternative build for playing with 
performance guided optimizations.  are you sure you need that module ?

</F>


From kristjan at ccpgames.com  Thu Aug 31 14:03:25 2006
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_V=2E_J=F3nsson?=)
Date: Thu, 31 Aug 2006 12:03:25 -0000
Subject: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8
Message-ID: <129CEF95A523704B9D46959C922A280002FE98F1@nemesis.central.ccp.cc>

I see.  There is a file, called pythoncore_pgo_link.txt where you have to add the object too.
 
But pythoncore_pgo is a bit broken in other ways at the moment.  I am working on making it better in the trunk.
(I also think that MS should improve their tools to make PGO building a one step process.  the current
recommended solution involves two project configurations...)
 
Kristj?n


________________________________

	From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Muguntharaj Subramanian
	Sent: 31. ?g?st 2006 11:46
	To: Fredrik Lundh
	Cc: python-dev at python.org
	Subject: Re: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8
	
	


		> I checked pythoncore_pgo.vcproj file, it alreadu contains:
		>
		> RelativePath="..\Modules\_typesmodule.c" 
		>
		> Can anyone give me some clue to solve this issue.
		
		googling for error messages is always a good idea:
		
		http://thread.gmane.org/gmane.comp.python.devel/83341 
		
		(see the last post in that thread for what I believe is a workaround)
		
		</F>
		



	Yes, I have already done a bit of searching before posting that message.
	I have seen that previous post.  
	
	That error mentioned in that post was in "pythoncore" module.
	My error is while compiling "pythoncore_pgo" module.
	And the workaround solution mentioned there is already there in the pythoncore_pgo.vcproj file. Thats why i mentioned RelativePath="..\Modules\ 

		_typesmodule.c" is already there.


	 
	I am still getting the same error. I am not able to find the cause yet. Any clue regarding this. 
	
	Thanks & Regards,
	
	Mugunth
	
	
	
	http://mugunth.blogspot.com
	http://thamizha.com
	http://tamilblogs.com 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060831/217a8a24/attachment.htm 

From ncoghlan at gmail.com  Thu Aug 31 14:08:12 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 31 Aug 2006 22:08:12 +1000
Subject: [Python-Dev] Py2.5 issue: decimal context manager
 misimplemented, misdesigned, and misdocumented
In-Reply-To: <44F6B524.6060504@gmail.com>
References: <44F4D9D2.2040804@ewtllc.com> <44F6B524.6060504@gmail.com>
Message-ID: <44F6D12C.4040808@gmail.com>

Nick Coghlan wrote:
> Raymond Hettinger wrote:
>> The right way to do it was presented in PEP343.  The implementation was 
>> correct and the API was simple.
> 
> Raymond's persuaded me that he's right on the API part at the very least. The 
> current API was a mechanical replacement of the initial __context__ based API 
> with a normal method, whereas I should have reverted back to the module-level 
> localcontext() function from PEP343 and thrown the method on Context objects 
> away entirely.
> 
> I can fix it on the trunk (and add those missing tests!), but I'll need 
> Anthony and/or Neal's permission to backport it and remove the get_manager() 
> method from Python 2.5 before we get stuck with it forever.

I committed this fix as 51664 on the trunk (although the docstrings are still 
example free because doctest doesn't understand __future__ statements).

Anthony, Neal: I'd like to backport this change to the 2.5 maintenance branch. 
I realise it is an API change between the release candidate and the actual 
release, but this really is a small tweak to something nobody is actually 
using yet.

If that's not acceptable, I'd like to go with Raymond's original option: rip 
it out entirely for 2.5 so we don't get stuck maintaining it for the rest of 
the 2.x series.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From skip at pobox.com  Thu Aug 31 16:06:28 2006
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 31 Aug 2006 09:06:28 -0500
Subject: [Python-Dev] link to community buildbot?
Message-ID: <17654.60644.37459.683994@montanaro.dyndns.org>


There was a buildbot set up recently to test various external packages
(like twisted) against Python's source code.  I don't recall where it is,
but it would be nice if relevant links to it existed on this page:

    http://www.python.org/dev/buildbot/

In addition, if a 2.3.6 release is going to happen it seems like it would be
a good idea to set up the relevant bits on both buildbots.

Skip

From alan.mcintyre at gmail.com  Thu Aug 31 16:22:30 2006
From: alan.mcintyre at gmail.com (Alan McIntyre)
Date: Thu, 31 Aug 2006 10:22:30 -0400
Subject: [Python-Dev] Windows build slave downtime
Message-ID: <1d36917a0608310722i49cb9404ic5e90fe8248945bf@mail.gmail.com>

Hi all,

The Windows build slave ("x86 XP trunk") will be down for most of
today while I move to a new apartment.  As long as the phone company
manages to provide me with an internet connection by the end of today,
it should be available again some time this evening.

Alan

From grig.gheorghiu at gmail.com  Thu Aug 31 16:34:37 2006
From: grig.gheorghiu at gmail.com (Grig Gheorghiu)
Date: Thu, 31 Aug 2006 07:34:37 -0700
Subject: [Python-Dev] link to community buildbot?
In-Reply-To: <17654.60644.37459.683994@montanaro.dyndns.org>
References: <17654.60644.37459.683994@montanaro.dyndns.org>
Message-ID: <3f09d5a00608310734k7c391fdbj8b26e9a08bb5bb7b@mail.gmail.com>

On 8/31/06, skip at pobox.com <skip at pobox.com> wrote:
>
>
> There was a buildbot set up recently to test various external packages
> (like twisted) against Python's source code.  I don't recall where it is,
> but it would be nice if relevant links to it existed on this page:
>
>     http://www.python.org/dev/buildbot/
>
> In addition, if a 2.3.6 release is going to happen it seems like it would
> be
> a good idea to set up the relevant bits on both buildbots.
>
> Skip
> _______________________________________________



The link is this one:

http://www.python.org/dev/buildbot/community/all/

BTW, I haven't had many takers since I announced this project....So if
anybody on these lists is interested in setting up a buildslave, please let
me know. There's also a mailing list here:

http://lists2.idyll.org/listinfo/pybots

Thanks,

Grig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060831/ea00240d/attachment.html 

From rhettinger at ewtllc.com  Thu Aug 31 19:01:16 2006
From: rhettinger at ewtllc.com (Raymond Hettinger)
Date: Thu, 31 Aug 2006 10:01:16 -0700
Subject: [Python-Dev] Py2.5 issue: decimal context manager
 misimplemented, misdesigned, and misdocumented
In-Reply-To: <44F6B524.6060504@gmail.com>
References: <44F4D9D2.2040804@ewtllc.com> <44F6B524.6060504@gmail.com>
Message-ID: <44F715DC.1090001@ewtllc.com>

Nick Coghlan wrote:

> Raymond Hettinger wrote:
>
>> The right way to do it was presented in PEP343.  The implementation 
>> was correct and the API was simple.
>
>
> Raymond's persuaded me that he's right on the API part at the very 
> least. The current API was a mechanical replacement of the initial 
> __context__ based API with a normal method, whereas I should have 
> reverted back to the module-level localcontext() function from PEP343 
> and thrown the method on Context objects away entirely.
>
> I can fix it on the trunk (and add those missing tests!), but I'll 
> need Anthony and/or Neal's permission to backport it and remove the 
> get_manager() method from Python 2.5 before we get stuck with it forever.
>
Please go ahead and get the patch together for localcontext().  This 
should be an easy sell:

* simple bugs can be fixed in Py2.5.1 but API mistakes are forever. 
* currently, all of the docs, docstrings, and whatsnew are incorrect.
* the solution has already been worked-out in PEP343 -- it's nothing new.
* nothing else, anywhere depends on this code -- it is as safe a change 
as we could hope for.

Neal is tough, but he's not heartless ;-)


Raymond




From johann at rocholl.net  Thu Aug 31 21:55:23 2006
From: johann at rocholl.net (Johann C. Rocholl)
Date: Thu, 31 Aug 2006 21:55:23 +0200
Subject: [Python-Dev] That library reference, yet again
Message-ID: <8233478f0608311255o7058a1feo55c710e7eb8e6b6c@mail.gmail.com>

What is the status of http://effbot.org/lib/ ?
I think it's a step in the right direction. Is it still in progress?
Can I have access to toolchain source code and/or conversion output?

I would like to set up something like gotapi.com for the Python
library reference, or help the gotapi people to include it. But gotapi
needs link targets for functions etc., but the original Python library
reference doesn't have them.

I really like the simple linking idea here:
http://online.effbot.org/2006_01_01_archive.htm#link-doc

Here's another working example with (ugly) link targets:
http://www.zvon.org/other/python/PHP/search.php

Cheers,
Johann

From jsceballos at gmail.com  Thu Aug 31 22:18:04 2006
From: jsceballos at gmail.com (Javier Sanz)
Date: Thu, 31 Aug 2006 22:18:04 +0200
Subject: [Python-Dev] That library reference, yet again
In-Reply-To: <8233478f0608311255o7058a1feo55c710e7eb8e6b6c@mail.gmail.com>
References: <8233478f0608311255o7058a1feo55c710e7eb8e6b6c@mail.gmail.com>
Message-ID: <64c763bf0608311318n11afd9a7s6943c42905d16cad@mail.gmail.com>

+1 to that idea



-- 
Un saludo,

    Javier